NoSQL Databases — DynamoDB, Firestore & Cosmos DB

Core NoSQL Concepts

Foundational concepts of distributed serverless database models.

Document vs. Key-Value

NoSQL uses flexible models. Document stores (Firestore/Cosmos) serialize records as hierarchical JSON objects. Key-Value stores (DynamoDB) store records as flat attributes queryable by a primary key.

Partitioning & Sharding

Scale horizontally by spreading data across multiple physical storage partitions. Routing nodes use a hash of the Partition Key to instantly locate the record without full-table scans.

Consistency Modes

NoSQL trades consistency for speed. Eventual consistency serves reads faster but might return stale data. Strong consistency guarantees the absolute latest write but increases latency.

Real-time Subscriptions

Listen to database changes in real-time. Firestore provides native SDK listeners. DynamoDB and Cosmos DB offer Change Streams that trigger Lambdas or Azure Functions on every insert/update.

Secondary Indexing

Querying on fields other than the partition key requires secondary indexes (GSIs in DynamoDB, automatic indexing in Firestore/Cosmos DB). Essential for complex agent task queries.

TTL & Cache Eviction

Time-to-Live automatically evicts records after a specific timestamp. Crucial for purging short-term session state, token caches, and execution scratchpads without costs.

Comparing NoSQL Options

Comparing AWS DynamoDB, GCP Firestore, and Azure Cosmos DB.

Database	Cloud	Model	Key Features	Best For
DynamoDB	AWS	Key-Value / Document	Single-digit ms latency, DAX cache accelerator, streams	High-throughput agent messaging & flat sessions
Firestore	GCP / Firebase	Document Hierarchy	Real-time listeners, vector embeddings search, serverless SDK	Interactive chat applications, collaborative workspaces
Cosmos DB	Azure	Multi-Model (Document, Graph, SQL)	Turnkey global replication, multi-master writes, vector indexing	Enterprise agent states across multiple geographical regions

Setup Guide

Get your database SDK dependencies installed and configured.

DynamoDB (AWS)

                        # Install boto3 SDK

                        pip install boto3

                        # Local testing with DynamoDB Local

                        docker run -d -p 8000:8000 \

                          amazon/dynamodb-local

                        # Create a table via CLI

                        aws dynamodb create-table \

                          --table-name agent-sessions \

                          --attribute-definitions \

                            AttributeName=id,AttributeType=S \

                          --key-schema AttributeName=id,KeyType=HASH \

                          --billing-mode PAY_PER_REQUEST

Firestore (GCP)

                        # Install google-cloud-firestore

                        pip install google-cloud-firestore

                        # Local testing with Emulator

                        gcloud beta emulators firestore start \

                          --host-port=127.0.0.1:8080

                        # Set emulator env variable

                        export FIRESTORE_EMULATOR_HOST="127.0.0.1:8080"

Cosmos DB (Azure)

                        # Install Azure Cosmos SDK

                        pip install azure-cosmos

                        # Create a Cosmos account

                        az cosmosdb create \

                          --name my-cosmos-account \

                          --resource-group agent-rg \

                          --locations regionName=eastus failoverPriority=0

                        # Get connection keys

                        az cosmosdb keys list \

                          --name my-cosmos-account \

                          --resource-group agent-rg

Usage Examples

Comparing write and read operations across the databases in Python.

AWS DynamoDB (boto3)

                        import boto3

                        db = boto3.resource('dynamodb')

                        table = db.Table('agent-sessions')

                        # Put item (writes/updates)

                        table.put_item(Item={

                          'id': 'session_123',

                          'state': 'thinking',

                          'ttl': 1784650000

                        })

                        # Get item

                        res = table.get_item(Key={

                          'id': 'session_123'

                        })

                        print(res.get('Item'))

GCP Firestore (google-cloud)

                        from google.cloud import
                        firestore

                        db = firestore.Client()

                        doc_ref = db.collection('sessions')

                          .document('session_123')

                        # Set item (writes/creates)

                        doc_ref.set({

                          'state': 'thinking',

                          'updated': firestore.SERVER_TIMESTAMP

                        })

                        # Get item

                        snapshot = doc_ref.get()

                        print(snapshot.to_dict())

Azure Cosmos DB (SQL API)

                        from azure.cosmos import
                        CosmosClient

                        client = CosmosClient(url, key)

                        db = client.get_database_client(

                          'agent-db'

                        )

                        container = db.get_container_client(

                          'sessions'

                        )

                        # Upsert item

                        container.upsert_item({

                          'id': 'session_123',

                          'state': 'thinking'

                        })

                        # Read item

                        res = container.read_item(

                          item='session_123',

                          partition_key='session_123'

                        )

                        print(res)

Best Practices

Essential practices for designing scalable cloud NoSQL storage.

🔑 Partition Key Strategy

• Avoid "hot partitions" by choosing high-cardinality keys like session_id
• Do not use dates or constant strings as partition keys
• Use composite keys (e.g. partition key + sort key) to query historical ranges
• Restrict scan operations; use explicit key queries instead

💡 Query Design

• Model data based on query patterns, not relational structures
• Design indexes (GSI/LSI) specifically for your application filter routes
• Avoid excessive indexes to save on write capacity costs
• Leverage projections to return only required attributes

💰 Cost Optimization

• Enable auto-scaling or on-demand provisioning for serverless
• Define a TTL attribute on sessions to auto-cleanup data
• Cache frequently accessed items in Redis to reduce reads
• Use batch APIs (BatchWriteItem, etc.) for bulk data inserts

Frequently Asked Questions

Which NoSQL database is best for AI agent memory?▶

Firestore is excellent for interactive interfaces due to native real-time listeners. DynamoDB is the standard for high-speed key-value reads/writes (single-digit ms latency) inside backend agent loops. Cosmos DB is best for enterprise deployments that require multi-region active-active writes.

Does DynamoDB support vector similarity search?▶

No, DynamoDB does not have a native vector search engine. For vector RAG tasks on AWS, you should couple DynamoDB with Amazon OpenSearch Service, pgvector on RDS, or store vectors directly in Bedrock Knowledge Bases. Firestore and Cosmos DB, however, do support native vector indexes.

What is Single-Table Design in DynamoDB?▶

It is a design pattern where all entity types (e.g. Users, Sessions, Log entries) are stored inside a single table. Entities are linked via generic keys (PK and SK) rather than separate schemas. This guarantees that all required view data can be fetched in a single request, optimizing read capacity.

How do I handle transactions in serverless NoSQL?▶

All three databases support multi-document ACID transactions: DynamoDB has TransactWriteItems, Firestore has transactions via the client, and Cosmos DB supports multi-document transactions using stored procedures written in JavaScript. However, transactions consume more capacity units and should be used sparingly.

What is the capacity model (RCU/WCU) in DynamoDB?▶

DynamoDB measures performance in Read Capacity Units (RCUs) (1 strong read/sec up to 4KB) and Write Capacity Units (WCUs) (1 write/sec up to 1KB). You can choose provisioned billing (fixed limit) or on-demand billing (pay-per-request). On-demand is recommended for serverless applications with variable traffic spikes.