1 Core Concepts
Embeddings
Dense numerical vectors (e.g., 768 or 1536 dimensions) that represent semantic meaning of text, images, or audio. Generated by models like OpenAI text-embedding-3-small, Cohere embed-v4, or open-source all-MiniLM-L6-v2.
Similarity Search (ANN)
Approximate Nearest Neighbor search finds the most semantically similar vectors without scanning every record. Distance metrics include Cosine Similarity, Euclidean (L2), and Dot Product.
HNSW Index
Hierarchical Navigable Small World — the dominant indexing algorithm. Builds a multi-layer graph of vectors for O(log n) search. Tunable via ef_construction (build quality) and M (max connections per node).
IVFFlat Index
Inverted File with Flat compression. Clusters vectors into nlist partitions using k-means, then searches only the closest nprobe clusters. Faster build time than HNSW, lower memory usage, but slightly lower recall.
Metadata Filtering
Attach structured key-value metadata (JSON payloads) to vectors and apply filters before or after ANN search. Critical for multi-tenant RAG, date-range filtering, and access-controlled agent memory.
Collections & Sharding
A collection is a named group of vectors with the same dimensionality and distance metric. Large collections are sharded across nodes for horizontal scaling. Each shard holds a partition of the index.
Hybrid Search
Combines dense vector similarity with sparse keyword matching (BM25). Fuses scores via Reciprocal Rank Fusion (RRF) or linear weighting. Essential for production RAG where keyword precision matters alongside semantic recall.
RAG Pipeline
Retrieval-Augmented Generation: Query embeddings retrieve relevant context chunks from a vector store, which are injected into the LLM prompt. The database serves as the agent's long-term memory layer.
2 Database Comparison
3 Setup Guide
Qdrant (Docker)
Milvus (Docker Compose)
pgvector (PostgreSQL)
4 Usage Examples
Qdrant — Python RAG Pipeline
pgvector — SQL-Native Similarity Search
5 Best Practices
Chunk Strategy
Split documents into 256–512 token chunks with 10–20% overlap. Use semantic chunking (by paragraph or section) over fixed-size splitting. Smaller chunks improve retrieval precision; larger chunks preserve context.
Embedding Model Selection
Benchmark on your domain data using MTEB. text-embedding-3-large (3072d) for max quality, all-MiniLM-L6-v2 (384d) for low-latency self-hosted. Match dimensionality to your performance budget.
Index Tuning
HNSW: set ef_construction=128 for high-quality builds. At query time, tune ef_search — higher values (200+) increase recall at the cost of latency. Profile recall@10 vs. p99 latency for your SLA.
Payload Indexing
Create indexed payload fields for any metadata you filter on. Without payload indexes, Qdrant/Milvus fall back to brute-force post-filtering, which destroys performance on large collections.
Multi-Tenancy
Use tenant_id as an indexed payload field rather than separate collections per tenant. Single-collection multi-tenancy with payload filtering is more memory-efficient and simpler to manage.
Quantization
Enable scalar (int8) or binary quantization to reduce memory by 4–32×. Binary quantization + rescoring achieves 95%+ recall while cutting RAM usage and increasing throughput significantly.
6 Scaling & Production
Collection Sharding
Distribute a collection across multiple shards on different nodes. Qdrant and Milvus both support automatic sharding. Set shard count at creation — more shards = more write throughput but higher query fan-out latency.
Replication for HA
Set replication_factor ≥ 2 so each shard has replicas on separate nodes. Qdrant uses Raft consensus for leader election. Milvus replicates via its message queue (Pulsar/Kafka). Replicas serve read queries for horizontal read scaling.
Billion-Vector Strategy
For 1B+ vectors: use binary quantization (32× RAM reduction) + disk-based HNSW (Qdrant) or DiskANN (Milvus). Only load frequently accessed segments into RAM. Use segment-level caching for hot data.
Write Throughput
Batch upserts in chunks of 100–1000 points for optimal throughput. Use async clients. Disable indexing during bulk import (indexing_threshold=0 in Qdrant), then rebuild. Milvus supports bulk insert via Parquet files.
pgvector Scaling Limits
pgvector is single-node — scale via PostgreSQL read replicas for read throughput and partitioning by tenant for multi-tenant workloads. Practical limit: ~10M vectors per table with HNSW. Beyond that, migrate to a purpose-built vector DB.
Monitoring at Scale
Track p99 search latency, recall@K, indexing lag, and memory/disk usage per shard. Set alerts when search latency exceeds SLA (e.g., 50ms p99). Monitor segment merge operations that temporarily increase latency.