Vector databases in production: ingestion, indexes and quality
We'll cover the following
Visual summary

Download imageFree to share, please link back to this lesson.
Retrieval theory (chunking, hybrid search, reranking) is only half the job. The other half is operating the store: getting documents in reliably, keeping them fresh, choosing an index you can afford, and noticing when quality quietly drops.
The ingestion pipeline
source (S3, Drive, Jira, wiki)
-> detect change (etag, modified time, webhook, CDC)
-> parse and clean
-> chunk
-> embed (batched, rate limited)
-> upsert with metadata
-> verify (counts, sample queries)
failures -> retry -> dead-letter queue- Idempotent upserts: give each chunk a deterministic id (document id plus chunk index or content hash). Re-running a job then overwrites instead of duplicating.
- Store metadata with every chunk: document id and version, source,
updated_at, chunk position, tenant and access-control fields. You cannot filter on what you did not store. - Incremental updates: re-process only changed documents. A full re-embed is slow and costs money.
- Deletes are the part people forget. If a document is removed or a permission is revoked, its chunks must go too, or the assistant will keep quoting it. Propagate deletes explicitly and test them.
- Failures: retry transient errors, send poison documents to a dead-letter queue, and alert on a growing backlog.
Changing the embedding model
Vectors from different models are not comparable, and dimensions often differ. You cannot mix them in one index. Build a new index version, backfill it, evaluate it on your golden queries, then switch traffic with an alias and keep the old index for rollback.
Index types and trade-offs
| Index | How it works | Trade-off |
|---|---|---|
| Flat (exact) | Compare the query with every vector | 100% recall, slow at scale. Fine for small sets and as a recall baseline |
| HNSW | A layered graph of neighbours | High recall and low latency, large memory, slower builds. Tuned with M, efConstruction, efSearch |
| IVF | Cluster the vectors, search the nearest clusters (nprobe) | Smaller and faster to build, recall depends on nprobe and training data |
| Quantization (PQ, int8) | Compress vectors | 4x or more less memory, some recall loss |
Memory maths: 10 million vectors of 1,536 float32 dimensions is 10M x 1,536 x 4 bytes, about 61 GB before index overhead. That is why dimension, quantization and sharding decide the bill.
Choosing a store
| Option | Good for | Watch out for |
|---|---|---|
| pgvector in PostgreSQL | Vectors next to relational data, transactions, one less system | Scale limits and memory pressure on a shared database |
| OpenSearch or Elasticsearch | Keyword plus vector (hybrid) in one place, filters, mature operations | Cluster tuning and cost |
| Dedicated stores (Pinecone, Qdrant, Weaviate, Milvus) | Large scale, managed scaling, rich filtering | Another system to run, pay for and secure |
- Filtering: post-filtering (search, then filter) can return too few results, and some indexes degrade when a filter is very selective. Test your real filters, especially permission filters.
- Multi-tenancy: a namespace or collection per tenant isolates data and eases deletion. A metadata filter is cheaper but a bug leaks across tenants.
Measuring retrieval quality over time
- Golden queries with labelled relevant chunks, run on a schedule. Track recall@k and MRR so you see a drop before users do.
- ANN recall: compare approximate results with exact search on a sample. If HNSW recall is 85%, you are losing relevant chunks to the index itself.
- Freshness lag: time from a source change to it being searchable.
- Operational signals: p95 query latency, empty-result rate, ingestion backlog, index size growth.
Check yourselfA user deleted a confidential document yesterday, but the assistant still quotes it. What most likely went wrong?
The delete never propagated to the vector store, so its chunks (and possibly cached answers) remain. Fix by propagating deletes and permission revocations explicitly, keeping a tombstone or reconciliation job that compares source ids with indexed ids, and adding a test that deletes a document and queries for it.
Check yourselfYou switch to a better embedding model. Why not just re-embed new documents with it?
Vectors from different models live in different spaces, so queries embedded with one model cannot be compared with documents embedded by another. Build a new index version with the new model, backfill everything, evaluate on golden queries, switch with an alias, and keep the old index for rollback.
Summary: say it in the interview
- "Deterministic ids and idempotent upserts, metadata and ACLs on every chunk, explicit deletes, a new index version for every embedding change, and golden-query recall tracked on a schedule."