Vector databases in production: ingestion, indexes and quality

Lesson 7.1 · 6 min read · by Vinay C

We'll cover the following

Visual summary

Infographic summarizing Vector databases in production: ingestion, indexes and quality: Retrieval theory (chunking, hybrid search, reranking) is only half the job. The other half is operating the store:…

Download imageFree to share, please link back to this lesson.

Retrieval theory (chunking, hybrid search, reranking) is only half the job. The other half is operating the store: getting documents in reliably, keeping them fresh, choosing an index you can afford, and noticing when quality quietly drops.

The ingestion pipeline

source (S3, Drive, Jira, wiki)
  -> detect change (etag, modified time, webhook, CDC)
  -> parse and clean
  -> chunk
  -> embed (batched, rate limited)
  -> upsert with metadata
  -> verify (counts, sample queries)
  failures -> retry -> dead-letter queue
  • Idempotent upserts: give each chunk a deterministic id (document id plus chunk index or content hash). Re-running a job then overwrites instead of duplicating.
  • Store metadata with every chunk: document id and version, source, updated_at, chunk position, tenant and access-control fields. You cannot filter on what you did not store.
  • Incremental updates: re-process only changed documents. A full re-embed is slow and costs money.
  • Deletes are the part people forget. If a document is removed or a permission is revoked, its chunks must go too, or the assistant will keep quoting it. Propagate deletes explicitly and test them.
  • Failures: retry transient errors, send poison documents to a dead-letter queue, and alert on a growing backlog.

Changing the embedding model

Vectors from different models are not comparable, and dimensions often differ. You cannot mix them in one index. Build a new index version, backfill it, evaluate it on your golden queries, then switch traffic with an alias and keep the old index for rollback.

Index types and trade-offs

IndexHow it worksTrade-off
Flat (exact)Compare the query with every vector100% recall, slow at scale. Fine for small sets and as a recall baseline
HNSWA layered graph of neighboursHigh recall and low latency, large memory, slower builds. Tuned with M, efConstruction, efSearch
IVFCluster the vectors, search the nearest clusters (nprobe)Smaller and faster to build, recall depends on nprobe and training data
Quantization (PQ, int8)Compress vectors4x or more less memory, some recall loss

Memory maths: 10 million vectors of 1,536 float32 dimensions is 10M x 1,536 x 4 bytes, about 61 GB before index overhead. That is why dimension, quantization and sharding decide the bill.

Choosing a store

OptionGood forWatch out for
pgvector in PostgreSQLVectors next to relational data, transactions, one less systemScale limits and memory pressure on a shared database
OpenSearch or ElasticsearchKeyword plus vector (hybrid) in one place, filters, mature operationsCluster tuning and cost
Dedicated stores (Pinecone, Qdrant, Weaviate, Milvus)Large scale, managed scaling, rich filteringAnother system to run, pay for and secure
  • Filtering: post-filtering (search, then filter) can return too few results, and some indexes degrade when a filter is very selective. Test your real filters, especially permission filters.
  • Multi-tenancy: a namespace or collection per tenant isolates data and eases deletion. A metadata filter is cheaper but a bug leaks across tenants.

Measuring retrieval quality over time

  • Golden queries with labelled relevant chunks, run on a schedule. Track recall@k and MRR so you see a drop before users do.
  • ANN recall: compare approximate results with exact search on a sample. If HNSW recall is 85%, you are losing relevant chunks to the index itself.
  • Freshness lag: time from a source change to it being searchable.
  • Operational signals: p95 query latency, empty-result rate, ingestion backlog, index size growth.
Check yourselfA user deleted a confidential document yesterday, but the assistant still quotes it. What most likely went wrong?

The delete never propagated to the vector store, so its chunks (and possibly cached answers) remain. Fix by propagating deletes and permission revocations explicitly, keeping a tombstone or reconciliation job that compares source ids with indexed ids, and adding a test that deletes a document and queries for it.

Check yourselfYou switch to a better embedding model. Why not just re-embed new documents with it?

Vectors from different models live in different spaces, so queries embedded with one model cannot be compared with documents embedded by another. Build a new index version with the new model, backfill everything, evaluate on golden queries, switch with an alias, and keep the old index for rollback.

Summary: say it in the interview

  • "Deterministic ids and idempotent upserts, metadata and ACLs on every chunk, explicit deletes, a new index version for every embedding change, and golden-query recall tracked on a schedule."