Chunking
We'll cover the following
Chunking decides what the retriever can possibly find. If the answer is cut in half across two chunks, no retriever can return it whole.
Size is a trade-off
| Chunk size | Upside | Downside |
|---|---|---|
| Too small | Precise match | Loses surrounding context; the answer gets split |
| Too large | Keeps context | The embedding is a blur of several topics, so relevance drops; wastes prompt tokens |
Fixes worth naming
- Overlap between neighbouring chunks, so a sentence on the boundary lives in both.
- Structure-aware splitting: split on headings, paragraphs or code blocks instead of a fixed character count.
- Parent-child chunks: search over small child chunks for precision, then return the bigger parent for context.
Check yourselfWhat is parent-child chunking?
You embed and search small child chunks, because they match precisely. When one matches, you return its larger parent chunk to the LLM, because the parent carries the context needed to answer.
Summary: say it in the interview
- "I would not use a fixed size blindly. I split on document structure, add some overlap, and tune the size against recall@k on a test set."