Over the past few days, I’ve been spending time reading about embedding models—not just which ones rank highest, but how they actually behave in real systems like semantic search and RAG.
What I found changed how I think about embeddings. Turns out, most problems people blame on “the model” usually come from everything around it.
Here are the biggest takeaways.
1. Basics - What are Embeddings Models
An embedding model is a type of AI model that turns things like text, images, or audio into numbers (called vectors) in a way that captures their meaning.
Think of it like this:
- Words, sentences, or documents are hard for computers to compare.
- Numbers are easy for computers to compare.
- An embedding model translates meaning into numbers.
Simple example
If you give the model these sentences:
- “I love dogs”
- “I like puppies”
- “The stock market crashed”
The first two will be turned into number sets that are close together, because they mean similar things. The last one will be far away, because it means something different.
What embeddings are used for
Embedding models help with:
- Search (finding relevant results even if wording is different)
- Recommendations (“people who liked this also liked…”)
- Chatbots & Q&A (understanding what you mean)
- Clustering (grouping similar content)
One-line summary
👉 An embedding model converts content into meaningful numbers so a computer can understand similarity and context.
2. Leaderboards are a good starting point, But,..
Benchmarks like MTEB are useful, but they don’t tell the full story. Each model would be trained on a different dataset.
In practice:
- A top-ranked model can still perform poorly on your data
- Internal docs, tickets, or policies behave very differently from benchmark datasets
A simple rule that actually works: Use leaderboards to shortlist → test on your own data → then decide.
3. Normalization is optional, but recommended!
One of the most common mistakes I see is skipping vector normalization.
Why it matters:
- Without it, similarity scores get biased toward longer or “denser” text
- You end up comparing size instead of meaning
The simplest way to think about it:
Normalization removes “loudness” so similarity focuses on “direction” (meaning).
If you’re using cosine similarity, this step is foundational.
If you don't know what normalization is - I suggest you read about it, it is a must-know if you want to build high quality AI solutions.
4. Chunking matters more than the model itself
You don’t embed documents. You embed chunks. (duh!)
And bad chunking quietly kills retrieval quality:
- Chunks too large → meaning gets muddy
- Chunks too small → context disappears
- Bad splits → answers get cut in half
Even the best embedding model can’t fix this after the fact is embedded poorly. so, be cautious and spend time thinking more about this.
5. Fixed chunking works — semantic chunking works better
There are a few common ways people chunk text:
- Fixed-size chunks (easy, not great results always)
- Fixed-size with overlap (industry standard)
- Sentence or paragraph-based chunking (I need to explore more on results of this)
- Semantic chunking, where you split when topics change (best recommendation by experts)
The goal of semantic chunking is simple:
One chunk = one idea = one answerable unit
This alone can noticeably improve retrieval quality and reduce hallucinations in RAG systems. There are several approaches on how to perform semantic chunking, you can explore yourself and be amazed!
6. Embedding models don’t handle chunking — your pipeline does
Embedding models don’t care how you chunk. They assume you’ve done it sensibly.
What does matter:
- Reasonable chunk sizes
- Consistent preprocessing
- Matching formats at index time and query time
Chunking is a pipeline design choice, not a model feature. So this is a pre-requisite, and more of a foundational decision, poor choice here will lead to poor decisions.
Final takeaway
Picking a good embedding model matters—but how you prepare, chunk, normalize, and retrieve your data matters just as much (if not more).
Most “bad RAG” systems don’t fail only because of the model. They can fail because of data preparation and retrieval design.
Originally published on LinkedIn.