Why this matters
Choosing the right embedding model determines how well your system retrieves relevant text. In real NLP Engineer work, this impacts:
- Semantic search quality for support tickets, docs, and FAQs
- RAG (Retrieval-Augmented Generation) grounding accuracy
- Latency, throughput, and infrastructure cost at scale
- Multilingual coverage and domain specificity
Real tasks you will do
- Pick a dense embedding model for product search and justify trade-offs
- Design a quick evaluation with Recall@k and MRR
- Estimate vector storage and decide embedding dimensionality
- Choose distance metric (cosine/dot/L2) and normalization
- Plan a safe rollout with A/B testing and shadow traffic
Concept explained simply
Embeddings turn text into vectors so similar meanings are close together. Different models learn different "notions of closeness" based on their training. Your job is to match the model’s strengths to your data and constraints.
Mental model
Think of models as lenses. Some lenses are great for short questions, some for long documents, some for multiple languages, and some for exact keywords. You choose the lens that shows the most relevant items clearly with acceptable speed and cost.
Key criteria and trade-offs
- Task fit: retrieval vs classification vs clustering; prefer retrieval-trained models for search
- Language coverage: monolingual vs multilingual; check your top languages
- Domain match: general vs domain-tuned (legal, code, finance, healthcare)
- Input length: max tokens, chunking strategy, and whether model handles long contexts
- Dimensionality: higher dims may help but increase storage and latency; diminishing returns are common
- Similarity metric: cosine (with normalization) vs dot vs L2; be consistent across indexing and querying
- Latency and throughput: small/medium models often suffice; batch where possible
- Cost and licensing: inferencing cost, open weights vs paid APIs, redistribution terms
- Vector DB compatibility: supported distance metrics, HNSW/IVF settings, index build time
- Sparse vs dense: sparse (BM25/keyword) shines for exact terms; dense excels at semantics; hybrids often win
Cosine vs dot vs L2 (quick guide)
- If embeddings are L2-normalized, cosine similarity equals dot product ranking
- Cosine is scale-invariant; dot can be sensitive to vector norms
- L2 distance can work but is less common for normalized text embeddings
How to choose in 5 steps
- Define success: pick 1–2 primary metrics (e.g., Recall@10, MRR@10) and target latency
- Narrow candidates: 2–4 models that match language, domain, and context length
- Build a tiny eval set: 50–200 query→relevant doc pairs from your data
- Run a fast bake-off: same chunking, same index, compare metrics and latency
- Roll out safely: A/B with partial traffic, monitor relevance, clicks, and complaints
Worked examples
Example 1: Semantic search for support tickets (English)
Constraints: English only, queries are short, docs moderately long, latency budget 150 ms, cosine distance.
- Shortlist: small/medium English retrieval models with 384–768 dims
- Eval: Recall@10, MRR@10 on 150 labeled pairs
- Result: A 384-dim model scored within 1% of a 768-dim model but was 35% faster → choose 384-dim
Example 2: Multilingual FAQ (EN/ES/FR)
Constraints: 3 languages, mixed length, mobile latency important.
- Shortlist: multilingual retrieval models with ≤768 dims
- Eval per language: Recall@5; ensure no language regresses >5%
- Result: Model X had balanced scores across languages; slightly lower EN score but best overall → choose X
Example 3: Code search for Python repositories
Constraints: domain-specific (code), queries are natural language, docs are code snippets.
- Shortlist: models trained or adapted for code-text alignment
- Eval: Recall@10 on 200 NL→code pairs; test out-of-repo generalization
- Result: Code-aware model improved Recall@10 by 12 points over general text model → choose code-aware
Quick evaluation recipe
- Collect 100–200 query→positive pairs; add 5–10 hard negatives per query
- Chunk docs consistently (e.g., 300–500 tokens with 10–15% overlap)
- Index with the same distance metric you will use in prod
- Measure: Recall@k (k=5/10), MRR@k, and p95 latency
- Pick the model with the best metric-latency balance; confirm on a second random split
What good numbers look like
- Recall@10: aim for +5–10 points over baseline keyword search
- MRR@10: higher early precision; watch improvements on head queries
- Latency: ensure p95 fits your SLO; batch and cache where safe
Common mistakes and self-check
- Mixing metrics: training on cosine, querying with L2; fix by standardizing metric and normalization
- Noisy eval: unlabeled or inconsistent positives; fix by quick labeling pass with clear guidelines
- Overfitting to a tiny eval: verify on a second split or time-slice
- Ignoring chunking: wrong chunk size can mask model quality; tune chunking first
- Choosing maximum dimensions by default: check if smaller dims achieve near-identical recall
Self-check prompts
- Did I evaluate per segment (language/domain) not just overall?
- Are my negatives hard enough to stress the model?
- Do p95 and p99 latency meet the SLO, not just averages?
Practical projects
- Build a mini semantic search: index 5k docs with two candidate models, compare Recall@10 and latency
- Create a multilingual eval: 50 queries per language; visualize per-language recall and pick the winner
Exercises
Do these to practice. The quick test is available to everyone; only logged-in users get saved progress.
- Exercise 1: Compare two ranking outputs and compute Recall@3 and MRR@3
- Exercise 2: Estimate vector storage for your corpus under two dimensionalities
- Exercise 3: Draft a selection and rollout plan for multilingual support search
Exercise checklist
- I computed Recall@k and MRR correctly per query, then averaged
- I considered index overhead in storage estimates
- I wrote a rollout that includes shadow/A-B and success metrics
Who this is for
NLP Engineers, Data Scientists, and ML Engineers building search or RAG systems who need practical, fast ways to choose embeddings.
Prerequisites
- Basic Python or tooling to run embedding inference
- Familiarity with vector databases and similarity search
- Understanding of precision/recall and ranking metrics
Learning path
- Start: Embedding basics and distance metrics
- Then: Indexing, chunking, and hybrid search
- Next: Model selection and evaluation (this page)
- After: Reranking, domain adaptation, and monitoring
Next steps
- Run a 2–4 model bake-off on your data
- Adopt hybrid (dense+sparse) if exact terms matter
- Plan A/B rollout with guardrails on latency and relevance
Mini challenge
You have 1 hour to choose a model for a 3-language FAQ search. Prepare: your shortlist (2–3 models), metrics, a 100-pair eval design, and a safe rollout plan. Keep it concise and decision-focused.