Mental model
Module 05: Search by meaning, not only words- Understand embeddings conceptually first, then learn similarity, model identity and why embedding choice must be evaluated on your corpus.
RAG & Knowledge Systems Engineering · Module 05 of 18
Builder
Embeddings: Intuition to Engineering
Understand embeddings conceptually first, then learn similarity, model identity and why embedding choice must be evaluated on your corpus.
Engineering focus: Search by meaning, not only words
In plain English
What problem are we solving?
Understand embeddings conceptually first, then learn similarity, model identity and why embedding choice must be evaluated on your corpus.
You are not expected to know the engineering vocabulary before using the idea. Formal terminology comes after the mental model.
Running case · Evidence Desk
Improve the same system, module by module.
Carry one evidence-grounded knowledge assistant through all 18 modules. Preserve source IDs, retrieval evidence, evaluation cases, authority decisions and security assumptions so every improvement is measurable.
Mental model
Understand it before you optimize it.
An embedding turns meaning into coordinates so nearby vectors can be searched. Nearby does not mean true, current or authorized.
01Text
02Embedding model
03Vector
04Similarity
05Candidate ranking
Remember
Similarity is a retrieval signal, not a truth score.
Learning objectives
By the end of this module, you should be able to:
- Explain embeddings without requiring advanced mathematics.
- Describe cosine similarity conceptually.
- Record embedding model identity and dimensions as part of index provenance.
Meaning as coordinates
An embedding maps text to a numeric vector. Texts with similar meaning tend to be closer in the learned representation space.
Similarity is not truth
A high similarity score means the model thinks two texts are related; it does not prove factual correctness or authorization.
Model identity matters
Changing the embedding model changes vector geometry. Re-index when model identity or preprocessing changes.
Multilingual and domain fit
Embedding models differ by language, domain, size and speed. Evaluate with your own questions and documents.
Engineering depth
Engineering deep dive
- Pin embedding model name/version and preprocessing choices in index metadata.
- Never mix vector spaces produced by incompatible embedding models.
- Evaluate multilingual/domain behaviour with positive, hard-negative and near-duplicate pairs.
- Monitor vector dimensionality/index compatibility during migrations.
Worked examples
Ranking is not authority
query: "current VPN reset policy"
0.92 old-policy.md (superseded)
0.88 current-policy.md (active)
Semantic rank says old-policy is close.
Authority metadata says current-policy is eligible.- Similarity answers relevance, not policy status.
- Production ranking often combines multiple signals.
Diagnostic lens
If this goes wrong, where do you look first?
Symptom
Semantically odd documents rank highly.
Likely causes
- Embedding model is a poor domain fit
- Vectors from different models were mixed
- Similarity is being over-interpreted
First checks
- Record embedding model/version
- Compare known positive and negative pairs
- Inspect raw retrieved text
Engineering principle
Similarity must be validated on your corpus.
Try it · core lab · account-free
Lab 05 — Embeddings: Intuition to Engineering
Goal: Compute cosine similarity on tiny hand-written vectors, then explain what the score does and does not mean.
Before you start
- Core lab requires only Python 3 and local files unless otherwise stated.
- Running case: build the same evidence-grounded knowledge assistant across the course.
- Keep source IDs and intermediate evidence so each result is inspectable.
- Reflection: Write one sentence describing what this step proves and one sentence describing what it does not prove.
Run the baseline
python3 - <<'PY2'
from math import sqrt
def cos(a,b): return sum(x*y for x,y in zip(a,b))/(sqrt(sum(x*x for x in a))*sqrt(sum(y*y for y in b)))
print('similar=',round(cos([1,1,0],[.9,.8,.1]),3)); print('different=',round(cos([1,1,0],[0,0,1]),3))
PY2Inspect the evidence
Record the intermediate source IDs, ranks, metadata or state produced by the baseline. Explain which pipeline layer this evidence belongs to.
Prove similarity is not truth
Create one highly similar vector labelled superseded and one slightly lower-scoring vector labelled active. Explain why authority must remain a separate signal.
Diagnose and restore
Similarity must be validated on your corpus. State the failing layer, change only that layer, then repeat the verification checks.
Write the engineering note
Record: hypothesis → evidence → change → observed result → remaining limitation. Keep this note with the running Evidence Desk project.
Verify
- The similar pair scores higher than the unrelated pair.
- You can explain that semantic similarity is a ranking signal, not evidence authority.
- The baseline and the deliberately broken case produce different, explainable evidence.
- You can name the pipeline layer that owns the failure and the control that repairs it.
Expected outcome
- A reproducible local result plus a short evidence note.
- A short before/after record showing the failure, diagnosis and repaired state.
Break / fix · diagnose
- Inspect the intermediate evidence before changing the model or prompt.
- If a command fails, preserve the error and classify which pipeline layer failed.
- Change one pipeline variable at a time and rerun the same test case.
- Do not accept a plausible answer as proof; verify the intermediate evidence and source identity.
Production view
Production RAG improves when every change can be traced to evidence at a specific pipeline layer. In production, retain the before/after evidence so the same failure can become a regression test.
Production checklist
What would an engineer verify before shipping this layer?
Embedding model/version pinned
No mixed vector spaces
Domain/language fit tested
Avoid these traps
Common beginner and engineering mistakes
- Treating similarity score as confidence of factual truth.
- Mixing vectors from different embedding models in one index.
- Choosing an embedding model only from a leaderboard.
Knowledge check
Can you explain the idea and diagnose the system?
1What is an embedding?
A numeric representation used to compare semantic similarity.
2Does high cosine similarity prove a statement is correct?
No. It is a relevance signal, not factual verification.
3Why record embedding model identity?
Because changing the model changes vectors and retrieval behaviour.
4When should you re-index?
When the embedding model or important preprocessing changes.
My Learning
Record your progress for this module.
Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.
Attempt the core lab and knowledge checks before marking the module complete.