Gnu Group Learning AcademyGnu Group ↗
AI Engineering Fundamentals · Module 05 of 12

Search by meaning, not only keywords

Learn how text can be represented numerically so similar ideas can be found even when they use different words. Then compare keyword, vector and hybrid retrieval and measure whether search actually finds the right evidence.

Engineering focus: Embeddings, semantic search and retrieval

In plain English

What problem are we solving?

Learn how text can be represented numerically so similar ideas can be found even when they use different words. Then compare keyword, vector and hybrid retrieval and measure whether search actually finds the right evidence.

The rest of this module introduces the formal terminology and production controls behind that idea. You are not expected to memorize the vocabulary before using it.

Running case · Atlas Assist

Carry the same support assistant forward: approved KB, source citations, explicit insufficient-evidence fallback and one bounded read-only asset lookup. Reuse prior artifacts so each module adds evidence to one production AI system.

Learning objectives

By the end of this module, you should be able to:

  • Explain what embeddings represent operationally and how similarity search is used.
  • Design chunks and metadata around retrieval units rather than arbitrary character counts.
  • Measure retrieval quality separately from generation quality.

Embeddings are representations for similarity

Embedding models map inputs to numeric vectors so semantically related items can be compared. They do not provide truth, permissions or freshness by themselves.

Chunking changes what can be retrieved

A chunk should be large enough to preserve the fact needed to answer, but small enough to avoid irrelevant context. Structure-aware splitting by headings, paragraphs or records often beats fixed-size chopping.

Metadata is part of retrieval

Source ID, title, timestamp, product, access label and version enable filtering, citation and freshness decisions. A vector database without useful metadata is difficult to govern.

Vector search is not always sufficient

Exact IDs, error codes, names and rare tokens may be better served by lexical search. Hybrid retrieval combines semantic and lexical evidence, then can rerank the candidate set.

Evaluate retrieval before blaming generation

Measure whether the correct evidence appears in top-k results. If retrieval recall is poor, changing the generation prompt cannot create missing evidence reliably.

Worked examples

Retrieval pipeline

document → parse → chunk → metadata → embedding/index
query → transform/filter → retrieve top-k → optional rerank → context
  • Indexing and query-time behaviour both affect evidence quality.

Two separate questions

Retrieval: Did we fetch the evidence needed?
Generation: Did the model use the fetched evidence correctly?
  • Separating these failure modes prevents random prompt tuning.
Try it · core lab · account-free

Lab 05 — Build and evaluate a tiny offline retriever

Goal: Implement a simple lexical baseline first so retrieval quality can be measured without a model dependency.

Before you start

  • No paid API or cloud account is required for the core lab.
  • Core labs use Python 3 and local files only.
  • Running case: you are engineering 'Atlas Assist', an internal support assistant over a small approved knowledge base with one read-only inventory tool.
  • Carry requirements, eval cases, threat assumptions and operational evidence forward across modules instead of starting over.
STEP 1

Create a small KB

cat > /tmp/atlas-kb.json <<'EOF'
[
 {"id":"kb-17","text":"Password reset requires service desk approval and identity verification."},
 {"id":"kb-22","text":"Database maintenance occurs Sundays at 02:00 UTC."},
 {"id":"kb-31","text":"Asset srv-104 belongs to the payments service."}
]
EOF
STEP 2

Implement a token-overlap baseline

python3 - <<'PY2'
import json,re
kb=json.load(open('/tmp/atlas-kb.json'))
def toks(s): return set(re.findall(r'[a-z0-9-]+',s.lower()))
def search(q):
 q=toks(q); return sorted(kb,key=lambda d:len(q&toks(d['text'])),reverse=True)
for q,expected in [('How do I reset a password?','kb-17'),('When is database maintenance?','kb-22')]:
 top=search(q)[0]['id']; print(q,'->',top); assert top==expected
print('PASS: baseline retrieval')
PY2
STEP 3

Record retrieval cases

Create at least five query→expected-source cases, including one exact identifier and one paraphrase. Later compare lexical and embedding retrieval on the same cases.

Verify

  • Retrieval is tested independently of generation.
  • The dataset includes paraphrase and exact-token cases.

Expected outcome

  • A measurable lexical baseline and a retrieval evaluation set.

Break / fix · diagnose

  • If top-k is always increased to hide misses, inspect chunking/query design instead.
  • If access labels are absent from metadata, retrieval cannot enforce document-level visibility.
Production view

A strong RAG system starts with measurable information retrieval, not with a vector database brand.

Local real-tool practicum · optional but recommended

Optional practicum — embeddings with Ollama

Generate normalized embeddings and compare semantic similarity for the same small KB.

Prerequisites

  • Ollama installed.
  • An embedding model such as embeddinggemma, qwen3-embedding or another approved embedding model is already available, or you explicitly choose to download one.

Resources created

  • Only local disposable files/processes used by the practicum.

Cost / risk

  • No cloud account required; model downloads consume disk and network bandwidth.
TOOL 1
EMBED_MODEL=${EMBED_MODEL:-embeddinggemma}; ollama list | head
TOOL 2
EMBED_MODEL=${EMBED_MODEL:-embeddinggemma}; curl -s http://127.0.0.1:11434/api/embed -H 'Content-Type: application/json' -d "{\"model\":\"$EMBED_MODEL\",\"input\":[\"Password reset requires approval\",\"How can I change my password?\"]}" > /tmp/embed.json
TOOL 3
python3 - <<'PY2'
import json,math
x=json.load(open('/tmp/embed.json'))['embeddings']
a,b=x[0],x[1]
dot=sum(i*j for i,j in zip(a,b)); print('cosine/dot for normalized vectors:',round(dot,4),'dims',len(a))
PY2

Verify

  • Two embeddings are returned with matching dimensions and a computed similarity.

Cleanup

  • Remove only the disposable files created for the lab; keep model caches only if you intend to reuse them.

Warnings

  • Use the same embedding model for indexing and querying unless you have validated cross-model compatibility.
Avoid these traps

Common mistakes

  • Treating embeddings as facts.
  • Using fixed chunks without considering document structure.
  • Ignoring exact-token retrieval needs.
  • Evaluating only end-to-end answers and never retrieval recall.
Knowledge check

Can you connect the AI behaviour to a system control?

Concept and scenario checks. Reveal the answer only after you commit to a decision.

1What does an embedding provide?

A numeric representation useful for similarity; it does not provide truth or authorization.

2Why is metadata important?

It enables filtering, access control inputs, freshness/version reasoning, citation and source attribution.

3When can lexical search beat vector search?

For exact identifiers, rare terms, error codes and other token-sensitive queries.

4Scenario: the answer is wrong because the correct document never appeared in top-k. Which subsystem should be fixed first?

Retrieval/indexing/query design, because generation cannot reliably use evidence it never received.

My Learning

Record your progress for this module.

Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.

Not started

Attempt the core lab and knowledge checks before marking the module complete.