The Search Problem: Why Keyword Search Fails for Semantic Queries

Traditional search — the kind that powers a Ctrl+F find-in-page, or a basic database LIKE query — works by matching LITERAL text. Search for 'cheap flights' and it looks for the exact words 'cheap' and 'flights' appearing in documents. This is fast and works fine when the searcher happens to use the same words the document uses. But it breaks down the moment they don't.

Consider a document that says 'affordable airfare' and a user who searches 'cheap flights.' To a human, these obviously mean almost the same thing. To keyword search, they share ZERO words in common, so the document simply never gets found — a complete miss, even though it's exactly what the user wanted. This is called the vocabulary mismatch problem, and it's pervasive: synonyms ('car' vs 'automobile'), paraphrases ('how do I reset my password' vs 'forgot login credentials'), different levels of specificity, and simple phrasing differences all defeat literal keyword matching, even when the underlying MEANING lines up perfectly.

Keyword search also has the opposite failure: it can match documents that share words but mean something completely different. Search 'apple' looking for the fruit and you might get results about the tech company — the word matches perfectly, but the meaning doesn't. Keyword search has no concept of meaning at all; it's purely mechanical string matching (sometimes with light stemming, like matching 'running' to 'run').

Vector search (also called semantic search) solves both problems at once, using exactly the embeddings from the previous unit. Instead of matching literal words, it embeds the query into the same vector space as the documents, then finds documents whose embeddings land CLOSEST to the query's embedding — 'affordable airfare' and 'cheap flights' end up near each other in embedding space because they mean similar things, even though they share no words. This is precisely why RAG systems (which need to find genuinely RELEVANT context for an LLM, not just keyword-matching context) rely on vector search rather than traditional keyword search: an LLM's usefulness depends on retrieving documents that are actually relevant in MEANING, and meaning is exactly what vector search was built to find. (In practice, many production systems combine both — 'hybrid search' — since keyword search is still useful for exact matches like product IDs or names that vector search can sometimes blur together.)

# A minimal side-by-side comparison: keyword overlap vs a cosine-
# similarity-style semantic match, on a genuine vocabulary-mismatch case.

def keyword_match_score(query: str, document: str) -> float:
    """Fraction of query words that literally appear in the document."""
    q_words = set(query.lower().split())
    d_words = set(document.lower().split())
    if not q_words:
        return 0.0
    return len(q_words & d_words) / len(q_words)

# Toy 'semantic similarity' stand-in -- in a real system this comes
# from cosine similarity between embeddings (previous unit)
semantic_scores = {
    "Affordable airfare deals this month": 0.91,   # means the same thing
    "Apple unveils new iPhone": 0.05,               # unrelated meaning
}

query = "cheap flights"
for doc, sem_score in semantic_scores.items():
    kw_score = keyword_match_score(query, doc)
    print(f"{doc!r}")
    print(f"  keyword match:  {kw_score:.2f}  (literal word overlap)")
    print(f"  semantic match: {sem_score:.2f}  (meaning-based)")
# Keyword search completely misses the relevant document (0.00 overlap)
# that semantic/vector search correctly identifies as highly relevant