Semantic Search vs Keyword Search: A Practical Migration Guide
The pitch for semantic search is compelling — understand meaning, not just matching terms — and for a knowledge base client struggling with users phrasing the same question a dozen different ways, it delivered a real jump in relevant results. But semantic search alone made a specific class of query worse: exact product codes, SKUs, and error codes, where users know precisely what string they're looking for and embedding similarity actively hurt.
The fix wasn't choosing one over the other, it was hybrid search — running keyword (BM25) and semantic search in parallel and merging results with a weighted reranker. On that same knowledge base, hybrid search improved top-3 relevance by 22% over semantic search alone, almost entirely by recovering the exact-match queries that pure embedding search had been getting wrong.
Migration itself needs a shadow period, not a cutover. We ran the new hybrid search alongside the existing keyword search in production for three weeks, logging both result sets without showing users the new one, before switching over. That shadow period caught a chunking issue — documents split mid-sentence — that was quietly degrading semantic relevance and would have been much harder to diagnose after go-live.
Embedding model choice affects migration cost more than teams expect, because switching embedding models later means re-embedding the entire corpus. We picked a model with a track record of stability and backward compatibility for one client specifically to avoid that cost, accepting slightly lower benchmark performance than the newest model available in exchange for not having to replan a reindex every few months.
Query analytics before migration told us more than any benchmark did. Pulling the actual distribution of query types from the old keyword search logs showed one client's queries were 70% exact lookups and only 30% conceptual questions — the opposite of what the team assumed going in — which changed our hybrid weighting significantly toward keyword relevance rather than the semantic-heavy default we'd started with.
