Why this matters
- "How many paid leaves are allowed?" failed vector search until the query was expanded with "privilege leave, casual leave, PTO."
- A second-stage re-ranker moved the true handbook page from position 7 to position 1, which is the difference between a correct answer and a shrug.
- These techniques are how you talk about relevance in a senior interview after you have already explained BM25 and cosine.
- Each one costs latency. You add them when the eval set says the gold chunk is near the top but not at the top, or is missing because of wording.
Query expansion
Users do not speak in handbook headings. Expansion adds synonyms and related phrases, then searches with the richer query.
Original: "What is leave policy?"
Expanded: "leave policy, paid leave, vacation, casual leave, sick leave, privilege leave, PTO."
You can expand with a curated synonym list (precise, boring, stable) or with an LLM call (flexible, and sometimes it adds a phrase that pulls in the wrong policy). Prefer the list for a closed HR domain. Prefer the LLM when the corpus is broad and you can check the expansions.
from langchain_openai import ChatOpenAI
from langchain_core.prompts import PromptTemplate
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
prompt = PromptTemplate.from_template(
"Generate 5 search phrases for this question. "
"Return only the phrases, one per line.\nQuestion: {question}"
)
expansions = llm.invoke(prompt.format(question="What is the leave policy?")).content
Search once with the original and once with each phrase, or concatenate them into one lexical query. Deduplicate chunk ids before you build the prompt. Five searches that return the same page should not paste that page five times.
Multi-query retrieval
Instead of synonyms, ask the model for several different questions that cover other angles of the same need.
"What is our leave policy?" might become:
- "How many casual leave days do employees get?"
- "What is privilege leave and who is eligible?"
- "Are unused leaves carried forward?"
Retrieve for each, union the chunks, drop duplicates. This helps when one wording hits the entitlement table and another wording hits the carry-forward exception. It hurts when the side questions wander into a different policy and you union the junk in. Cap the number of queries (three is plenty) and cap the total chunks.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 6 sections
- Step-back prompting
- Hypothetical document embeddings (HyDE)
- Metadata filtering
- Re-ranking
- Maximal marginal relevance (MMR)
- A production retrieval stack
Test yourself
Answer these before moving on — recall is what makes it stick.