Why this matters
- PolicyDesk's first month of bugs clustered into ten categories. None of them were "the transformer architecture is wrong."
- Interviewers ask for a failure and a mitigation, in pairs. A list of fears without a fix is not an answer.
- Cost and latency show up as product bugs: users retry, you pay twice, and the second answer contradicts the first.
- Security and privacy are retrieval bugs as much as they are model bugs. The chunk you should not have fetched is already a leak.
Poor retrieval
The retrieved chunks are about the wrong topic, or the right topic in the wrong document.
Why. Weak embeddings for your jargon, chunks that mix topics, no keyword index for acronyms, k too small, or no metadata filter so every department competes.
What to do. Use a strong embedding model only after chunks are clean. Add hybrid BM25. Overlap chunks so rules stay intact. Store department, product, and version and filter on them. Re-rank the top few dozen. Read failed queries weekly and add the missing synonym to expansion or to the document itself (a glossary line: "CL means casual leave").
Hallucinations that survive retrieval
The right paragraph is in the prompt and the answer still adds a rule.
Why. The prompt says "be helpful." Temperature is high. The model prefers a complete answer to a refusal. Two chunks contradict each other and the model blends them.
What to do. A strict prompt: only the context, and say you do not know. Ask for citations that must match a provided source id, and drop sentences whose citation does not. Lower temperature. If chunks conflict, prefer the newest version metadata or ask the user to clarify. Run a faithfulness check (a second call or a rubric) on high-stakes answers. A smaller, more obedient model sometimes beats a larger, more inventive one on this metric. Measure it.
Documents that do not fit
A 300-page handbook or a decade of tickets will not fit in a prompt, and a naive split will create thousands of near-duplicate windows.
Why. Context windows are finite, attention gets worse as the prompt grows, and cost is linear in tokens.
What to do. Hierarchical retrieval: chapter summary first, chunks second. Summarize very long sections into a retrieval-oriented abstract, and keep the original for the final quote. Do not raise k until the prompt is a novel. For multi-hop questions, retrieve in steps rather than pasting the universe.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 7 sections
- Many formats
- Latency
- Scale
- Security and privacy
- Stale knowledge
- Cost
- You cannot fix what you do not measure
Test yourself
Answer these before moving on — recall is what makes it stick.