Why this matters
- PolicyDesk's naive pipeline answers "how many casual leaves?" and falls over on "compare casual leave for probationary staff in India versus the UK handbook, and cite both."
- Interviewers want you to pick an architecture from the question shape, not to recite a catalog.
- Graph and hierarchical designs show up when the corpus is a web of entities or a deep manual, not a flat FAQ.
- Self-RAG and agents add loops. Loops add latency, cost, and new ways to wander. You should say when you refuse them.
Start from the question, not the diagram
| Question shape | Architecture that fits |
|---|---|
| One fact in one paragraph | Naive RAG |
| A stable product with separate index, retrieve, and generate services | Modular RAG |
| The system must decide whether to search, calculate, or ask a follow-up | Agentic RAG |
| The answer is a relationship: who owns this policy, which form it requires | Graph RAG |
| A long manual with chapters, sections, and clauses | Hierarchical RAG |
| Text plus screenshots or diagrams | Multi-modal RAG |
| Sometimes the model's own knowledge is enough and search would add noise | Adaptive or self-RAG |
| Acronyms and paraphrases in the same corpus | Hybrid retrieval inside whichever architecture you chose |
One fact in one paragraph
Architecture that fitsNaive RAGA stable product with separate index, retrieve, and generate services
Architecture that fitsModular RAGThe system must decide whether to search, calculate, or ask a follow-up
Architecture that fitsAgentic RAGThe answer is a relationship: who owns this policy, which form it requires
Architecture that fitsGraph RAGA long manual with chapters, sections, and clauses
Architecture that fitsHierarchical RAGText plus screenshots or diagrams
Architecture that fitsMulti-modal RAGSometimes the model's own knowledge is enough and search would add noise
Architecture that fitsAdaptive or self-RAGAcronyms and paraphrases in the same corpus
Architecture that fitsHybrid retrieval inside whichever architecture you chose
Naive RAG
Naive RAG is the pipeline you already built. Index chunks. On a question, embed it, take top-k, stuff the prompt, generate once.
It is easy to debug and cheap. It fails when the first retrieval is wrong, when the answer needs two documents that do not look like each other, or when a tool (SQL, calculator, web) is required. Do not abandon it because a conference talk used a more elaborate picture.
Modular RAG
Modular RAG keeps the same stages and makes them replaceable services: ingestion, embedding, retrieval, prompt building, generation, and evaluation. You can put a re-ranker in front of the prompt builder without rewriting ingestion. You can shadow a new embedding model on a copy of the index.
This is an engineering architecture more than a retrieval trick. PolicyDesk wants it as soon as two people are changing the system. The failure mode is distributed-systems theater: five services and no eval set. Modularity pays when you can still trace one request across the boxes with a shared id.
Modules worth separating
- Ingestion and parsing, because PDF bugs should not require a model deploy.
- Embedding and the vector store, because re-indexing is a batch job.
- Retrieval, including filters and hybrid fusion.
- Prompt assembly and the LLM call.
- Evaluation, offline, on a fixed question set.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 7 sections
- Agentic RAG
- Graph RAG
- Hierarchical RAG
- Multi-modal RAG
- Adaptive retrieval and self-RAG
- Hybrid, in the architecture sense
- What production adds
Test yourself
Answer these before moving on — recall is what makes it stick.