Why this matters
- The same retrieved paragraph produced a clean 8-day answer or an invented carry-forward rule depending only on the instruction line.
- Interviewers ask for a template, not a definition of "prompt engineering."
- Regulated answers (HR, finance, legal) need a format you can grade: decision, citation, or an explicit "not in the policy."
- A long prompt full of examples can crowd out the chunks. The evidence has to fit.
What you are designing
Prompt engineering is the design of the instructions that steer the model. Outside RAG, that means tone and format. Inside RAG, it also means what counts as evidence. The retrieved chunks are not decoration. They are the only source the answer is allowed to use.
A good RAG prompt does four things at once: it states the role, it forbids unsupported claims, it shows the context in a delimited block, and it asks for a shape of answer the product can render (a sentence, a list, or a JSON object with citations).
Why the prompt is part of the system, not the copy
- The model will otherwise blend the handbook with whatever leave policy is common on the public web.
- It will answer in a paragraph when your UI expects bullets, or in prose when a downstream service expects JSON.
- "I don't know" has to be a permitted, even rewarded, outcome. If the prompt only says "be helpful," the model will help by inventing.
- Domain apps (HR, finance, legal) need tighter contracts than a general chatbot.
A baseline template
You are PolicyDesk, a careful HR assistant for Northline Bank.
Answer the question using only the context below.
If the answer is not in the context, say "I don't know"
and do not guess.
Mention the source title when you use a fact.
Context:
---
{context}
---
Question: {question}
Answer:
The delimiter (---) matters. It makes the boundary visible to the model and to you when you log the prompt. Put the question after the context or before it, and keep it consistent. What you must not do is concatenate the user text and the document text with no label. The model then cannot tell a user instruction from a sentence in the handbook — a problem that becomes a security bug once documents can contain instructions.
Variants and when they earn their place
| Pattern | Shape | Use it when |
|---|---|---|
| Zero-shot | Role, rules, context, question. No examples. | The task is clear and you want every token spent on evidence. This is the PolicyDesk default. |
| Few-shot | One to three examples of question, context, and a good answer. | Format is subtle: how to cite, how to refuse, how to say 'probationary employees are excluded.' |
| Chain-of-thought | Ask the model to reason in steps before the final line. | Multi-hop questions ('does a probationary employee in sales get casual leave?') where the chunks interact. Hide the reasoning from the user if the product should show only the decision. |
| Structured output | Demand JSON with fields such as answer, citations, and known. | A service will parse the result. Pair it with schema validation, not hope. |
| Role prompt | A specific expert identity plus the same grounding rules. | You need the vocabulary of that job. The role does not replace the grounding rules. |
Zero-shot
ShapeRole, rules, context, question. No examples.Use it whenThe task is clear and you want every token spent on evidence. This is the PolicyDesk default.Few-shot
ShapeOne to three examples of question, context, and a good answer.Use it whenFormat is subtle: how to cite, how to refuse, how to say 'probationary employees are excluded.'Chain-of-thought
ShapeAsk the model to reason in steps before the final line.Use it whenMulti-hop questions ('does a probationary employee in sales get casual leave?') where the chunks interact. Hide the reasoning from the user if the product should show only the decision.Structured output
ShapeDemand JSON with fields such as answer, citations, and known.Use it whenA service will parse the result. Pair it with schema validation, not hope.Role prompt
ShapeA specific expert identity plus the same grounding rules.Use it whenYou need the vocabulary of that job. The role does not replace the grounding rules.
Few-shot examples should be short and should include one refusal example. If every example is a confident answer, the model learns that confidence is the format, including when the context is empty.
Chain-of-thought increases latency and can leak rambling reasoning into a UI that expected a sentence. It also sometimes reduces faithfulness if the model talks itself into a fact that was not in the chunk. Use it for composition across chunks, and still require the final line to cite.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 4 sections
- Four templates, from loose to strict
- Controls that change behavior
- Where the prompt sits in the flow
- Practices
Test yourself
Answer these before moving on — recall is what makes it stick.