Retrieval Augmented Prompting Basics

Why this matters

Retrieval-Augmented Generation (RAG) lets you ground model answers in trusted, up-to-date sources. As a Prompt Engineer, you will:

• Build Q&A assistants that only answer from product docs or policies.
• Summarize collections of notes, tickets, or reports with source citations.
• Draft code or procedures using internal APIs/specs, not the model’s guesses.
• Reduce hallucinations and control scope with clear guardrails.

Who this is for

• Prompt engineers and data practitioners who need reliable, source-grounded answers.
• Developers creating assistants for documentation, support, analytics, or internal knowledge bases.

Prerequisites

• Basic prompt-writing (roles, tasks, constraints).
• Familiarity with tokens and context windows.
• Access to some text corpus (docs, policies, FAQs) or sample passages.

Concept explained simply

RAG = Retrieve relevant documents first, then generate an answer using only that material. Instead of asking the model to remember everything, you provide the right snippets at the right time.

Core components

• Query construction: How you turn the user question into a retrieval query (may include expansion, synonyms, or restatement).
• Retrieval method: Keyword, vector, or hybrid retrieval. Aim for high recall, then filter for precision.
• Context packaging: Concatenate top-k passages with clear separators, IDs, source names, and short summaries if needed.
• Instruction template: Role, task, domain boundary, constraints (cite, refuse if no evidence, style), and output format.
• Guardrails: Domain limits, refusal rules, handling ambiguity, and do not fabricate reminders.
• Evaluation loop: Spot-check for citation coverage, relevance, and absence of unsupported claims.

Worked examples

System: You are a precise assistant that answers ONLY from the provided policy excerpts.
User question: {question}

Context (do not alter):
{passages}

Assistant instructions:
- Answer using ONLY the context above.
- Cite passage IDs in square brackets like [P2], [P4].
- If the context is insufficient, say "I don't have enough information" and suggest what is missing.
- Keep it under 120 words.

Output:
- Direct answer followed by citations.

[P1] Remote Work Policy: Employees may work remotely up to 3 days/week with manager approval.
[P2] Equipment: The company provides a laptop; accessories are reimbursable up to $200/year.
[P3] Time Zones: Core hours are 10:00–16:00 local team time; exceptions require written approval.

System: You are an analyst summarizing the top themes from support tickets.
User request: Summarize key themes from the retrieved tickets and cite snippets.

Context (tickets):
[T1] Users report 2FA failure on Android 14 after app update 5.2.
[T2] SMS OTP delays in APAC between 18:00–21:00 SGT.
[T3] 2FA failure workaround: clear app cache; retry.

Instructions:
- List 2–3 themes with brief evidence.
- Include at least one citation per theme: [T#].
- Provide 1–2 actionable suggestions.

System: You write code strictly following the provided API docs. If a method is not in the docs, refuse.
User goal: Write a function to create a session token.

Context (docs):
[D1] AuthClient.createToken(user_id: string, ttl_minutes: number) returns Token.
[D2] Token has fields: value, expires_at.

Instructions:
- Return a minimal, correct example in Python.
- Cite the specific doc IDs used.
- If any parameter is unknown, ask a clarifying question.

Hands-on: Build your RAG prompt (step cards)

1. Define the goal: What question type or task should the model handle?
2. Set domain boundaries: State “Answer only from the provided context.”
3. Choose retrieval scope: top-k (e.g., 3–5), recency filters, or sections.
4. Design context format: Include passage IDs, titles, and brief snippets, separated clearly.
5. Write instructions: cite sources, refuse on insufficient evidence, style/length, output format.
6. Test and iterate: Try ambiguous queries; check if the model asks for clarification or refuses correctly.

System: You are a grounded assistant. Use ONLY the provided context.
User: {question}
Context:
{[ID] Title or source: snippet}
...
Instructions:
- Cite sources like [ID].
- If context is insufficient, say so and request the missing detail.
- Keep the tone concise and neutral.
Output:
- Final answer
- Sources: [ID, ID]

Exercises

Try these exercises. You can take the quick test afterward. Note: Everyone can access the test; sign in to save your progress.

Exercise 1 — Design a minimal RAG Q&A prompt

Mirror of Exercise ID: ex1

Scenario: You must answer questions about a company’s Remote Work Policy using only provided passages.

Passages:

[P1] Remote Work: Employees may work remotely up to 3 days/week with manager approval.
[P2] Equipment Reimbursement: Accessories reimbursed up to $200/year with receipt.
[P3] Core Hours: 10:00–16:00 local team time; exceptions require written approval.

Your task: Write a prompt template that:

• Accepts {question} and {top_passages} variables.
• Requires citations like [P#].
• Instructs refusal if evidence is missing.
• Limits answers to 100 words.

System: ...
User: {question}
Context: {top_passages}
Instructions: ...
Output: ...

Exercise 2 — Tighten guardrails and evaluate citations

Mirror of Exercise ID: ex2

Passages:

[D1] Refunds: Full refund within 30 days of purchase with receipt.
[D2] Store Credit: After 30 days, issues store credit only; no cash refunds.
[D3] Exclusions: Gift cards are non-refundable.

Question: "Can I get a cash refund after 40 days?"

Candidate answers:

1. A: “Yes, you can get a cash refund up to 60 days after purchase.”
2. B: “After 30 days, only store credit is available [D2].”
3. C: “Probably no refund unless it’s a gift card.”

Your task:

• Identify which answer(s) satisfy RAG constraints.
• Propose 2–3 prompt rules to reduce hallucinations and enforce citations.

Self-check checklist

• Your template explicitly says “Answer ONLY from the provided context.”
• You require citations using passage IDs.
• You include a clear refusal rule when evidence is insufficient.
• Your output format is short and structured.
• You tested with a tricky question and verified refusal/citation behavior.

Common mistakes and how to self-check

• Under-specifying domain: The model pulls from its own memory. Fix: Add “Use ONLY the provided context; if insufficient, say so.”
• Missing citations: Answers appear confident but untraceable. Fix: Require [ID] citations for each key claim.
• Over-long context: Irrelevant passages crowd out signal. Fix: Reduce top-k; summarize or highlight key lines.
• Ambiguity: Vague user questions lead to bad retrieval. Fix: Add a clarifying-question step when intent is unclear.
• Query drift: Reformulated queries change meaning. Fix: Constrain rewrites to synonyms and key entities; include original question in the prompt.

Practical projects

• Policy Assistant: Answer HR policy questions with citations and a refusal mode.
• Ticket Theme Miner: Summarize top 3 themes from support tickets with evidence snippets.
• API Guide Buddy: Generate code examples strictly from internal API docs; refuse unknown methods.

Learning path

• Start: Retrieval Augmented Prompting Basics (this lesson).
• Next: Query rewriting and hybrid retrieval to improve recall.
• Then: Context compression (summaries, salient sentence extraction).
• Finally: Evaluation frameworks (citation coverage, factuality checks) and automation.

Next steps

• Complete the exercises above.
• Take the Quick Test below to confirm understanding. Note: Everyone can take the test; sign in to save your progress.
• Move on to more advanced retrieval and evaluation techniques.

Mini challenge

Given 5 passages about “Premium Plan Limits,” craft a two-sentence answer to: “Does the Premium plan include unlimited projects?” Require at least one citation and refusal if evidence is missing. Keep it under 50 words.