Prompting Concepts

Why this matters

As an AI Product Manager, you won’t always write final prompts, but you will define product outcomes, constraints, and evaluation. Clear prompting transforms vague business needs into reliable model behavior. You will use prompting to prototype features, reduce hallucinations, enforce output formats, and align the model with your product’s tone, safety, and compliance needs.

• Ship faster: Validate ideas with well-structured prompts before investing in custom models.
• Reduce risk: Specify constraints and checks that lower hallucinations and off-brand outputs.
• Measure quality: Turn prompts into testable specifications with objective criteria.

Concept explained simply

A prompt is a set of instructions and context that tells a model what to do, how to do it, with what constraints, and how to present the result.

• Role: Who the model should act as (e.g., Support Agent).
• Objective: What must be produced (e.g., Summarize a ticket).
• Context: Relevant facts, data, or examples.
• Constraints: Rules, tone, forbidden behaviors, safety boundaries.
• Output format: The structure to return (e.g., valid JSON object with required fields).
• Quality checks: Self-check or validation steps (e.g., If unsure, say unknown).

Mental model

Use the RICCE-V checklist to design reliable prompts:

• R  Role: Define who is speaking.
• I  Instruction: State the task and success criteria.
• C  Context: Provide only the data needed.
• C  Constraints: Tone, safety, forbidden behaviors.
• E  Examples: 1 concise demonstrations.
• V  Validation: Output schema + self-check rules.

Think of the model as a talented but literal intern: give it the goal, relevant files, the format template, and a short checklist to self-verify before submitting.

Worked examples

Role: You are a support triage assistant.
Task: Extract fields from the ticket text.
Context: The ticket text is delimited by <ticket> ... </ticket>.
Constraints:
- If a field cannot be determined with high confidence, set it to "unknown".
- Only use information from the ticket text.
Output format: Return valid JSON with keys: urgency (low|medium|high), issue_type (billing|bug|access|other), summary (string, max 25 words).
Self-check: Ensure JSON is valid and keys exist.

Examples:
Input:
<ticket>Cannot access my account since yesterday. Urgent for my deadline.</ticket>
Output:
{"urgency":"high","issue_type":"access","summary":"User cannot access account and needs urgent help."}

Now extract for this input:
<ticket>The invoice total seems off by $20 and I need a corrected bill.</ticket>

Role: You are a precise product documentation assistant.
Instruction: Answer the user question strictly using the provided context. If the answer is not present, reply: "Not found in docs." Include a citation field listing the file IDs used.
Context: <docs>[doc_12: ...] [doc_34: ...]</docs>
Output format: JSON with keys: answer (string), citations (array of file IDs).
Validation: If no doc supports the answer, answer="Not found in docs." and citations=[].

Role: Senior customer success analyst.
Instruction: Summarize the chat into a single CRM note.
Tone: Neutral, factual, no promises, no medical or legal advice.
Length: 607 words.
Output format: A single paragraph; no bullet points.
Self-check: Confirm no speculative claims and length within bounds.
Input chat: <chat> ... </chat>

How to evaluate prompts

• Define a rubric: correctness, groundedness (uses provided context), clarity, format validity, safety, and style adherence.
• Create a small test set (102 cases) with expected outcomes or acceptability thresholds.
• Measure: pass rate on schema validity, average score on rubric (e.g., 15 scale), latency, and cost.
• Iterate: A/B prompts offline; promote the best to limited production; monitor real feedback.

Saving progress note: The quick test is available to everyone; only logged-in users will have their progress saved.

Common mistakes

• Vague instructions: Leads to inconsistent results.
• No output schema: Hard to parse or validate.
• Overloading with irrelevant context: Increases confusion and cost.
• Missing fallback rules: Forces the model to guess instead of saying unknown.
• Too many or too long examples: Drowns the key instruction.
• Ignoring evaluation: Shipping prompts without tests reduces reliability.

Exercises

Complete these hands-on tasks. They mirror the exercises below and include checklists to guide you.

Practical projects

• Build a prompt kit: A small library of reusable prompt templates (extraction, summarization, classification) with JSON schemas and brief rubrics.
• RAG prototype: Insert retrieved snippets into a prompt and enforce Not found in docs behavior; measure groundedness.
• Prompt A/B study: Compare two prompts on 20 labeled cases; report accuracy, schema validity, and average latency.

Who this is for

• AI Product Managers and PMs exploring LLM features.
• Designers and Analysts collaborating on AI-assisted experiences.
• Engineers who want product-ready prompt specs.

Prerequisites

• Basic understanding of LLM capabilities and limitations.
• Comfort with JSON and structured outputs.
• Familiarity with your products safety and tone guidelines.

Learning path

• Start: This subskill (Prompting Concepts) to craft reliable prompts.
• Next: Guardrails and evaluation to quantify quality and safety.
• Then: RAG and tool use to ground answers and integrate external systems.
• Finally: Experiment design and monitoring for production readiness.

Next steps

• Refine your prompts using the RICCE-V checklist.
• Create a 10-case test set for your most important prompt.
• Run an offline A/B and document results and trade-offs.

Mini challenge

Write a production-ready prompt spec for a feature your team cares about (e.g., email triage, knowledge-based Q&A). Include role, instruction, context delimiters, constraints, 2 examples, JSON schema, and a short validation rule. Test on 5 real inputs. Aim for at least 80% rubric pass rate and 100% valid JSON.