Advanced Prompt Engineering: Techniques, Patterns & Production Systems
Advanced Prompt Engineering: Techniques, Patterns & Production Systems
Master prompt engineering from fundamentals to production systems — chain-of-thought, few-shot, RAG prompting, agent prompts, and evaluation frameworks.
Introduction
Prompt engineering is the discipline of designing inputs to elicit optimal outputs from LLMs. This guide covers production-grade techniques — not just tips and tricks.
Prompt Engineering Maturity Model
| Level | Approach | Example |
|-------|----------|---------|
| 1 | Ad-hoc prompting | "Write a summary" |
| 2 | Structured templates | "Summarize in 3 bullets: {text}" |
| 3 | Few-shot + CoT | Examples + "Think step by step" |
| 4 | RAG-integrated | Context-aware with citations |
| 5 | Agent prompts | Multi-step with tool use |
| 6 | Self-improving | Prompt optimization loops |
Part 1: Foundational Techniques
Zero-Shot Prompting
# prompting/zero_shot.py
from dataclasses import dataclass
from typing import Optional
@dataclass
class PromptTemplate:
system: str
user: str
variables: list[str]
def format(self, **kwargs) -> dict:
return {
"system": self.system,
"user": self.user.format(**kwargs)
}
# Basic templates
SUMMARIZE = PromptTemplate(
system="You are a concise summarizer.",
user="Summarize the following text in {num_sentences} sentences:\n\n{text}",
variables=["num_sentences", "text"]
)
EXTRACT_ENTITIES = PromptTemplate(
system="You extract structured entities from text.",
user="Extract {entity_types} from this text:\n\n{text}\n\nOutput as JSON.",
variables=["entity_types", "text"]
)
CLASSIFY = PromptTemplate(
system="You are a precise classifier.",
user="Classify this text into ONE category: {categories}\n\nText: {text}\n\nCategory:",
variables=["categories", "text"]
)
Few-Shot Prompting
# prompting/few_shot.py
from typing import List, Dict
def build_few_shot_prompt(
task_description: str,
examples: List[Dict[str, str]],
test_input: str,
output_format: str = ""
) -> str:
"""Build a few-shot prompt with consistent formatting."""
parts = [task_description, ""]
for i, ex in enumerate(examples, 1):
parts.append(f"Example {i}:")
parts.append(f"Input: {ex['input']}")
parts.append(f"Output: {ex['output']}")
parts.append("")
parts.append(f"Input: {test_input}")
if output_format:
parts.append(f"Output ({output_format}):")
else:
parts.append("Output:")
return "\n".join(parts)
# Example: SQL generation
SQL_EXAMPLES = [
{
"input": "Find all users who signed up last month",
"output": "SELECT * FROM users WHERE signup_date >= date('now', '-1 month')"
},
{
"input": "Count orders per customer",
"output": "SELECT customer_id, COUNT(*) as order_count FROM orders GROUP BY customer_id"
},
{
"input": "Get products with price > 100",
"output": "SELECT * FROM products WHERE price > 100"
}
]
prompt = build_few_shot_prompt(
task_description="Convert natural language to SQLite SQL.",
examples=SQL_EXAMPLES,
test_input="Show me the top 5 customers by total spend",
output_format="SQL only, no markdown"
)
Chain-of-Thought (CoT)
# prompting/cot.py
COT_PROMPT = """Solve this problem step by step.
Problem: {problem}
Think through this carefully:
1. First, identify what is being asked
2. Break down the problem into sub-problems
3. Solve each sub-problem
4. Combine results for final answer
Show your reasoning at each step.
Step 1:"""
COT_PROMPT_STRUCTURED = """{problem}
Let's think step by step.
{scratchpad_start}"""
# Self-consistency: multiple CoT paths, majority vote
async def self_consistency_cot(llm, problem: str, n: int = 5) -> str:
"""Run CoT multiple times, return majority answer."""
import asyncio
from collections import Counter
async def single_cot():
response = await llm.ainvoke(COT_PROMPT.format(problem=problem))
# Extract final answer (last number or conclusion)
return extract_answer(response)
results = await asyncio.gather(*[single_cot() for _ in range(n)])
counter = Counter(results)
return counter.most_common(1)[0][0]
def extract_answer(text: str) -> str:
# Extract final answer from CoT output
import re
# Look for "Answer: X" or final number
answer_match = re.search(r"(?:Answer|Final answer|Result):\s*(.+)", text, re.IGNORECASE)
if answer_match:
return answer_match.group(1).strip()
# Fallback: last sentence
return text.strip().split('\n')[-1]
Part 2: Advanced Patterns
Tree of Thoughts (ToT)
# prompting/tot.py
from dataclasses import dataclass
from typing import List, Optional
import asyncio
@dataclass
class Thought:
content: str
score: float = 0.0
parent: Optional['Thought'] = None
children: List['Thought'] = None
def __post_init__(self):
if self.children is None:
self.children = []
class TreeOfThoughts:
def __init__(self, llm, branching_factor: int = 3, max_depth: int = 3):
self.llm = llm
self.branching_factor = branching_factor
self.max_depth = max_depth
async def solve(self, problem: str) -> str:
# Generate initial thoughts
root = Thought(content=problem)
current_level = [root]
for depth in range(self.max_depth):
next_level = []
for thought in current_level:
# Generate branches
branches = await self._generate_branches(thought.content)
for branch in branches[:self.branching_factor]:
child = Thought(content=branch, parent=thought)
thought.children.append(child)
next_level.append(child)
# Evaluate and prune
scored = await self._evaluate_thoughts(next_level, problem)
scored.sort(key=lambda x: x.score, reverse=True)
current_level = scored[:self.branching_factor]
# Get best path
best = current_level[0]
path = []
while best:
path.append(best.content)
best = best.parent
return " → ".join(reversed(path))
async def _generate_branches(self, thought: str) -> List[str]:
prompt = f"""Given this partial solution: {thought}
Generate {self.branching_factor} different next steps or approaches.
Each should be a distinct reasoning direction.
Return as numbered list."""
response = await self.llm.ainvoke(prompt)
# Parse numbered list
return [line.split('. ', 1)[1] for line in response.split('\n') if '. ' in line]
async def _evaluate_thoughts(self, thoughts: List[Thought], problem: str) -> List[Thought]:
for thought in thoughts:
prompt = f"""Problem: {problem}
Proposed solution path: {thought.content}
Rate the promise of this approach on a scale of 0-10.
Consider: correctness, completeness, feasibility.
Return only a number."""
response = await self.llm.ainvoke(prompt)
try:
thought.score = float(response.strip())
except:
thought.score = 0.0
return thoughts
ReAct (Reasoning + Acting)
# prompting/react.py
REACT_TEMPLATE = """You are an agent that can use tools to solve problems.
Available tools:
{tools}
Format:
Question: {question}
Thought: I need to...
Action: tool_name
Action Input: {{"arg": "value"}}
Observation: [tool result]
... (repeat as needed)
Thought: I now know the answer
Final Answer: {answer}
Begin!
Question: {question}
Thought:"""
class ReActPromptBuilder:
def __init__(self, tools: list[dict]):
self.tools = tools
def build(self, question: str) -> str:
tool_desc = "\n".join(f"- {t['name']}: {t['description']}" for t in self.tools)
return REACT_TEMPLATE.format(tools=tool_desc, question=question)
def parse_response(self, response: str) -> dict:
import re
# Check for final answer
final_match = re.search(r"Final Answer:\s*(.+)", response, re.DOTALL)
if final_match:
return {"type": "final", "content": final_match.group(1).strip()}
# Check for action
action_match = re.search(r"Action:\s*(\w+)", response)
input_match = re.search(r"Action Input:\s*(\{.+\})", response)
if action_match and input_match:
import json
return {
"type": "action",
"tool": action_match.group(1),
"args": json.loads(input_match.group(1))
}
return {"type": "thought", "content": response}
RAG Prompting with Citations
# prompting/rag_prompts.py
RAG_PROMPT = """You are a precise assistant that answers questions using ONLY the provided context.
## Context Documents
{context}
## Instructions
1. Answer ONLY from the provided context
2. If context is insufficient, say "I don't have enough information to answer this"
3. Cite sources inline using [doc_N] format where N is the document number
4. Be concise but complete
5. If asked for a list, use bullet points
## Question
{question}
## Answer:"""
RAG_PROMPT_STRUCTURED = """{context}
## Question
{question}
## Output Format (JSON)
{{
"answer": "Your answer with [doc_N] citations",
"citations": [
{{"doc_id": "doc_1", "quote": "exact quote from doc", "relevance": "high"}}
],
"confidence": 0.95,
"insufficient_context": false
}}"""
def format_rag_context(docs: list, max_chars: int = 8000) -> str:
"""Format retrieved documents for RAG prompt."""
parts = []
total = 0
for i, doc in enumerate(docs, 1):
content = doc.get('content', '')[:2000] # Limit per doc
source = doc.get('metadata', {}).get('source', 'Unknown')
part = f"[doc_{i}] Source: {source}\n{content}"
if total + len(part) > max_chars:
break
parts.append(part)
total += len(part)
return "\n\n---\n\n".join(parts)
Part 3: Agent System Prompts
Supervisor Agent
# prompting/agent_prompts.py
SUPERVISOR_PROMPT = """You are a Supervisor Agent managing a team of specialist agents.
## Your Role
- Decompose complex user requests into subtasks
- Delegate subtasks to appropriate specialists
- Synthesize specialist outputs into coherent responses
- Track progress and handle failures
## Specialist Agents Available
{specialists}
## Delegation Protocol
When delegating, use this format:
DELEGATE: {{"agent": "agent_name", "task": "specific task description", "context": {{}}}}
## Response Format
1. If task is complete: Provide final synthesized answer
2. If delegation needed: Output DELEGATE command only
3. If clarification needed: Ask specific question
## Current Request
{request}
## Previous Delegation Results
{history}
Your response:"""
### Specialist Agent (Researcher)
RESEARCHER_PROMPT = """You are a Research Specialist Agent.
Your Capabilities
- Web search for current information
- Document lookup and summarization
- Fact verification and cross-referencing
- Technical concept explanation
Tools Available
{tools}
Instructions
1. Break down the research task into specific queries
2. Execute searches in parallel when possible
3. Synthesize findings with citations
4. Flag uncertainties or conflicting sources
5. Return structured results
Task
{task}
Context
{context}
Your response (use DELEGATE format for tool calls):"""
### Code Generation Agent
CODER_PROMPT = """You are a Senior Software Engineer Agent.
Your Capabilities
- Write production-quality code in multiple languages
- Debug and fix existing code
- Write tests and documentation
- Refactor for performance and clarity
- Architecture and design decisions
Principles
- Write clean, readable, well-documented code
- Follow language idioms and best practices
- Include error handling and edge cases
- Write tests for non-trivial logic
- Explain design decisions
Tools Available
{tools}
Task
{task}
Context (existing code, requirements, constraints)
{context}
Output Format
Provide code in markdown blocks with language tags.
Include brief explanation of approach.
If using tools, use DELEGATE format.
Your response:"""
## Part 4: Prompt Optimization
### Automated Prompt Optimization
prompting/optimization.py
from dataclasses import dataclass
from typing import List, Callable
import asyncio
@dataclass
class PromptCandidate:
prompt: str
score: float = 0.0
metadata: dict = None
class PromptOptimizer:
def init(
self,
llm,
evaluator: Callable[[str, str], float], # (prompt, expected) -> score
population_size: int = 10,
generations: int = 5
):
self.llm = llm
self.evaluator = evaluator
self.population_size = population_size
self.generations = generations
async def optimize(
self,
base_prompt: str,
test_cases: List[dict], # {"input": "", "expected": ""}
mutation_rate: float = 0.3
) -> PromptCandidate:
# Initialize population
population = await self._initialize_population(base_prompt)
for gen in range(self.generations):
# Evaluate
scored = await self._evaluate_population(population, test_cases)
scored.sort(key=lambda x: x.score, reverse=True)
# Select top performers
elite = scored[:self.population_size // 2]
# Generate next generation
population = elite.copy()
while len(population) < self.population_size:
parent = elite[0] # Best parent
if random.random() < mutation_rate:
child = await self._mutate(parent.prompt)
else:
child = await self._crossover(elite[0].prompt, elite[1].prompt)
population.append(PromptCandidate(prompt=child))
return scored[0]
async def _initialize_population(self, base: str) -> List[PromptCandidate]:
population = [PromptCandidate(prompt=base)]
# Generate variations
variations = [
"Add more explicit instructions",
"Add few-shot examples",
"Add chain-of-thought prompting",
"Add output format specification",
"Add role/persona definition",
"Add constraint specifications",
"Add step-by-step reasoning",
"Add self-verification step",
"Add edge case handling",
"Add few-shot with diverse examples"
]
for var in variations[:self.population_size - 1]:
prompt = await self._apply_variation(base, var)
population.append(PromptCandidate(prompt=prompt))
return population
async def _mutate(self, prompt: str) -> str:
mutations = [
"Add 'Think step by step' to the prompt",
"Add 'Be concise' instruction",
"Add output format specification",
"Add 'Verify your answer before responding'",
"Add role definition at the start",
"Add constraint: no markdown unless asked",
"Add 'If uncertain, say so explicitly'",
"Restructure into clear sections"
]
return await self._apply_variation(prompt, random.choice(mutations))
async def _crossover(self, p1: str, p2: str) -> str:
# Combine best parts of two prompts
prompt = f"""Combine the best elements of these two prompts:
Prompt 1:
{p1}
Prompt 2:
{p2}
Create a single improved prompt that incorporates the strengths of both."""
return await self.llm.ainvoke(prompt)
async def _apply_variation(self, prompt: str, variation: str) -> str:
prompt_text = f"""Improve this prompt by: {variation}
Original prompt:
{prompt}
Return only the improved prompt."""
return await self.llm.ainvoke(prompt_text)
### DSPy-Style Optimization
prompting/dspy_style.py
from dataclasses import dataclass
from typing import List, Any
@dataclass
class Example:
inputs: dict
outputs: dict
class PromptModule:
def init(self, prompt_template: str, llm):
self.template = prompt_template
self.llm = llm
self.demos: List[Example] = []
def add_demo(self, inputs: dict, outputs: dict):
self.demos.append(Example(inputs=inputs, outputs=outputs))
def call(self, inputs) -> Any:
# Format prompt with demos
demo_text = ""
for demo in self.demos:
demo_text += f"Input: {demo.inputs}\nOutput: {demo.outputs}\n\n"
full_prompt = self.template.format(demos=demo_text, **inputs)
return self.llm.invoke(full_prompt)
def compile(self, trainset: List[Example], metric: Callable):
# Bootstrap demos from trainset using metric
scored = [(metric(ex.outputs, self(ex.inputs)), ex) for ex in trainset]
scored.sort(reverse=True)
self.demos = [ex for _, ex in scored[:3]] # Top 3
## Part 5: Evaluation Frameworks
### Prompt Evaluation Metrics
prompting/evaluation.py
from dataclasses import dataclass
from typing import List, Callable
import asyncio
@dataclass
class PromptMetrics:
accuracy: float
consistency: float # Self-consistency across runs
instruction_following: float
format_compliance: float
latency_ms: float
token_count: int
cost_usd: float
class PromptEvaluator:
def init(self, llm, judge_llm=None):
self.llm = llm
self.judge = judge_llm or llm
async def evaluate(
self,
prompt: str,
test_cases: List[dict],
n_runs: int = 3
) -> PromptMetrics:
all_outputs = []
latencies = []
token_counts = []
for tc in test_cases:
case_outputs = []
for _ in range(n_runs):
start = time.time()
output = await self.llm.ainvoke(prompt.format(**tc["input"]))
latency = (time.time() - start) * 1000
case_outputs.append(output)
latencies.append(latency)
token_counts.append(len(output.split()))
all_outputs.append(case_outputs)
# Accuracy via LLM judge
accuracy = await self._judge_accuracy(test_cases, all_outputs)
# Consistency: agreement across runs
consistency = self._compute_consistency(all_outputs)
# Instruction following
instruction_following = await self._judge_instruction_following(
prompt, test_cases, all_outputs
)
# Format compliance
format_compliance = self._check_format_compliance(all_outputs, prompt)
return PromptMetrics(
accuracy=accuracy,
consistency=consistency,
instruction_following=instruction_following,
format_compliance=format_compliance,
latency_ms=sum(latencies) / len(latencies),
token_count=sum(token_counts) // len(token_counts),
cost_usd=self._estimate_cost(token_counts)
)
async def _judge_accuracy(self, test_cases, all_outputs) -> float:
scores = []
for tc, outputs in zip(test_cases, all_outputs):
# Use first output for evaluation
output = outputs[0]
prompt = f"""Rate the accuracy of this response (0-1).
Expected: {tc.get('expected', 'N/A')}
Actual: {output}
Consider: factual correctness, completeness, relevance.
Return only a number between 0 and 1."""
response = await self.judge.ainvoke(prompt)
try:
scores.append(float(response.strip()))
except:
scores.append(0.5)
return sum(scores) / len(scores) if scores else 0
def _compute_consistency(self, all_outputs) -> float:
# Semantic similarity between runs (simplified)
agreements = 0
total = 0
for outputs in all_outputs:
for i in range(len(outputs)):
for j in range(i + 1, len(outputs)):
# Simple word overlap similarity
words_i = set(outputs[i].lower().split())
words_j = set(outputs[j].lower().split())
if words_i and words_j:
overlap = len(words_i & words_j) / len(words_i | words_j)
agreements += overlap
total += 1
return agreements / total if total > 0 else 1.0
### A/B Testing Framework
prompting/ab_test.py
from dataclasses import dataclass
from typing import List
import random
@dataclass
class ABTestResult:
variant_a: PromptMetrics
variant_b: PromptMetrics
winner: str # "A", "B", or "tie"
confidence: float
sample_size: int
class ABTester:
def init(self, evaluator: PromptEvaluator):
self.evaluator = evaluator
async def run_test(
self,
prompt_a: str,
prompt_b: str,
test_cases: List[dict],
min_samples: int = 50
) -> ABTestResult:
# Randomly assign test cases to variants
random.shuffle(test_cases)
split = len(test_cases) // 2
cases_a = test_cases[:split]
cases_b = test_cases[split:]
# Evaluate both
metrics_a = await self.evaluator.evaluate(prompt_a, cases_a)
metrics_b = await self.evaluator.evaluate(prompt_b, cases_b)
# Statistical significance (simplified)
winner = "A" if metrics_a.accuracy > metrics_b.accuracy else "B"
if abs(metrics_a.accuracy - metrics_b.accuracy) < 0.02:
winner = "tie"
return ABTestResult(
variant_a=metrics_a,
variant_b=metrics_b,
winner=winner,
confidence=self._compute_confidence(metrics_a, metrics_b),
sample_size=len(test_cases)
)
## Part 6: Production Prompt Management
### Prompt Versioning & Registry
prompting/registry.py
import sqlite3
import json
from dataclasses import dataclass
from datetime import datetime
from typing import Optional
@dataclass
class PromptVersion:
id: str
name: str
version: int
prompt: str
metadata: dict
created_at: str
parent_id: Optional[str] = None
class PromptRegistry:
def init(self, db_path: str = "prompts.db"):
self.conn = sqlite3.connect(db_path)
self._init_db()
def _init_db(self):
self.conn.execute("""
CREATE TABLE IF NOT EXISTS prompts (
id TEXT PRIMARY KEY,
name TEXT,
version INTEGER,
prompt TEXT,
metadata TEXT,
created_at TEXT,
parent_id TEXT
)
""")
self.conn.execute("""
CREATE INDEX IF NOT EXISTS idx_name_version ON prompts(name, version)
""")
self.conn.commit()
def save(self, prompt: PromptVersion):
self.conn.execute("""
INSERT INTO prompts (id, name, version, prompt, metadata, created_at, parent_id)
VALUES (?, ?, ?, ?, ?, ?, ?)
""", (
prompt.id, prompt.name, prompt.version,
prompt.prompt, json.dumps(prompt.metadata),
prompt.created_at, prompt.parent_id
))
self.conn.commit()
def get_latest(self, name: str) -> Optional[PromptVersion]:
cursor = self.conn.execute(
"SELECT * FROM prompts WHERE name = ? ORDER BY version DESC LIMIT 1",
(name,)
)
row = cursor.fetchone()
return self._row_to_prompt(row) if row else None
def get_version(self, name: str, version: int) -> Optional[PromptVersion]:
cursor = self.conn.execute(
"SELECT * FROM prompts WHERE name = ? AND version = ?",
(name, version)
)
row = cursor.fetchone()
return self._row_to_prompt(row) if row else None
def get_history(self, name: str) -> List[PromptVersion]:
cursor = self.conn.execute(
"SELECT * FROM prompts WHERE name = ? ORDER BY version",
(name,)
)
return [self._row_to_prompt(r) for r in cursor.fetchall()]
def _row_to_prompt(self, row) -> PromptVersion:
return PromptVersion(
id=row[0], name=row[1], version=row[2],
prompt=row[3], metadata=json.loads(row[4]),
created_at=row[5], parent_id=row[6]
)
### Prompt Deployment with Feature Flags
prompting/deployment.py
from dataclasses import dataclass
from typing: Dict, Optional
import random
@dataclass
class PromptConfig:
name: str
version: int
traffic_percentage: float = 100.0 # 0-100
conditions: Dict[str, any] = None # e.g., {"user_tier": "premium"}
class PromptDeployment:
def init(self, registry: PromptRegistry):
self.registry = registry
self.configs: Dict[str, PromptConfig] = {}
def register(self, config: PromptConfig):
self.configs[config.name] = config
def get_prompt(self, name: str, context: dict = None) -> str:
config = self.configs.get(name)
if not config:
# Fallback to latest
prompt = self.registry.get_latest(name)
return prompt.prompt if prompt else ""
# Check conditions
if config.conditions and context:
for key, value in config.conditions.items():
if context.get(key) != value:
# Condition not met, get previous version
prev = self.registry.get_version(name, config.version - 1)
return prev.prompt if prev else ""
# Check traffic percentage
if config.traffic_percentage < 100:
if random.random() * 100 > config.traffic_percentage:
prev = self.registry.get_version(name, config.version - 1)
return prev.prompt if prev else ""
prompt = self.registry.get_version(name, config.version)
return prompt.prompt if prompt else ""
## Part 7: Specialized Prompt Patterns
### Structured Output Enforcement
prompting/structured_output.py
from pydantic import BaseModel, Field
from typing import List, Optional, Literal
import instructor
class StructuredPrompts:
@staticmethod
def json_schema_prompt(schema: dict, examples: List[dict] = None) -> str:
"""Generate prompt that enforces JSON schema output."""
schema_str = json.dumps(schema, indent=2)
example_str = ""
if examples:
example_str = "\n\nExamples:\n"
for i, ex in enumerate(examples, 1):
example_str += f"Example {i}:\n{json.dumps(ex, indent=2)}\n\n"
return f"""Output ONLY valid JSON matching this schema:
{schema_str}
Rules:
- No markdown, no explanations, no extra text
- All required fields must be present
- Types must match exactly
- {example_str}"""
@staticmethod
def constrained_generation_prompt(constraints: List[str]) -> str:
"""Generate prompt with hard constraints."""
constraint_text = "\n".join(f"- {c}" for c in constraints)
return f"""You must follow these constraints STRICTLY:
{constraint_text}
Violating any constraint makes the output invalid.
Think carefully before each token."""
### Multilingual Prompting
prompting/multilingual.py
MULTILINGUAL_TEMPLATE = """You are a multilingual assistant. Respond in {target_language}.
Task
{task}
Context
{context}
Constraints
- Respond ONLY in {target_language}
- Maintain cultural appropriateness
- Use region-appropriate terminology
- {additional_constraints}
Output
"""
Language-specific adjustments
LANGUAGE_ADJUSTMENTS = {
"ja": "Use polite forms (desu/masu). Avoid direct translations.",
"zh": "Use Simplified Chinese. Prefer formal written style.",
"de": "Use formal 'Sie' form. Compound nouns appropriately.",
"es": "Use formal 'usted'. Consider regional variations.",
"fr": "Use formal 'vous'. Follow French typography rules.",
"ar": "Use Modern Standard Arabic. RTL-aware formatting.",
}
## Decision Framework
PROMPT PATTERN SELECTION:
Simple, deterministic task?
├─→ Yes → Zero-shot with clear template
└─→ No
├─→ Need specific format/style?
│ ├─→ Yes → Few-shot with examples
│ └─→ No
│ ├─→ Complex reasoning needed?
│ │ ├─→ Yes → Chain-of-Thought or Tree of Thoughts
│ │ └─→ No
│ │
│ ├─→ Need external info?
│ │ └─→ Yes → RAG prompting with citations
│ │
│ ├─→ Multi-step with tools?
│ │ └─→ Yes → ReAct or Agent system prompt
│ │
│ └─→ High-stakes / production?
│ └─→ Yes → Optimized + evaluated + versioned
Production Checklist
Prompt Quality
- [ ] Clear role/persona definition
- [ ] Explicit task description
- [ ] Specific output format requirements
- [ ] Constraints and guardrails
- [ ] Few-shot examples (3-5 diverse cases)
- [ ] Chain-of-thought for reasoning tasks
- [ ] Edge case handling instructions
- [ ] Self-verification step
Evaluation
- [ ] Golden test set (50+ cases)
- [ ] Automated metrics (accuracy, format, consistency)
- [ ] LLM judge for subjective quality
- [ ] A/B testing framework
- [ ] Regression detection in CI/CD
Management
- [ ] Version control for all prompts
- [ ] Prompt registry with metadata
- [ ] Feature flags for gradual rollout
- [ ] Canary deployment support
- [ ] Rollback capability (< 1 min)
- [ ] Usage analytics per prompt version
Security
- [ ] Input sanitization
- [ ] Injection attempt detection
- [ ] PII handling instructions
- [ ] Output validation
- [ ] Rate limiting awareness
Resources
- Prompt Engineering Guide (OpenAI)
- Chain-of-Thought Paper
- Tree of Thoughts Paper
- ReAct Paper
- DSPy Framework
- Instructor Library
- Prompt Engineering Patterns
---
Prompt engineering is not magic — it's systematic communication with a probabilistic system. Treat prompts as code: version, test, evaluate, and deploy with the same rigor.
---
📦 Want the production-ready version?
This article covers the fundamentals. The AI Prompt Engineering Mastery System pack goes further — prompt patterns, evaluation harnesses and production prompt systems, plus full source code and deployment configs.
→ Get AI Prompt Engineering Mastery System
Extracted from real production systems, not toy examples.
Mentions
AI and ML Papers Explained (HUGE list): dair-ai/ML-Papers-Explained Compiled by @omarsar0 @dair_ai 💯🏆🚀💥 #ML #MachineLearning #DataScience #Algorithms #NeuralNetworks #DeepLearning #Mathematics
4 Prompt Engineering Guide Lists papers, lessons & notebooks on prompting, context engineering, RAG & agents dair-ai/Prompt-Engineering-Guide
Prompt-Engineering-Guide ⭐ 79k A collection of guides and resources for developers and researchers learning prompt engineering. 🔗 dair-ai/Prompt-Engineering-Guide #opensource t.co
Trying to keep up with AI research? dair.ai's AI Papers of the Week repo picks the top papers every week: For example, Harness Learning from Carnegie Mellon. academy.dair.ai 👇 dair-ai/AI-Papers-of-the-Week t.co
Advanced Prompt Engineering: Techniques, Patterns & Production Systems Advanced Prompt Engineering: Techniques, Patterns & Production Systems Master prompt engineering from fundamentals to production systems — chain-of-thought, few-shot, RAG prompting, agent prompts, and evaluation frameworks. Introduction Prompt engineering is the discipline of designing inputs to elicit optimal outputs from LLMs. This guide covers production-grade techniques — not just tips and tricks. Prompt Engineering Maturity Model | Level | Approach | Example | |-------|----------|---------| | 1 | Ad-hoc prompting | "Write a summary" | | 2 | Structured templates | "Summarize in 3 bullets: {text}" | | 3 | Few-shot + CoT | Examples + "Think step by step" | | 4 | RAG-integrated | Context-aware with citations | | 5 | Agent prompts | Multi-step with tool use | | 6 | Self-improving | Prompt optimization loops | Part 1: Foundational Techniques Zero-Shot Prompting # prompting/zero_shot.py from dataclasses import dataclass from typing import Optional @dataclass class PromptTemplate: system: str user: str variables: list[str] def format(self, **kwargs) -> dict: return { "system": self.system, "user": self.user.format(**kwargs) } # Basic templates SUMMARIZE = PromptTemplate( system="You are a concise summarizer.", user="Summarize the following text in {num_sentences} sentences:\n\n{text}", variables=["num_sentences", "text"] ) EXTRACT_ENTITIES = PromptTemplate( system="You extract structured entities from text.", user="Extract {entity_types} from this text:\n\n{text}\n\nOutput as JSON.", variables=["entity_types", "text"] ) CLASSIFY = PromptTemplate( system="You are a precise classifier.", user="Classify this text into ONE category: {categories}\n\nText: {text}\n\nCategory:", variables=["categories", "text"] ) Few-Shot Prompting # prompting/few_shot.py from typing import List, Dict def build_few_shot_prompt( task_description: str, examples: List[Dict[str, str]], test_input: str, output_format: str = "" ) -> str: """Build a few-shot prompt with consistent formatting.""" parts = [task_description, ""] for i, ex in enumerate(examples, 1): parts.append(f"Example {i}:") parts.append(f"Input: {ex['input']}") parts.append(f"Output: {ex['output']}") parts.append("") parts.append(f"Input: {test_input}") if output_format: parts.append(f"Output ({output_format}):") else: parts.append("Output:") return "\n".join(parts) # Example: SQL generation SQL_EXAMPLES = [ { "input": "Find all users who signed up last month", "output": "SELECT * FROM users WHERE signup_date >= date('now', '-1 month')" }, { "input": "Count orders per customer", "output": "SELECT customer_id, COUNT(*) as order_count FROM orders GROUP BY customer_id" }, { "input": "Get products with price > 100", "output": "SELECT * FROM products WHERE price > 100" } ] prompt = build_few_shot_prompt( task_description="Convert natural language to SQLite SQL.", examples=SQL_EXAMPLES, test_input="Show me the top 5 customers by total spend", output_format="SQL only, no markdown" ) Chain-of-Thought (CoT) # prompting/cot.py COT_PROMPT = """Solve this problem step by step. Problem: {problem} Think through this carefully: 1. First, identify what is being asked 2. Break down the problem into sub-problems 3. Solve each sub-problem 4. Combine results for final answer Show your reasoning at each step. Step 1:""" COT_PROMPT_STRUCTURED = """{problem} Let's think step by step. {scratchpad_start}""" # Self-consistency: multiple CoT paths, majority vote async def self_consistency_cot(llm, problem: str, n: int = 5) -> str: """Run CoT multiple times, return majority answer.""" import asyncio from collections import Counter async def single_cot(): response = await llm.ainvoke(COT_PROMPT.format(problem=problem)) # Extract final answer (last number or conclusion) return extract_answer(response) results = await asyncio.gather(*[single_cot() for _ in range(n)]) counter = Counter(results) return counter.most_common(1)[0][0] def extract_answer(text: str) -> str: # Extract final answer from CoT output import re # Look for "Answer: X" or final number answer_match = re.search(r"(?:Answer|Final answer|Result):\s*(.+)", text, re.IGNORECASE) if answer_match: return answer_match.group(1).strip() # Fallback: last sentence return text.strip().split('\n')[-1] Part 2: Advanced Patterns Tree of Thoughts (ToT) # prompting/tot.py from dataclasses import dataclass from typing import List, Optional import asyncio @dataclass class Thought: content: str score: float = 0.0 parent: Optional['Thought'] = None children: List['Thought'] = None def __post_init__(self): if self.children is None: self.children = [] class TreeOfThoughts: def __init__(self, llm, branching_factor: int = 3, max_depth: int = 3): self.llm = llm self.branching_factor = branching_factor self.max_depth = max_depth async def solve(self, problem: str) -> str: # Generate initial thoughts root = Thought(content=problem) current_level = [root] for depth in range(self.max_depth): next_level = [] for thought in current_level: # Generate branches branches = await self._generate_branches(thought.content) for branch in branches[:self.branching_factor]: child = Thought(content=branch, parent=thought) thought.children.append(child) next_level.append(child) # Evaluate and prune scored = await self._evaluate_thoughts(next_level, problem) scored.sort(key=lambda x: x.score, reverse=True) current_level = scored[:self.branching_factor] # Get best path best = current_level[0] path = [] while best: path.append(best.content) best = best.parent return " → ".join(reversed(path)) async def _generate_branches(self, thought: str) -> List[str]: prompt = f"""Given this partial solution: {thought} Generate {self.branching_factor} different next steps or approaches. Each should be a distinct reasoning direction. Return as numbered list.""" response = await self.llm.ainvoke(prompt) # Parse numbered list return [line.split('. ', 1)[1] for line in response.split('\n') if '. ' in line] async def _evaluate_thoughts(self, thoughts: List[Thought], problem: str) -> List[Thought]: for thought in thoughts: prompt = f"""Problem: {problem} Proposed solution path: {thought.content} Rate the promise of this approach on a scale of 0-10. Consider: correctness, completeness, feasibility. Return only a number.""" response = await self.llm.ainvoke(prompt) try: thought.score = float(response.strip()) except: thought.score = 0.0 return thoughts ReAct (Reasoning + Acting) # prompting/react.py REACT_TEMPLATE = """You are an agent that can use tools to solve problems. Available tools: {tools} Format: Question: {question} Thought: I need to... Action: tool_name Action Input: {{"arg": "value"}} Observation: [tool result] ... (repeat as needed) Thought: I now know the answer Final Answer: {answer} Begin! Question: {question} Thought:""" class ReActPromptBuilder: def __init__(self, tools: list[dict]): self.tools = tools def build(self, question: str) -> str: tool_desc = "\n".join(f"- {t['name']}: {t['description']}" for t in self.tools) return REACT_TEMPLATE.format(tools=tool_desc, question=question) def parse_response(self, response: str) -> dict: import re # Check for final answer final_match = re.search(r"Final Answer:\s*(.+)", response, re.DOTALL) if final_match: return {"type": "final", "content": final_match.group(1).strip()} # Check for action action_match = re.search(r"Action:\s*(\w+)", response) input_match = re.search(r"Action Input:\s*(\{.+\})", response) if action_match and input_match: import json return { "type": "action", "tool": action_match.group(1), "args": json.loads(input_match.group(1)) } return {"type": "thought", "content": response} RAG Prompting with Citations # prompting/rag_prompts.py RAG_PROMPT = """You are a precise assistant that answers questions using ONLY the provided context. ## Context Documents {context} ## Instructions 1. Answer ONLY from the provided context 2. If context is insufficient, say "I don't have enough information to answer this" 3. Cite sources inline using [doc_N] format where N is the document number 4. Be concise but complete 5. If asked for a list, use bullet points ## Question {question} ## Answer:""" RAG_PROMPT_STRUCTURED = """{context} ## Question {question} ## Output Format (JSON) {{ "answer": "Your answer with [doc_N] citations", "citations": [ {{"doc_id": "doc_1", "quote": "exact quote from doc", "relevance": "high"}} ], "confidence": 0.95, "insufficient_context": false }}""" def format_rag_context(docs: list, max_chars: int = 8000) -> str: """Format retrieved documents for RAG prompt.""" parts = [] total = 0 for i, doc in enumerate(docs, 1): content = doc.get('content', '')[:2000] # Limit per doc source = doc.get('metadata', {}).get('source', 'Unknown') part = f"[doc_{i}] Source: {source}\n{content}" if total + len(part) > max_chars: break parts.append(part) total += len(part) return "\n\n---\n\n".join(parts) Part 3: Agent System Prompts Supervisor Agent # prompting/agent_prompts.py SUPERVISOR_PROMPT = """You are a Supervisor Agent managing a team of specialist agents. ## Your Role - Decompose complex user requests into subtasks - Delegate subtasks to appropriate specialists - Synthesize specialist outputs into coherent responses - Track progress and handle failures ## Specialist Agents Available {specialists} ## Delegation Protocol When delegating, use this format: DELEGATE: {{"agent": "agent_name", "task": "specific task description", "context": {{}}}} ## Response Format 1. If task is complete: Provide final synthesized answer 2. If delegation needed: Output DELEGATE command only 3. If clarification needed: Ask specific question ## Current Request {request} ## Previous Delegation Results {history} Your response:""" ### Specialist Agent (Researcher) RESEARCHER_PROMPT = """You are a Research Specialist Agent. Your Capabilities - Web search for current information - Document lookup and summarization - Fact verification and cross-referencing - Technical concept explanation Tools Available {tools} Instructions 1. Break down the research task into specific queries 2. Execute searches in parallel when possible 3. Synthesize findings with citations 4. Flag uncertainties or conflicting sources 5. Return structured results Task {task} Context {context} Your response (use DELEGATE format for tool calls):""" ### Code Generation Agent CODER_PROMPT = """You are a Senior Software Engineer Agent. Your Capabilities - Write production-quality code in multiple languages - Debug and fix existing code - Write tests and documentation - Refactor for performance and clarity - Architecture and design decisions Principles - Write clean, readable, well-documented code - Follow language idioms and best practices - Include error handling and edge cases - Write tests for non-trivial logic - Explain design decisions Tools Available {tools} Task {task} Context (existing code, requirements, constraints) {context} Output Format Provide code in markdown blocks with language tags. Include brief explanation of approach. If using tools, use DELEGATE format. Your response:""" ## Part 4: Prompt Optimization ### Automated Prompt Optimization prompting/optimization.py from dataclasses import dataclass from typing import List, Callable import asyncio @dataclass class PromptCandidate: prompt: str score: float = 0.0 metadata: dict = None class PromptOptimizer: def init( self, llm, evaluator: Callable[[str, str], float], # (prompt, expected) -> score population_size: int = 10, generations: int = 5 ): self.llm = llm self.evaluator = evaluator self.population_size = population_size self.generations = generations async def optimize( self, base_prompt: str, test_cases: List[dict], # {"input": "", "expected": ""} mutation_rate: float = 0.3 ) -> PromptCandidate: # Initialize population population = await self._initialize_population(base_prompt) for gen in range(self.generations): # Evaluate scored = await self._evaluate_population(population, test_cases) scored.sort(key=lambda x: x.score, reverse=True) # Select top performers elite = scored[:self.population_size // 2] # Generate next generation population = elite.copy() while len(population) < self.population_size: parent = elite[0] # Best parent if random.random() < mutation_rate: child = await self._mutate(parent.prompt) else: child = await self._crossover(elite[0].prompt, elite[1].prompt) population.append(PromptCandidate(prompt=child)) return scored[0] async def _initialize_population(self, base: str) -> List[PromptCandidate]: population = [PromptCandidate(prompt=base)] # Generate variations variations = [ "Add more explicit instructions", "Add few-shot examples", "Add chain-of-thought prompting", "Add output format specification", "Add role/persona definition", "Add constraint specifications", "Add step-by-step reasoning", "Add self-verification step", "Add edge case handling", "Add few-shot with diverse examples" ] for var in variations[:self.population_size - 1]: prompt = await self._apply_variation(base, var) population.append(PromptCandidate(prompt=prompt)) return population async def _mutate(self, prompt: str) -> str: mutations = [ "Add 'Think step by step' to the prompt", "Add 'Be concise' instruction", "Add output format specification", "Add 'Verify your answer before responding'", "Add role definition at the start", "Add constraint: no markdown unless asked", "Add 'If uncertain, say so explicitly'", "Restructure into clear sections" ] return await self._apply_variation(prompt, random.choice(mutations)) async def _crossover(self, p1: str, p2: str) -> str: # Combine best parts of two prompts prompt = f"""Combine the best elements of these two prompts: Prompt 1: {p1} Prompt 2: {p2} Create a single improved prompt that incorporates the strengths of both.""" return await self.llm.ainvoke(prompt) async def _apply_variation(self, prompt: str, variation: str) -> str: prompt_text = f"""Improve this prompt by: {variation} Original prompt: {prompt} Return only the improved prompt.""" return await self.llm.ainvoke(prompt_text) ### DSPy-Style Optimization prompting/dspy_style.py from dataclasses import dataclass from typing import List, Any @dataclass class Example: inputs: dict outputs: dict class PromptModule: def init(self, prompt_template: str, llm): self.template = prompt_template self.llm = llm self.demos: List[Example] = [] def add_demo(self, inputs: dict, outputs: dict): self.demos.append(Example(inputs=inputs, outputs=outputs)) def call(self, inputs) -> Any: # Format prompt with demos demo_text = "" for demo in self.demos: demo_text += f"Input: {demo.inputs}\nOutput: {demo.outputs}\n\n" full_prompt = self.template.format(demos=demo_text, **inputs) return self.llm.invoke(full_prompt) def compile(self, trainset: List[Example], metric: Callable): # Bootstrap demos from trainset using metric scored = [(metric(ex.outputs, self(ex.inputs)), ex) for ex in trainset] scored.sort(reverse=True) self.demos = [ex for _, ex in scored[:3]] # Top 3 ## Part 5: Evaluation Frameworks ### Prompt Evaluation Metrics prompting/evaluation.py from dataclasses import dataclass from typing import List, Callable import asyncio @dataclass class PromptMetrics: accuracy: float consistency: float # Self-consistency across runs instruction_following: float format_compliance: float latency_ms: float token_count: int cost_usd: float class PromptEvaluator: def init(self, llm, judge_llm=None): self.llm = llm self.judge = judge_llm or llm async def evaluate( self, prompt: str, test_cases: List[dict], n_runs: int = 3 ) -> PromptMetrics: all_outputs = [] latencies = [] token_counts = [] for tc in test_cases: case_outputs = [] for _ in range(n_runs): start = time.time() output = await self.llm.ainvoke(prompt.format(**tc["input"])) latency = (time.time() - start) * 1000 case_outputs.append(output) latencies.append(latency) token_counts.append(len(output.split())) all_outputs.append(case_outputs) # Accuracy via LLM judge accuracy = await self._judge_accuracy(test_cases, all_outputs) # Consistency: agreement across runs consistency = self._compute_consistency(all_outputs) # Instruction following instruction_following = await self._judge_instruction_following( prompt, test_cases, all_outputs ) # Format compliance format_compliance = self._check_format_compliance(all_outputs, prompt) return PromptMetrics( accuracy=accuracy, consistency=consistency, instruction_following=instruction_following, format_compliance=format_compliance, latency_ms=sum(latencies) / len(latencies), token_count=sum(token_counts) // len(token_counts), cost_usd=self._estimate_cost(token_counts) ) async def _judge_accuracy(self, test_cases, all_outputs) -> float: scores = [] for tc, outputs in zip(test_cases, all_outputs): # Use first output for evaluation output = outputs[0] prompt = f"""Rate the accuracy of this response (0-1). Expected: {tc.get('expected', 'N/A')} Actual: {output} Consider: factual correctness, completeness, relevance. Return only a number between 0 and 1.""" response = await self.judge.ainvoke(prompt) try: scores.append(float(response.strip())) except: scores.append(0.5) return sum(scores) / len(scores) if scores else 0 def _compute_consistency(self, all_outputs) -> float: # Semantic similarity between runs (simplified) agreements = 0 total = 0 for outputs in all_outputs: for i in range(len(outputs)): for j in range(i + 1, len(outputs)): # Simple word overlap similarity words_i = set(outputs[i].lower().split()) words_j = set(outputs[j].lower().split()) if words_i and words_j: overlap = len(words_i & words_j) / len(words_i | words_j) agreements += overlap total += 1 return agreements / total if total > 0 else 1.0 ### A/B Testing Framework prompting/ab_test.py from dataclasses import dataclass from typing import List import random @dataclass class ABTestResult: variant_a: PromptMetrics variant_b: PromptMetrics winner: str # "A", "B", or "tie" confidence: float sample_size: int class ABTester: def init(self, evaluator: PromptEvaluator): self.evaluator = evaluator async def run_test( self, prompt_a: str, prompt_b: str, test_cases: List[dict], min_samples: int = 50 ) -> ABTestResult: # Randomly assign test cases to variants random.shuffle(test_cases) split = len(test_cases) // 2 cases_a = test_cases[:split] cases_b = test_cases[split:] # Evaluate both metrics_a = await self.evaluator.evaluate(prompt_a, cases_a) metrics_b = await self.evaluator.evaluate(prompt_b, cases_b) # Statistical significance (simplified) winner = "A" if metrics_a.accuracy > metrics_b.accuracy else "B" if abs(metrics_a.accuracy - metrics_b.accuracy) < 0.02: winner = "tie" return ABTestResult( variant_a=metrics_a, variant_b=metrics_b, winner=winner, confidence=self._compute_confidence(metrics_a, metrics_b), sample_size=len(test_cases) ) ## Part 6: Production Prompt Management ### Prompt Versioning & Registry prompting/registry.py import sqlite3 import json from dataclasses import dataclass from datetime import datetime from typing import Optional @dataclass class PromptVersion: id: str name: str version: int prompt: str metadata: dict created_at: str parent_id: Optional[str] = None class PromptRegistry: def init(self, db_path: str = "prompts.db"): self.conn = sqlite3.connect(db_path) self._init_db() def _init_db(self): self.conn.execute(""" CREATE TABLE IF NOT EXISTS prompts ( id TEXT PRIMARY KEY, name TEXT, version INTEGER, prompt TEXT, metadata TEXT, created_at TEXT, parent_id TEXT ) """) self.conn.execute(""" CREATE INDEX IF NOT EXISTS idx_name_version ON prompts(name, version) """) self.conn.commit() def save(self, prompt: PromptVersion): self.conn.execute(""" INSERT INTO prompts (id, name, version, prompt, metadata, created_at, parent_id) VALUES (?, ?, ?, ?, ?, ?, ?) """, ( prompt.id, prompt.name, prompt.version, prompt.prompt, json.dumps(prompt.metadata), prompt.created_at, prompt.parent_id )) self.conn.commit() def get_latest(self, name: str) -> Optional[PromptVersion]: cursor = self.conn.execute( "SELECT * FROM prompts WHERE name = ? ORDER BY version DESC LIMIT 1", (name,) ) row = cursor.fetchone() return self._row_to_prompt(row) if row else None def get_version(self, name: str, version: int) -> Optional[PromptVersion]: cursor = self.conn.execute( "SELECT * FROM prompts WHERE name = ? AND version = ?", (name, version) ) row = cursor.fetchone() return self._row_to_prompt(row) if row else None def get_history(self, name: str) -> List[PromptVersion]: cursor = self.conn.execute( "SELECT * FROM prompts WHERE name = ? ORDER BY version", (name,) ) return [self._row_to_prompt(r) for r in cursor.fetchall()] def _row_to_prompt(self, row) -> PromptVersion: return PromptVersion( id=row[0], name=row[1], version=row[2], prompt=row[3], metadata=json.loads(row[4]), created_at=row[5], parent_id=row[6] ) ### Prompt Deployment with Feature Flags prompting/deployment.py from dataclasses import dataclass from typing: Dict, Optional import random @dataclass class PromptConfig: name: str version: int traffic_percentage: float = 100.0 # 0-100 conditions: Dict[str, any] = None # e.g., {"user_tier": "premium"} class PromptDeployment: def init(self, registry: PromptRegistry): self.registry = registry self.configs: Dict[str, PromptConfig] = {} def register(self, config: PromptConfig): self.configs[config.name] = config def get_prompt(self, name: str, context: dict = None) -> str: config = self.configs.get(name) if not config: # Fallback to latest prompt = self.registry.get_latest(name) return prompt.prompt if prompt else "" # Check conditions if config.conditions and context: for key, value in config.conditions.items(): if context.get(key) != value: # Condition not met, get previous version prev = self.registry.get_version(name, config.version - 1) return prev.prompt if prev else "" # Check traffic percentage if config.traffic_percentage < 100: if random.random() * 100 > config.traffic_percentage: prev = self.registry.get_version(name, config.version - 1) return prev.prompt if prev else "" prompt = self.registry.get_version(name, config.version) return prompt.prompt if prompt else "" ## Part 7: Specialized Prompt Patterns ### Structured Output Enforcement prompting/structured_output.py from pydantic import BaseModel, Field from typing import List, Optional, Literal import instructor class StructuredPrompts: @staticmethod def json_schema_prompt(schema: dict, examples: List[dict] = None) -> str: """Generate prompt that enforces JSON schema output.""" schema_str = json.dumps(schema, indent=2) example_str = "" if examples: example_str = "\n\nExamples:\n" for i, ex in enumerate(examples, 1): example_str += f"Example {i}:\n{json.dumps(ex, indent=2)}\n\n" return f"""Output ONLY valid JSON matching this schema: {schema_str} Rules: - No markdown, no explanations, no extra text - All required fields must be present - Types must match exactly - {example_str}""" @staticmethod def constrained_generation_prompt(constraints: List[str]) -> str: """Generate prompt with hard constraints.""" constraint_text = "\n".join(f"- {c}" for c in constraints) return f"""You must follow these constraints STRICTLY: {constraint_text} Violating any constraint makes the output invalid. Think carefully before each token.""" ### Multilingual Prompting prompting/multilingual.py MULTILINGUAL_TEMPLATE = """You are a multilingual assistant. Respond in {target_language}. Task {task} Context {context} Constraints - Respond ONLY in {target_language} - Maintain cultural appropriateness - Use region-appropriate terminology - {additional_constraints} Output """ Language-specific adjustments LANGUAGE_ADJUSTMENTS = { "ja": "Use polite forms (desu/masu). Avoid direct translations.", "zh": "Use Simplified Chinese. Prefer formal written style.", "de": "Use formal 'Sie' form. Compound nouns appropriately.", "es": "Use formal 'usted'. Consider regional variations.", "fr": "Use formal 'vous'. Follow French typography rules.", "ar": "Use Modern Standard Arabic. RTL-aware formatting.", } ## Decision Framework PROMPT PATTERN SELECTION: Simple, deterministic task? ├─→ Yes → Zero-shot with clear template └─→ No ├─→ Need specific format/style? │ ├─→ Yes → Few-shot with examples │ └─→ No │ ├─→ Complex reasoning needed? │ │ ├─→ Yes → Chain-of-Thought or Tree of Thoughts │ │ └─→ No │ │ │ ├─→ Need external info? │ │ └─→ Yes → RAG prompting with citations │ │ │ ├─→ Multi-step with tools? │ │ └─→ Yes → ReAct or Agent system prompt │ │ │ └─→ High-stakes / production? │ └─→ Yes → Optimized + evaluated + versioned Production Checklist Prompt Quality - [ ] Clear role/persona definition - [ ] Explicit task description - [ ] Specific output format requirements - [ ] Constraints and guardrails - [ ] Few-shot examples (3-5 diverse cases) - [ ] Chain-of-thought for reasoning tasks - [ ] Edge case handling instructions - [ ] Self-verification step Evaluation - [ ] Golden test set (50+ cases) - [ ] Automated metrics (accuracy, format, consistency) - [ ] LLM judge for subjective quality - [ ] A/B testing framework - [ ] Regression detection in CI/CD Management - [ ] Version control for all prompts - [ ] Prompt registry with metadata - [ ] Feature flags for gradual rollout - [ ] Canary deployment support - [ ] Rollback capability (< 1 min) - [ ] Usage analytics per prompt version Security - [ ] Input sanitization - [ ] Injection attempt detection - [ ] PII handling instructions - [ ] Output validation - [ ] Rate limiting awareness Resources - Prompt Engineering Guide (OpenAI) - Chain-of-Thought Paper - Tree of Thoughts Paper - ReAct Paper - DSPy Framework - Instructor Library - Prompt Engineering Patterns --- Prompt engineering is not magic — it's systematic communication with a probabilistic system. Treat prompts as code: version, test, evaluate, and deploy with the same rigor. --- 📦 Want the production-ready version? This article covers the fundamentals. The AI Prompt Engineering Mastery System pack goes further — prompt patterns, evaluation harnesses and production prompt systems, plus full source code and deployment configs. → Get AI Prompt Engineering Mastery System Extracted from real production systems, not toy examples.
4 Prompt Engineering Guide Prompting techniques behind every agent with papers, examples & lectures dair-ai/Prompt-Engineering-Guide