Skip to content
Research Agent Skillsby Kalaris Labs
Install
LLM Applicationsv1.0.0AdaptedMIT

dspy

Builds and optimizes language model programs with DSPy (Stanford NLP), using Signatures, modules (Predict, ChainOfThought, ReAct, ProgramOfThought) and optimizers (BootstrapFewShot, MIPRO, BootstrapFinetune). Use when replacing hand-written prompts with declarative signatures, when automatically tuning prompts against training data and a metric, when building multi-stage RAG pipelines or ReAct agents, when building classifiers or structured-output programs, or when saving and evaluating optimized LM programs across providers such as Anthropic, OpenAI, or Ollama. Not for quick prototypes or simple chains; use manual prompting or LangChain instead.

Install this skill

Works with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, and 70+ other harnesses.

skills.sh CLI (Recommended)
npx skills add KalarisLabs/research-agent-skills --skill dspy
uvx (Python tool runner)
uvx research-agent-skills install dspy

Documentation & Instructions

DSPy: Declarative Language Model Programming

When to Use This Skill

Use DSPy when you need to:

  • Build complex AI systems with multiple components and workflows
  • Program LMs declaratively instead of manual prompt engineering
  • Optimize prompts automatically using data-driven methods
  • Create modular AI pipelines that are maintainable and portable
  • Improve model outputs systematically with optimizers
  • Build RAG systems, agents, or classifiers with better reliability

Installation

# Stable release
pip install dspy

# Latest development version
pip install git+https://github.com/stanfordnlp/dspy.git

# With specific LM providers
pip install dspy[openai]        # OpenAI
pip install dspy[anthropic]     # Anthropic Claude
pip install dspy[all]           # All providers

Quick Start

Basic Example: Question Answering

import dspy

# Configure your language model
lm = dspy.Claude(model="claude-sonnet-4-5-20250929")
dspy.settings.configure(lm=lm)

# Define a signature (input → output)
class QA(dspy.Signature):
    """Answer questions with short factual answers."""
    question = dspy.InputField()
    answer = dspy.OutputField(desc="often between 1 and 5 words")

# Create a module
qa = dspy.Predict(QA)

# Use it
response = qa(question="What is the capital of France?")
print(response.answer)  # "Paris"

Chain of Thought Reasoning

import dspy

lm = dspy.Claude(model="claude-sonnet-4-5-20250929")
dspy.settings.configure(lm=lm)

# Use ChainOfThought for better reasoning
class MathProblem(dspy.Signature):
    """Solve math word problems."""
    problem = dspy.InputField()
    answer = dspy.OutputField(desc="numerical answer")

# ChainOfThought generates reasoning steps automatically
cot = dspy.ChainOfThought(MathProblem)

response = cot(problem="If John has 5 apples and gives 2 to Mary, how many does he have?")
print(response.rationale)  # Shows reasoning steps
print(response.answer)     # "3"

Core Concepts

1. Signatures

Signatures define the structure of your AI task (inputs → outputs):

# Inline signature (simple)
qa = dspy.Predict("question -> answer")

# Class signature (detailed)
class Summarize(dspy.Signature):
    """Summarize text into key points."""
    text = dspy.InputField()
    summary = dspy.OutputField(desc="bullet points, 3-5 items")

summarizer = dspy.ChainOfThought(Summarize)

When to use each:

  • Inline: Quick prototyping, simple tasks
  • Class: Complex tasks, type hints, better documentation

2. Modules

Modules are reusable components that transform inputs to outputs:

dspy.Predict

Basic prediction module:

predictor = dspy.Predict("context, question -> answer")
result = predictor(context="Paris is the capital of France",
                   question="What is the capital?")

dspy.ChainOfThought

Generates reasoning steps before answering:

cot = dspy.ChainOfThought("question -> answer")
result = cot(question="Why is the sky blue?")
print(result.rationale)  # Reasoning steps
print(result.answer)     # Final answer

dspy.ReAct

Agent-like reasoning with tools:

from dspy.predict import ReAct

class SearchQA(dspy.Signature):
    """Answer questions using search."""
    question = dspy.InputField()
    answer = dspy.OutputField()

def search_tool(query: str) -> str:
    """Search Wikipedia."""
    # Your search implementation
    return results

react = ReAct(SearchQA, tools=[search_tool])
result = react(question="When was Python created?")

dspy.ProgramOfThought

Generates and executes code for reasoning:

pot = dspy.ProgramOfThought("question -> answer")
result = pot(question="What is 15% of 240?")
# Generates: answer = 240 * 0.15

3. Optimizers

Optimizers improve your modules automatically using training data:

BootstrapFewShot

Learns from examples:

from dspy.teleprompt import BootstrapFewShot

# Training data
trainset = [
    dspy.Example(question="What is 2+2?", answer="4").with_inputs("question"),
    dspy.Example(question="What is 3+5?", answer="8").with_inputs("question"),
]

# Define metric
def validate_answer(example, pred, trace=None):
    return example.answer == pred.answer

# Optimize
optimizer = BootstrapFewShot(metric=validate_answer, max_bootstrapped_demos=3)
optimized_qa = optimizer.compile(qa, trainset=trainset)

# Now optimized_qa performs better!

MIPRO (Most Important Prompt Optimization)

Iteratively improves prompts:

from dspy.teleprompt import MIPRO

optimizer = MIPRO(
    metric=validate_answer,
    num_candidates=10,
    init_temperature=1.0
)

optimized_cot = optimizer.compile(
    cot,
    trainset=trainset,
    num_trials=100
)

BootstrapFinetune

Creates datasets for model fine-tuning:

from dspy.teleprompt import BootstrapFinetune

optimizer = BootstrapFinetune(metric=validate_answer)
optimized_module = optimizer.compile(qa, trainset=trainset)

# Exports training data for fine-tuning

4. Building Complex Systems

Multi-Stage Pipeline

import dspy

class MultiHopQA(dspy.Module):
    def __init__(self):
        super().__init__()
        self.retrieve = dspy.Retrieve(k=3)
        self.generate_query = dspy.ChainOfThought("question -> search_query")
        self.generate_answer = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question):
        # Stage 1: Generate search query
        search_query = self.generate_query(question=question).search_query

        # Stage 2: Retrieve context
        passages = self.retrieve(search_query).passages
        context = "\n".join(passages)

        # Stage 3: Generate answer
        answer = self.generate_answer(context=context, question=question).answer
        return dspy.Prediction(answer=answer, context=context)

# Use the pipeline
qa_system = MultiHopQA()
result = qa_system(question="Who wrote the book that inspired the movie Blade Runner?")

RAG System with Optimization

import dspy
from dspy.retrieve.chromadb_rm import ChromadbRM

# Configure retriever
retriever = ChromadbRM(
    collection_name="documents",
    persist_directory="./chroma_db"
)

class RAG(dspy.Module):
    def __init__(self, num_passages=3):
        super().__init__()
        self.retrieve = dspy.Retrieve(k=num_passages)
        self.generate = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question):
        context = self.retrieve(question).passages
        return self.generate(context=context, question=question)

# Create and optimize
rag = RAG()

# Optimize with training data
from dspy.teleprompt import BootstrapFewShot

optimizer = BootstrapFewShot(metric=validate_answer)
optimized_rag = optimizer.compile(rag, trainset=trainset)

LM Provider Configuration

Details, code examples and parameter tables: references/lm-provider-configuration.md. Read it when this step applies.

Common Patterns

Details, code examples and parameter tables: references/common-patterns.md. Read it when this step applies.

Evaluation and Metrics

Custom Metrics

def exact_match(example, pred, trace=None):
    """Exact match metric."""
    return example.answer.lower() == pred.answer.lower()

def f1_score(example, pred, trace=None):
    """F1 score for text overlap."""
    pred_tokens = set(pred.answer.lower().split())
    gold_tokens = set(example.answer.lower().split())

    if not pred_tokens:
        return 0.0

    precision = len(pred_tokens & gold_tokens) / len(pred_tokens)
    recall = len(pred_tokens & gold_tokens) / len(gold_tokens)

    if precision + recall == 0:
        return 0.0

    return 2 * (precision * recall) / (precision + recall)

Evaluation

from dspy.evaluate import Evaluate

# Create evaluator
evaluator = Evaluate(
    devset=testset,
    metric=exact_match,
    num_threads=4,
    display_progress=True
)

# Evaluate model
score = evaluator(qa_system)
print(f"Accuracy: {score}")

# Compare optimized vs unoptimized
score_before = evaluator(qa)
score_after = evaluator(optimized_qa)
print(f"Improvement: {score_after - score_before:.2%}")

Best Practices

1. Start Simple, Iterate

# Start with Predict
qa = dspy.Predict("question -> answer")

# Add reasoning if needed
qa = dspy.ChainOfThought("question -> answer")

# Add optimization when you have data
optimized_qa = optimizer.compile(qa, trainset=data)

2. Use Descriptive Signatures

# ❌ Bad: Vague
class Task(dspy.Signature):
    input = dspy.InputField()
    output = dspy.OutputField()

# ✅ Good: Descriptive
class SummarizeArticle(dspy.Signature):
    """Summarize news articles into 3-5 key points."""
    article = dspy.InputField(desc="full article text")
    summary = dspy.OutputField(desc="bullet points, 3-5 items")

3. Optimize with Representative Data

# Create diverse training examples
trainset = [
    dspy.Example(question="factual", answer="...).with_inputs("question"),
    dspy.Example(question="reasoning", answer="...").with_inputs("question"),
    dspy.Example(question="calculation", answer="...").with_inputs("question"),
]

# Use validation set for metric
def metric(example, pred, trace=None):
    return example.answer in pred.answer

4. Save and Load Optimized Models

# Save
optimized_qa.save("models/qa_v1.json")

# Load
loaded_qa = dspy.ChainOfThought("question -> answer")
loaded_qa.load("models/qa_v1.json")

5. Monitor and Debug

# Enable tracing
dspy.settings.configure(lm=lm, trace=[])

# Run prediction
result = qa(question="...")

# Inspect trace
for call in dspy.settings.trace:
    print(f"Prompt: {call['prompt']}")
    print(f"Response: {call['response']}")

Comparison to Other Approaches

Feature Manual Prompting LangChain DSPy
Prompt Engineering Manual Manual Automatic
Optimization Trial & error None Data-driven
Modularity Low Medium High
Type Safety No Limited Yes (Signatures)
Portability Low Medium High
Learning Curve Low Medium Medium-High

When to choose DSPy:

  • You have training data or can generate it
  • You need systematic prompt improvement
  • You're building complex multi-stage systems
  • You want to optimize across different LMs

When to choose alternatives:

  • Quick prototypes (manual prompting)
  • Simple chains with existing tools (LangChain)
  • Custom optimization logic needed

Resources

See Also

  • references/modules.md - Detailed module guide (Predict, ChainOfThought, ReAct, ProgramOfThought)
  • references/optimizers.md - Optimization algorithms (BootstrapFewShot, MIPRO, BootstrapFinetune)
  • references/examples.md - Real-world examples (RAG, agents, classifiers)

Agent operating procedure

  1. Check the environment. Confirm the framework version, model provider, API keys and rate limits.
  2. Pin down the inputs. Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.
  3. Run a small version first. Test a single call or chain with a known input and inspect raw outputs.
  4. Execute the full task using the instructions and references above.
  5. Validate the result. Evaluate on a small labeled set; check structured outputs against their schema; log prompts and responses.
  6. Report. State what was run (versions, commands, parameters), what was checked, and what is still uncertain.
If this happens Do this
Outputs do not match the expected schema Add validation and retries, tighten the schema, or simplify the prompt.
A function, flag or endpoint in these instructions is missing in the installed version Check the installed version's own documentation (help(), --help, official docs), adapt, and tell the user. Never invent an API.
A required input, identifier or parameter is ambiguous Ask the user, or state the assumption explicitly before running.

Integrity rules

  • Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.
  • Never send private or sensitive data to external APIs without the user's consent.
  • Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.
  • Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.

Trigger Prompts & Evaluation Corpus

These evaluation prompts test that the agent correctly selects dspy for relevant tasks and leaves it out for near misses.

Should trigger (3)

  • should trigger

    “How do I turn a dspy Signature into an optimized program?”

    expects Explains defining a Signature (inline or class), wrapping it in a module such as Predict or ChainOfThought, then compiling with an optimizer (e.g. BootstrapFewShot) using a trainset and a metric. Notes that the installed DSPy version and LM configuration should be checked, and does not invent APIs.

  • should trigger

    “I have a hand-written few-shot prompt in prompts/triage.txt that classifies 2,000 clinical notes in notes_labeled.csv into 6 urgency classes with claude-sonnet. Accuracy is stuck around 71%. I'd like to port it to DSPy and use MIPROv2 to tune it, and I keep getting 'AttributeError: module dspy has no attribute OpenAI' when I configure the LM. Can you help?”

    expects Diagnoses the error as an outdated API (newer DSPy uses dspy.LM('anthropic/...') with dspy.configure) and advises checking the installed version. Builds a class Signature with typed output for the urgency label, splits the CSV into train/dev Examples, defines an accuracy metric, runs MIPRO, and evaluates on held-out data. Asks for consent before sending patient data to an external API and before spending on optimization runs.

  • should trigger

    “I'm building a question-answering system over our lab's 5,000 protocol PDFs. The retrieval step then answer generation needs to be tuned automatically against about 150 question/answer pairs I wrote, so I don't have to keep rewriting prompts by hand every time we swap the underlying model. What's a good approach?”

    expects Recommends a declarative multi-stage RAG program (retrieve then generate with ChainOfThought) in DSPy, with a metric over the 150 labeled pairs and an optimizer such as BootstrapFewShot or MIPRO to compile it. Explains that re-optimizing on a new LM replaces manual prompt rewriting, and suggests saving and evaluating the compiled program on a held-out split.

Near misses (2)

  • near miss

    “Set up a LangChain agent with a ReAct loop, a Tavily search tool, and ConversationBufferMemory so my chatbot can answer questions about recent papers.”

    expects Should use the langchain skill, not DSPy. The answer builds a LangChain agent with tool calling, search tool and memory, with no automatic prompt optimization.

  • near miss

    “Write me a quick one-off prompt that asks GPT-4 to summarize this abstract into three bullet points. I just need it once.”

    expects Gives a simple manual prompt directly. It does not set up DSPy signatures or optimizers, because a one-off task needs no optimization framework.