outlines
Generates guaranteed-valid structured output from LLMs with Outlines (dottxt.ai), constraining token sampling via finite state machines for JSON schemas, Pydantic models, regex, choice lists, and integer/float types, on Transformers, llama.cpp, vLLM, and limited OpenAI backends. Use when you need output that always parses as valid JSON or matches a regex. Use when extracting typed data into Pydantic models from a local model. Use when classifying text into a fixed set of categories. Use when running batch structured generation on vLLM or Transformers. Use when controlling token sampling at the grammar level. Not for API models needing automatic retries (use Instructor).
Install this skill
Works with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, and 70+ other harnesses.
npx skills add KalarisLabs/research-agent-skills --skill outlinesuvx research-agent-skills install outlinesDocumentation & Instructions
Outlines: Structured Text Generation
When to Use This Skill
Use Outlines when you need to:
- Guarantee valid JSON/XML/code structure during generation
- Use Pydantic models for type-safe outputs
- Support local models (Transformers, llama.cpp, vLLM)
- Maximize inference speed with zero-overhead structured generation
- Generate against JSON schemas automatically
- Control token sampling at the grammar level
Installation
# Base installation
pip install outlines
# With specific backends
pip install outlines transformers # Hugging Face models
pip install outlines llama-cpp-python # llama.cpp
pip install outlines vllm # vLLM for high-throughput
Quick Start
Basic Example: Classification
import outlines
from typing import Literal
# Load model
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate with type constraint
prompt = "Sentiment of 'This product is amazing!': "
generator = outlines.generate.choice(model, ["positive", "negative", "neutral"])
sentiment = generator(prompt)
print(sentiment) # "positive" (guaranteed one of these)
With Pydantic Models
from pydantic import BaseModel
import outlines
class User(BaseModel):
name: str
age: int
email: str
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Generate structured output
prompt = "Extract user: John Doe, 30 years old, john@example.com"
generator = outlines.generate.json(model, User)
user = generator(prompt)
print(user.name) # "John Doe"
print(user.age) # 30
print(user.email) # "john@example.com"
Core Concepts
1. Constrained Token Sampling
Outlines uses Finite State Machines (FSM) to constrain token generation at the logit level.
How it works:
- Convert schema (JSON/Pydantic/regex) to context-free grammar (CFG)
- Transform CFG into Finite State Machine (FSM)
- Filter invalid tokens at each step during generation
- Fast-forward when only one valid token exists
Benefits:
- Zero overhead: Filtering happens at token level
- Speed improvement: Fast-forward through deterministic paths
- Guaranteed validity: Invalid outputs impossible
import outlines
# Pydantic model -> JSON schema -> CFG -> FSM
class Person(BaseModel):
name: str
age: int
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# Behind the scenes:
# 1. Person -> JSON schema
# 2. JSON schema -> CFG
# 3. CFG -> FSM
# 4. FSM filters tokens during generation
generator = outlines.generate.json(model, Person)
result = generator("Generate person: Alice, 25")
2. Structured Generators
Outlines provides specialized generators for different output types.
Choice Generator
# Multiple choice selection
generator = outlines.generate.choice(
model,
["positive", "negative", "neutral"]
)
sentiment = generator("Review: This is great!")
# Result: One of the three choices
JSON Generator
from pydantic import BaseModel
class Product(BaseModel):
name: str
price: float
in_stock: bool
# Generate valid JSON matching schema
generator = outlines.generate.json(model, Product)
product = generator("Extract: iPhone 15, $999, available")
# Guaranteed valid Product instance
print(type(product)) # <class '__main__.Product'>
Regex Generator
# Generate text matching regex
generator = outlines.generate.regex(
model,
r"[0-9]{3}-[0-9]{3}-[0-9]{4}" # Phone number pattern
)
phone = generator("Generate phone number:")
# Result: "555-123-4567" (guaranteed to match pattern)
Integer/Float Generators
# Generate specific numeric types
int_generator = outlines.generate.integer(model)
age = int_generator("Person's age:") # Guaranteed integer
float_generator = outlines.generate.float(model)
price = float_generator("Product price:") # Guaranteed float
3. Model Backends
Outlines supports multiple local and API-based backends.
Transformers (Hugging Face)
import outlines
# Load from Hugging Face
model = outlines.models.transformers(
"microsoft/Phi-3-mini-4k-instruct",
device="cuda" # Or "cpu"
)
# Use with any generator
generator = outlines.generate.json(model, YourModel)
llama.cpp
# Load GGUF model
model = outlines.models.llamacpp(
"./models/llama-3.1-8b-instruct.Q4_K_M.gguf",
n_gpu_layers=35
)
generator = outlines.generate.json(model, YourModel)
vLLM (High Throughput)
# For production deployments
model = outlines.models.vllm(
"meta-llama/Llama-3.1-8B-Instruct",
tensor_parallel_size=2 # Multi-GPU
)
generator = outlines.generate.json(model, YourModel)
OpenAI (Limited Support)
# Basic OpenAI support
model = outlines.models.openai(
"gpt-4o-mini",
api_key="your-api-key"
)
# Note: Some features limited with API models
generator = outlines.generate.json(model, YourModel)
4. Pydantic Integration
Outlines has first-class Pydantic support with automatic schema translation.
Basic Models
from pydantic import BaseModel, Field
class Article(BaseModel):
title: str = Field(description="Article title")
author: str = Field(description="Author name")
word_count: int = Field(description="Number of words", gt=0)
tags: list[str] = Field(description="List of tags")
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, Article)
article = generator("Generate article about AI")
print(article.title)
print(article.word_count) # Guaranteed > 0
Nested Models
class Address(BaseModel):
street: str
city: str
country: str
class Person(BaseModel):
name: str
age: int
address: Address # Nested model
generator = outlines.generate.json(model, Person)
person = generator("Generate person in New York")
print(person.address.city) # "New York"
Enums and Literals
from enum import Enum
from typing import Literal
class Status(str, Enum):
PENDING = "pending"
APPROVED = "approved"
REJECTED = "rejected"
class Application(BaseModel):
applicant: str
status: Status # Must be one of enum values
priority: Literal["low", "medium", "high"] # Must be one of literals
generator = outlines.generate.json(model, Application)
app = generator("Generate application")
print(app.status) # Status.PENDING (or APPROVED/REJECTED)
Common Patterns
Details, code examples and parameter tables: references/common-patterns.md. Read it when this step applies.
Backend Configuration
Details, code examples and parameter tables: references/backend-configuration.md. Read it when this step applies.
Best Practices
1. Use Specific Types
# ✅ Good: Specific types
class Product(BaseModel):
name: str
price: float # Not str
quantity: int # Not str
in_stock: bool # Not str
# ❌ Bad: Everything as string
class Product(BaseModel):
name: str
price: str # Should be float
quantity: str # Should be int
2. Add Constraints
from pydantic import Field
# ✅ Good: With constraints
class User(BaseModel):
name: str = Field(min_length=1, max_length=100)
age: int = Field(ge=0, le=120)
email: str = Field(pattern=r"^[\w\.-]+@[\w\.-]+\.\w+$")
# ❌ Bad: No constraints
class User(BaseModel):
name: str
age: int
email: str
3. Use Enums for Categories
# ✅ Good: Enum for fixed set
class Priority(str, Enum):
LOW = "low"
MEDIUM = "medium"
HIGH = "high"
class Task(BaseModel):
title: str
priority: Priority
# ❌ Bad: Free-form string
class Task(BaseModel):
title: str
priority: str # Can be anything
4. Provide Context in Prompts
# ✅ Good: Clear context
prompt = """
Extract product information from the following text.
Text: iPhone 15 Pro costs $999 and is currently in stock.
Product:
"""
# ❌ Bad: Minimal context
prompt = "iPhone 15 Pro costs $999 and is currently in stock."
5. Handle Optional Fields
from typing import Optional
# ✅ Good: Optional fields for incomplete data
class Article(BaseModel):
title: str # Required
author: Optional[str] = None # Optional
date: Optional[str] = None # Optional
tags: list[str] = [] # Default empty list
# Can succeed even if author/date missing
Comparison to Alternatives
| Feature | Outlines | Instructor | Guidance | LMQL |
|---|---|---|---|---|
| Pydantic Support | ✅ Native | ✅ Native | ❌ No | ❌ No |
| JSON Schema | ✅ Yes | ✅ Yes | ⚠️ Limited | ✅ Yes |
| Regex Constraints | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes |
| Local Models | ✅ Full | ⚠️ Limited | ✅ Full | ✅ Full |
| API Models | ⚠️ Limited | ✅ Full | ✅ Full | ✅ Full |
| Zero Overhead | ✅ Yes | ❌ No | ⚠️ Partial | ✅ Yes |
| Automatic Retrying | ❌ No | ✅ Yes | ❌ No | ❌ No |
| Learning Curve | Low | Low | Low | High |
When to choose Outlines:
- Using local models (Transformers, llama.cpp, vLLM)
- Need maximum inference speed
- Want Pydantic model support
- Require zero-overhead structured generation
- Control token sampling process
When to choose alternatives:
- Instructor: Need API models with automatic retrying
- Guidance: Need token healing and complex workflows
- LMQL: Prefer declarative query syntax
Performance Characteristics
Speed:
- Zero overhead: Structured generation as fast as unconstrained
- Fast-forward optimization: Skips deterministic tokens
- 1.2-2x faster than post-generation validation approaches
Memory:
- FSM compiled once per schema (cached)
- Minimal runtime overhead
- Efficient with vLLM for high throughput
Accuracy:
- 100% valid outputs (guaranteed by FSM)
- No retry loops needed
- Deterministic token filtering
Resources
- Documentation: https://outlines-dev.github.io/outlines
- GitHub: https://github.com/outlines-dev/outlines
- Discord: https://discord.gg/R9DSu34mGd
- Blog: https://blog.dottxt.co
See Also
references/json_generation.md- Comprehensive JSON and Pydantic patternsreferences/backends.md- Backend-specific configurationreferences/examples.md- Production-ready examples
Agent operating procedure
- Check the environment. Confirm the framework version, model provider, API keys and rate limits.
- Pin down the inputs. Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.
- Run a small version first. Test a single call or chain with a known input and inspect raw outputs.
- Execute the full task using the instructions and references above.
- Validate the result. Evaluate on a small labeled set; check structured outputs against their schema; log prompts and responses.
- Report. State what was run (versions, commands, parameters), what was checked, and what is still uncertain.
| If this happens | Do this |
|---|---|
| Outputs do not match the expected schema | Add validation and retries, tighten the schema, or simplify the prompt. |
| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (help(), --help, official docs), adapt, and tell the user. Never invent an API. |
| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |
Integrity rules
- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.
- Never send private or sensitive data to external APIs without the user's consent.
- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.
- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.
Trigger Prompts & Evaluation Corpus
These evaluation prompts test that the agent correctly selects outlines for relevant tasks and leaves it out for near misses.
Should trigger (3)
- should trigger
“How do I force a local Llama model to only output one of 'positive', 'negative', or 'neutral' for my tweets?”
expects Uses Outlines choice/Literal-constrained generation on a local Transformers (or llama.cpp) model so output is always one of the three labels. Shows model loading and a generator call, and notes the guarantee comes from token-level constraint, not retries.
- should trigger
“I'm running Qwen2.5-7B-Instruct via vLLM on our lab GPU box and need to extract patient-free trial metadata from 2,000 abstracts in abstracts.jsonl into a Pydantic model (fields: sample_size int, design enum, primary_endpoint str, funding Optional[str]). My json.loads keeps failing on about 8% of outputs with 'Expecting , delimiter'. How do I make this always parse?”
expects Recommends Outlines with the Pydantic model as the JSON schema on the vLLM backend, with batch generation over the abstracts. Uses Enum/Literal for design and Optional for funding, and runs a small test first and validates outputs against the model. Checks the installed Outlines version's API rather than inventing calls, and notes that syntactically valid JSON does not guarantee semantically correct extraction.
- should trigger
“I have a small local model and need every generated sample ID to look like 'SMP-2024-0001' (three letters, year, four digits), with no malformed IDs ever coming out. Can I enforce this at decoding time instead of filtering afterward?”
expects Proposes grammar-level constrained decoding with Outlines using a regex generator, e.g. SMP-\d{4}-\d{4} or a stricter pattern, on a local backend. Explains that invalid tokens are masked during sampling, so no post-filtering is needed. Suggests a quick test on a few samples.
Near misses (2)
- near miss
“I'm calling the Claude API to extract invoice fields into Pydantic models, and sometimes validation fails. I want it to automatically retry with the validation error fed back to the model. What should I use?”
expects Recommends Instructor, not Outlines, because this is an API model that needs automatic retries with validation-error feedback. Shows a Pydantic response_model with max_retries.
- near miss
“I want to write a Guidance program with a context-free grammar and token healing so the model fills in a template for a chemistry lab report, running through the OpenAI backend.”
expects Points to Guidance, which provides CFG-based constraints, token healing, and @guidance functions, rather than Outlines. Does not use Outlines.