lasta/AI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

참고소스 수정본

Browse Source
This commit is contained in:
LASTA_DEV01\lasta
2026-05-12 19:40:31 +09:00
parent 0f34a451fc
commit 2e9204243d
8708 changed files with 3259488 additions and 869 deletions
Download Patch File Download Diff File Expand all files Collapse all files

673
참고/instructor-main/docs/integrations/anthropic.md Normal file
View File

@@ -0,0 +1,673 @@
---
title: "Anthropic Claude Tutorial: Structured Outputs with Instructor"
description: "Complete guide to using Anthropic's Claude models with Instructor for structured data extraction. Learn how to use Claude Haiku for type-safe outputs in Python."
---
## See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
- [Mode Comparison](../modes-comparison.md) - Using Anthropic's tool calling
# Anthropic Claude Tutorial: Structured Outputs with Instructor
Learn how to use Anthropic's Claude Haiku models with Instructor to extract structured, validated data. This tutorial covers everything from basic setup to advanced patterns for production use.
## Quick Start: Install Instructor for Claude
Get started with Claude and Instructor for structured outputs:
```
pip install "instructor[anthropic]"
```
Once we've done so, getting started is as simple as using our `from_provider` method to patch the client up.
### Basic Usage
```python
# Standard library imports
import os
from typing import List
# Third-party imports
import anthropic
import instructor
from pydantic import BaseModel, Field
# Set up environment (typically handled before script execution)
# os.environ["ANTHROPIC_API_KEY"] = "your-api-key" # Uncomment and replace with your API key if not set
# Define your models with proper type annotations
class Properties(BaseModel):
"""Model representing a key-value property."""
name: str = Field(description="The name of the property")
value: str = Field(description="The value of the property")
class User(BaseModel):
"""Model representing a user with properties."""
name: str = Field(description="The user's full name")
age: int = Field(description="The user's age in years")
properties: List[Properties] = Field(description="List of user properties")
client = instructor.from_provider(
"anthropic/claude-4-5-haiku-latest",
mode=instructor.Mode.TOOLS
)
try:
# Extract structured data
user_response = client.create(
max_tokens=1024,
messages=[
{
"role": "system",
"content": "Extract structured information based on the user's request."
},
{
"role": "user",
"content": "Create a user for a model with a name, age, and properties.",
}
],
response_model=User,
)
# Print the result as formatted JSON
print(user_response.model_dump_json(indent=2))
# Expected output:
# {
# "name": "John Doe",
# "age": 35,
# "properties": [
# {
# "name": "City",
# "value": "New York"
# },
# {
# "name": "Occupation",
# "value": "Software Engineer"
# }
# ]
# }
except instructor.exceptions.InstructorError as e:
print(f"Validation error: {e}")
except Exception as e:
print(f"Unexpected error: {e}")
```
### Async Example
```python
import asyncio
async_client = instructor.from_provider(
"anthropic/claude-4-5-haiku-latest",
async_client=True,
mode=instructor.Mode.TOOLS,
)
async def extract_user():
return await async_client.create(
messages=[{"role": "user", "content": "Extract: Jason is 25 years old"}],
response_model=User,
)
user = asyncio.run(extract_user())
print(user)
```
### Parallel Tool Calling
Parallel tool mode is automatically detected when your response model is `Iterable[Union[Model1, Model2, ...]]`. Just use `Mode.TOOLS` (or let it default) and the handler will automatically:
- Set tool_choice to "auto" (required for parallel)
- Generate schemas for all union members
- Return a generator yielding each tool result
```python
from typing import Iterable, Literal
from pydantic import BaseModel
import instructor
class Weather(BaseModel):
location: str
units: Literal["imperial", "metric"]
class GoogleSearch(BaseModel):
query: str
# No need to specify Mode.PARALLEL_TOOLS - it's auto-detected!
client = instructor.from_provider(
"anthropic/claude-3-5-haiku-latest",
mode=instructor.Mode.TOOLS, # or just omit and use default
)
results = client.create(
messages=[
{"role": "system", "content": "You must always use tools"},
{
"role": "user",
"content": "What is the weather in toronto and dallas and who won the super bowl?",
},
],
response_model=Iterable[Weather | GoogleSearch], # Auto-detects parallel mode
)
for item in results:
print(item)
```
**How it works**: When Instructor detects `Iterable[Union[...]]`, it automatically:
1. Sets `tool_choice` to `"auto"` (allows model to call any tool)
2. Generates tool schemas from all union members
3. Returns a generator that yields each extracted tool call
4. Each yielded item is validated against its corresponding Pydantic model
## Multimodal
> We've provided a few different sample files for you to use to test out these new features. All examples below use these files.
>
> - (Image) : An image of some blueberry plants [image.jpg](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg)
> - (PDF) : A sample PDF file which contains a fake invoice [invoice.pdf](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf)
Instructor provides a unified, provider-agnostic interface for working with multimodal inputs like images, PDFs, and audio files. With Instructor's multimodal objects, you can easily load media from URLs, local files, or base64 strings using a consistent API that works across different AI providers (OpenAI, Anthropic, Mistral, etc.).
Instructor handles all the provider-specific formatting requirements behind the scenes, ensuring your code remains clean and future-proof as provider APIs evolve.
Let's see how to use the Image and PDF classes.
### Image
> For a more in-depth walkthrough of the Image component, check out the [docs here](../concepts/multimodal.md)
Instructor makes it easy to analyse and extract semantic information from images using Anthropic's claude models. [Click here](https://docs.anthropic.com/en/docs/about-claude/models/all-models) to check if the model you'd like to use has vison capabilities.
Let's see an example below with the sample image above where we'll load it in using our `from_url` method.
Note that we support local files and base64 strings too with the `from_path` and the `from_base64` class methods.
```python
from instructor.processing.multimodal import Image
from pydantic import BaseModel, Field
import instructor
from anthropic import Anthropic
class ImageDescription(BaseModel):
objects: list[str] = Field(..., description="The objects in the image")
scene: str = Field(..., description="The scene of the image")
colors: list[str] = Field(..., description="The colors in the image")
client = instructor.from_provider("anthropic/claude-4-5-haiku-latest")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg"
# Multiple ways to load an image:
response = client.create(
response_model=ImageDescription,
max_tokens=1000,
messages=[
{
"role": "user",
"content": [
"What is in this image?",
# Option 1: Direct URL with autodetection
Image.from_url(url),
# Option 2: Local file
# Image.from_path("path/to/local/image.jpg")
# Option 3: Base64 string
# Image.from_base64("base64_encoded_string_here")
# Option 4: Autodetect
# Image.autodetect(<url|path|base64>)
],
},
],
)
print(response)
# Example output:
# ImageDescription(
# objects=['blueberries', 'leaves'],
# scene='A blueberry bush with clusters of ripe blueberries and some unripe ones against a cloudy sky',
# colors=['green', 'blue', 'purple', 'white']
# )
```
### PDF
Instructor makes it easy to analyse and extract semantic information from PDFs using Anthropic's Claude line of models.
Let's see an example below with the sample PDF above where we'll load it in using our `from_url` method.
Note that we support local files and base64 strings too with the `from_path` and the `from_base64` class methods.
```python
from instructor.processing.multimodal import PDF
from pydantic import BaseModel, Field
import instructor
from anthropic import Anthropic
class Receipt(BaseModel):
total: int
items: list[str]
client = instructor.from_provider("anthropic/claude-4-5-haiku-latest")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
# Multiple ways to load an PDF:
response = client.create(
response_model=Receipt,
max_tokens=1000,
messages=[
{
"role": "user",
"content": [
"Extract out the total and line items from the invoice",
# Option 1: Direct URL
PDF.from_url(url),
# Option 2: Local file
# PDF.from_path("path/to/local/invoice.pdf"),
# Option 3: Base64 string
# PDF.from_base64("base64_encoded_string_here")
# Option 4: Autodetect
# PDF.autodetect(<url|path|base64>)
],
},
],
)
print(response)
# > Receipt(total=220, items=['English Tea', 'Tofu'])
```
If you'd like to cache the PDF and use it across multiple different requests, we support that with the `PdfWithCacheControl` class which we can see below.
```python
from instructor.processing.multimodal import PdfWithCacheControl
from pydantic import BaseModel
import instructor
from anthropic import Anthropic
class Receipt(BaseModel):
total: int
items: list[str]
client = instructor.from_provider("anthropic/claude-4-5-haiku-latest")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
# Multiple ways to load an PDF:
response, completion = client.create_with_completion(
response_model=Receipt,
max_tokens=1000,
messages=[
{
"role": "user",
"content": [
"Extract out the total and line items from the invoice",
# Option 1: Direct URL
PdfWithCacheControl.from_url(url),
# Option 2: Local file
# PDF.from_path("path/to/local/invoice.pdf"),
# Option 3: Base64 string
# PDF.from_base64("base64_encoded_string_here")
# Option 4: Autodetect
# PDF.autodetect(<url|path|base64>)
],
},
],
)
assert (
completion.usage.cache_creation_input_tokens > 0
or completion.usage.cache_read_input_tokens > 0
)
print(response)
# > Receipt(total=220, items=['English Tea', 'Tofu'])
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
### Partials
You can use our `create_partial` method to stream a single object. Note that validators should not be declared in the response model when streaming objects because it will break the streaming process.
```python
# Standard library imports
import os
# Third-party imports
import anthropic
import instructor
from pydantic import BaseModel, Field
# Set up environment (typically handled before script execution)
# os.environ["ANTHROPIC_API_KEY"] = "your-api-key" # Uncomment and replace with your API key if not set
# Initialize client with explicit mode
client = instructor.from_provider(
"anthropic/claude-4-5-haiku-latest",
mode=instructor.Mode.TOOLS,
)
# Define your model with proper annotations
class User(BaseModel):
"""Model representing a user profile."""
name: str = Field(description="The user's full name")
age: int = Field(description="The user's age in years")
bio: str = Field(description="A biographical description of the user")
try:
# Stream partial objects as they're generated
for partial_user in client.create_partial(
messages=[
{"role": "system", "content": "Create a detailed user profile based on the information provided."},
{"role": "user", "content": "Create a user profile for Jason, age 25"},
],
response_model=User,
max_tokens=4096,
):
print(f"Current state: {partial_user}")
# Expected output:
# > Current state: name='Jason' age=None bio=None
# > Current state: name='Jason' age=25 bio='Jason is a 25-year-old with an adventurous spirit and a love for technology. He is'
# > Current state: name='Jason' age=25 bio='Jason is a 25-year-old with an adventurous spirit and a love for technology. He is always on the lookout for new challenges and opportunities to grow both personally and professionally.'
except Exception as e:
print(f"Error during streaming: {e}")
```
### Iterable Example
You can also use our `create_iterable` method to stream a list of objects. This is helpful when you'd like to extract multiple instances of the same response model from a single prompt.
```python
# Standard library imports
import os
# Third-party imports
import anthropic
from instructor import from_provider
from pydantic import BaseModel, Field
# Set up environment (typically handled before script execution)
# os.environ["ANTHROPIC_API_KEY"] = "your-api-key" # Uncomment and replace with your API key if not set
# Initialize client with explicit mode
client = from_provider(
mode=instructor.Mode.TOOLS
)
# Define your model with proper annotations
class User(BaseModel):
"""Model representing a basic user."""
name: str = Field(description="The user's full name")
age: int = Field(description="The user's age in years")
try:
# Create an iterable of user objects
users = client.create_iterable(
messages=[
{
"role": "system",
"content": "Extract all users from the provided text into structured format."
},
{
"role": "user",
"content": """
Extract users:
1. Jason is 25 years old
2. Sarah is 30 years old
3. Mike is 28 years old
""",
},
],
max_tokens=4096,
response_model=User,
)
# Process each user as it's extracted
for user in users:
print(user)
# Expected output:
# > name='Jason' age=25
# > name='Sarah' age=30
# > name='Mike' age=28
except Exception as e:
print(f"Error during iteration: {e}")
```
## Instructor Modes
We provide several modes to make it easy to work with the different response models that Anthropic supports
1. `instructor.Mode.JSON` : This uses the text completion API from the Anthropic API and then extracts out the desired response model from the text completion model
2. `instructor.Mode.TOOLS` : This uses Anthropic's [tools calling API](https://docs.anthropic.com/en/docs/build-with-claude/tool-use) to return structured outputs. Automatically detects parallel tools from `Iterable[Union[...]]` response models.
3. `instructor.Mode.PARALLEL_TOOLS` : **Deprecated** - Use `Mode.TOOLS` with `Iterable[Union[Model1, Model2, ...]]` instead. Auto-detected automatically.
### Mode Auto-Detection
`Mode.TOOLS` now intelligently adapts based on your response model and parameters:
| Response Model | Parameters | Behavior |
|---|---|---|
| `Model` | Regular | Single tool (forced) |
| `Model` | `thinking={...}` | Single tool with extended thinking (auto) |
| `Iterable[Union[Model1, Model2]]` | Regular | Parallel tools (auto) |
| `Iterable[Union[Model1, Model2]]` | `thinking={...}` | Parallel with thinking |
In general, we recommend using `Mode.TOOLS` because it automatically handles all these cases and is the best way to ensure you have the desired response schema.
## Caching
If you'd like to use caching with the Anthropic Client, we also support it for images and text input.
### Caching Text Input
Here's how you can implement caching for text input ( assuming you have a giant `book.txt` file that you read in).
We've written a comprehensive walkthrough of how to use caching to implement Anthropic's new Contextual Retrieval method that gives a significant bump to retrieval accuracy.
```python
# Standard library imports
import os
# Third-party imports
import instructor
from anthropic import Anthropic
from pydantic import BaseModel, Field
# Set up environment (typically handled before script execution)
# os.environ["ANTHROPIC_API_KEY"] = "your-api-key" # Uncomment and replace with your API key if not set
# Define your Pydantic model with proper annotations
class Character(BaseModel):
"""Model representing a character extracted from text."""
name: str = Field(description="The character's full name")
description: str = Field(description="A description of the character")
# Initialize client with explicit mode and prompt caching
client = instructor.from_provider(
"anthropic/claude-4-5-haiku-latest",
mode=instructor.Mode.TOOLS,
)
try:
# Load your large context
with open("./book.txt", "r") as f:
book = f.read()
# Make multiple calls using the cached context
for _ in range(2):
# The first time processes the large text, subsequent calls use the cache
resp, completion = client.create_with_completion(
messages=[
{
"role": "system",
"content": "Extract character information from the provided text."
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "<book>" + book + "</book>",
"cache_control": {"type": "ephemeral"}, # Mark for caching
},
{
"type": "text",
"text": "Extract a character from the text given above",
},
],
},
],
response_model=Character,
max_tokens=1000,
)
# Process the result
print(f"Character: {resp.name}")
print(f"Description: {resp.description}")
# The completion contains the raw response
print(f"Raw completion length: {len(completion)}")
# Note: Second iteration should be faster due to cache hit
except Exception as e:
print(f"Error: {e}")
```
### Caching Images
We also support caching for images. This helps significantly, especially if you're using images repeatedly to save on costs. Read more about it [here](../concepts/caching.md)
```python
# Standard library imports
import os
# Third-party imports
import instructor
from anthropic import Anthropic
from pydantic import BaseModel, Field
# Set up environment (typically handled before script execution)
# os.environ["ANTHROPIC_API_KEY"] = "your-api-key" # Uncomment and replace with your API key if not set
# Define your model for image analysis
class ImageAnalyzer(BaseModel):
"""Model for analyzing image content."""
content_description: str = Field(description="Description of what appears in the images")
objects: list[str] = Field(description="List of objects visible in the images")
scene_type: str = Field(description="Type of scene shown in the images (indoor, outdoor, etc.)")
# Initialize client with explicit mode and image caching enabled
client = instructor.from_provider(
"anthropic/claude-4-5-haiku-latest",
mode=instructor.Mode.TOOLS,
)
try:
# Configure cache control for images
cache_control = {"type": "ephemeral"}
# Make a request with cached images
response = client.create(
response_model=ImageAnalyzer,
messages=[
{
"role": "system",
"content": "Analyze the content of the provided images in detail."
},
{
"role": "user",
"content": [
"What is in these two images?",
# Remote image with caching
{
"type": "image",
"source": "https://example.com/image.jpg",
"cache_control": cache_control
},
# Local image with caching
{
"type": "image",
"source": "path/to/image.jpg",
"cache_control": cache_control
},
]
}
],
autodetect_images=True # Automatically handle image content
)
# Process the results
print(f"Description: {response.content_description}")
print(f"Objects: {', '.join(response.objects)}")
print(f"Scene type: {response.scene_type}")
# Subsequent identical requests will use cached images
except Exception as e:
print(f"Error during image analysis: {e}")
```
## Thinking (Extended Thinking)
Anthropic supports extended thinking with their Claude models, enabling the model to think through complex problems before providing structured outputs. In Instructor, use `Mode.TOOLS` with the `thinking` parameter to enable this feature.
### Using Extended Thinking with TOOLS
```python
from anthropic import Anthropic
import instructor
from pydantic import BaseModel
class Answer(BaseModel):
answer: float
client = instructor.from_provider("anthropic/claude-3-5-haiku-latest")
response = client.create(
response_model=Answer,
messages=[
{
"role": "user",
"content": "Which is larger, 9.11 or 9.8?",
},
],
temperature=1,
max_tokens=2000,
thinking={"type": "enabled", "budget_tokens": 1024},
)
# Response is a validated Answer object
assert isinstance(response, Answer)
assert response.answer == 9.8
```
### How It Works
When you provide the `thinking` parameter with `type: "enabled"`:
1. **Automatic Mode Detection**: `Mode.TOOLS` automatically detects the thinking parameter and adjusts the tool choice strategy to `auto` (required by Anthropic's API when thinking is enabled)
2. **Model Reasoning**: Claude uses the allocated `budget_tokens` to reason about the problem
3. **Structured Output**: After reasoning, the model returns a valid tool call with your response model
4. **Validation**: The response is automatically validated against your Pydantic model
### Deprecation Notice
`Mode.ANTHROPIC_REASONING_TOOLS` is deprecated. Use `Mode.TOOLS` with the `thinking` parameter instead. Both modes now support thinking, but using the standard `TOOLS` mode is preferred and more flexible.

96
참고/instructor-main/docs/integrations/anyscale.md Normal file
View File

@@ -0,0 +1,96 @@
---
title: Anyscale
description: Guide to using instructor with Anyscale
---
# Structured outputs with Anyscale, a complete guide w/ instructor
[Anyscale](https://www.anyscale.com/) is a platform that provides access to various open-source LLMs like Mistral and Llama models. This guide shows how to use instructor with Anyscale to get structured outputs from these models.
## Quick Start
First, install the required packages:
```bash
pip install instructor
```
You'll need an Anyscale API key which you can set as an environment variable:
```bash
export ANYSCALE_API_KEY=your_api_key_here
```
## Basic Example
Here's how to extract structured data from Anyscale models:
```python
import instructor
from pydantic import BaseModel
# Initialize the client with Anyscale base URL
client = instructor.from_provider(
"anyscale/Mixtral-8x7B-Instruct-v0.1",
mode=instructor.Mode.JSON_SCHEMA,
)
class UserExtract(BaseModel):
name: str
age: int
# Extract structured data
user = client.create(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
print(user)
# Output: UserExtract(name='Jason', age=25)
```
### Async Example
```python
import asyncio
import instructor
from pydantic import BaseModel
async_client = instructor.from_provider(
"anyscale/Mixtral-8x7B-Instruct-v0.1",
async_client=True,
mode=instructor.Mode.JSON_SCHEMA,
)
class UserExtract(BaseModel):
name: str
age: int
async def fetch_user():
return await async_client.create(
messages=[{"role": "user", "content": "Extract jason is 25 years old"}],
response_model=UserExtract,
)
user = asyncio.run(fetch_user())
print(user)
```
## Supported Modes
Anyscale supports the following instructor modes:
- `Mode.TOOLS`
- `Mode.JSON`
- `Mode.JSON_SCHEMA`
- `Mode.MD_JSON`
## Models
Anyscale provides access to various models, including:
- Mistral models (e.g., `mistralai/Mixtral-8x7B-Instruct-v0.1`)
- Llama models
- Other open-source LLMs available through their platform

320
참고/instructor-main/docs/integrations/azure.md Normal file
View File

@@ -0,0 +1,320 @@
---
title: Structured outputs with Azure OpenAI, a complete guide w/ instructor
description: Learn how to use Azure OpenAI with instructor for structured outputs, including async/sync implementations, streaming, and validation.
---
# Structured Outputs with Azure OpenAI
This guide demonstrates how to use Azure OpenAI with instructor for structured outputs. Azure OpenAI provides the same powerful models as OpenAI but with enterprise-grade security and compliance features through Microsoft Azure.
## Installation
We can use the same installation as we do for OpenAI since the default `openai` client ships with an AzureOpenAI client.
First, install the required dependencies:
```bash
pip install instructor
```
Next, make sure that you've enabled Azure OpenAI in your Azure account and have a deployment for the model you'd like to use. [Here is a guide to get started](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal)
Once you've done so, you'll have an endpoint and a API key to be used to configure the client.
```bash
instructor.exceptions.InstructorRetryException: Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.'}
```
If you see an error like the one above, make sure you've set the correct endpoint and API key in the client.
## Authentication
To use Azure OpenAI, you'll need:
1. Azure OpenAI endpoint
2. API key
3. Deployment name
```python
import os
from openai import AzureOpenAI
import instructor
# Configure Azure OpenAI client
client = AzureOpenAI(
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2024-02-01",
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"]
)
# Patch the client with instructor
client = instructor.from_provider("azure_openai/gpt-4o-mini")
```
## Using Auto Client (Recommended)
The easiest way to get started with Azure OpenAI is using the `from_provider` method:
```python
import instructor
import os
# Set your Azure OpenAI credentials
os.environ["AZURE_OPENAI_API_KEY"] = "your-api-key"
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://your-resource.openai.azure.com/"
# Create client using the provider string
client = instructor.from_provider("azure_openai/gpt-4o-mini")
# Or async client
async_client = instructor.from_provider("azure_openai/gpt-4o-mini", async_client=True)
```
You can also pass credentials as parameters:
```python
import instructor
client = instructor.from_provider(
"azure_openai/gpt-4o-mini",
api_key="your-api-key",
azure_endpoint="https://your-resource.openai.azure.com/",
api_version="2024-02-01" # Optional, defaults to 2024-02-01
)
```
## Basic Usage
Here's a simple example using a Pydantic model:
```python
import os
import instructor
from openai import AzureOpenAI
from pydantic import BaseModel
client = AzureOpenAI(
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2024-02-01",
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
)
client = instructor.from_provider("azure_openai/gpt-4o-mini")
class User(BaseModel):
name: str
age: int
# Synchronous usage
user = client.create(
messages=[{"role": "user", "content": "John is 30 years old"}],
response_model=User,
)
print(user)
# > name='John' age=30
```
## Async Implementation
Azure OpenAI supports async operations:
```python
import os
import instructor
import asyncio
from openai import AsyncAzureOpenAI
from pydantic import BaseModel
client = AsyncAzureOpenAI(
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2024-02-15-preview",
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
)
client = instructor.from_provider("azure_openai/gpt-4o-mini")
class User(BaseModel):
name: str
age: int
async def get_user_async():
return await client.create(
messages=[{"role": "user", "content": "John is 30 years old"}],
response_model=User,
)
# Run async function
user = asyncio.run(get_user_async())
print(user)
# > name='John' age=30
```
## Nested Models
Azure OpenAI handles complex nested structures:
```python
import os
import instructor
from openai import AzureOpenAI
from pydantic import BaseModel
client = AzureOpenAI(
api_key=os.environ["AZURE_OPENAI_API_KEY"],
api_version="2024-02-01",
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
)
client = instructor.from_provider("azure_openai/gpt-4o-mini")
class Address(BaseModel):
street: str
city: str
country: str
class UserWithAddress(BaseModel):
name: str
age: int
addresses: list[Address]
resp = client.create(
messages=[
{
"role": "user",
"content": """
John is 30 years old and has two addresses:
1. 123 Main St, New York, USA
2. 456 High St, London, UK
""",
}
],
response_model=UserWithAddress,
)
print(resp)
# {
# 'name': 'John',
# 'age': 30,
# 'addresses': [
# {
# 'street': '123 Main St',
# 'city': 'New York',
# 'country': 'USA'
# },
# {
# 'street': '456 High St',
# 'city': 'London',
# 'country': 'UK'
# }
# ]
# }
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
### Partials
You can use our `create_partial` method to stream a single object. Note that validators should not be declared in the response model when streaming objects because it will break the streaming process.
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider("azure_openai/gpt-4o-mini")
class User(BaseModel):
name: str
age: int
bio: str
# Stream partial objects as they're generated
user = client.create_partial(
messages=[
{"role": "user", "content": "Create a user profile for Jason, age 25"},
],
response_model=User,
)
for user_partial in user:
print(user_partial)
# > name='Jason' age=None bio='None'
# > name='Jason' age=25 bio='A tech'
# > name='Jason' age=25 bio='A tech enthusiast'
# > name='Jason' age=25 bio='A tech enthusiast who loves coding, gaming, and exploring new'
# > name='Jason' age=25 bio='A tech enthusiast who loves coding, gaming, and exploring new technologies'
```
## Iterable Responses
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider("azure_openai/gpt-4o-mini")
class User(BaseModel):
name: str
age: int
# Extract multiple users from text
users = client.create_iterable(
messages=[
{
"role": "user",
"content": """
Extract users:
1. Jason is 25 years old
2. Sarah is 30 years old
3. Mike is 28 years old
""",
},
],
response_model=User,
)
for user in users:
print(user)
#> name='Jason' age=25
# > name='Sarah' age=30
# > name='Mike' age=28
```
## Instructor Modes
We provide several modes to make it easy to work with the different response models that OpenAI supports
1. `instructor.Mode.TOOLS` : This uses the [tool calling API](https://platform.openai.com/docs/guides/function-calling) to return structured outputs to the client
2. `instructor.Mode.JSON` : This forces the model to return JSON by using [OpenAI's JSON mode](https://platform.openai.com/docs/guides/structured-outputs#json-mode).
3. `instructor.Mode.FUNCTIONS` : This uses OpenAI's function calling API to return structured outputs and will be deprecated in the future.
4. `instructor.Mode.PARALLEL_TOOLS` : This uses the [parallel tool calling API](https://platform.openai.com/docs/guides/function-calling#configuring-parallel-function-calling) to return structured outputs to the client. This allows the model to generate multiple calls in a single response.
5. `instructor.Mode.MD_JSON` : This makes a simple call to the OpenAI chat completion API and parses the raw response as JSON.
6. `instructor.Mode.TOOLS_STRICT` : This uses the new Open AI structured outputs API to return structured outputs to the client using constrained grammar sampling. This restricts users to a subset of the JSON schema.
7. `instructor.Mode.JSON_O1` : This is a mode for the `O1` model. We created a new mode because `O1` doesn't support any system messages, tool calling or streaming so you need to use this mode to use Instructor with `O1`.
In general, we recommend using `Mode.Tools` because it's the most flexible and future-proof mode. It has the largest set of features that you can specify your schema in and makes things significantly easier to work with.
## Best Practices
## Additional Resources
- [Azure OpenAI Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/)
- [Instructor Documentation](https://instructor-ai.github.io/instructor/)
- [Azure OpenAI Pricing](https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/)

336
참고/instructor-main/docs/integrations/bedrock.md Normal file
View File

@@ -0,0 +1,336 @@
---
title: Structured Outputs with AWS Bedrock and Pydantic
description: Learn how to use AWS Bedrock with Instructor for structured JSON outputs using Pydantic models. Create type-safe, validated responses from AWS Bedrock LLMs with Python.
---
# Structured Outputs with AWS Bedrock
This guide demonstrates how to use AWS Bedrock with Instructor to generate structured outputs. You'll learn how to use AWS Bedrock's LLM models with Pydantic to create type-safe, validated responses.
## Prerequisites
You'll need to have an AWS account with access to Bedrock and the appropriate permissions. You'll also need to set up your AWS credentials.
```bash
pip install "instructor[bedrock]"
```
### See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Mode Migration Guide](../concepts/mode-migration.md) - Move to core modes
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
- [AWS Integration Guide](../examples/index.md#aws-integration) - More AWS examples
# AWS Bedrock
AWS Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon through a single API.
## Auto Client Setup
For simplified setup, you can use the auto client pattern:
```python
import instructor
# Auto client with model specification
client = instructor.from_provider("bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0")
# The auto client automatically handles:
# - AWS credential detection from environment
# - Region configuration (defaults to us-east-1)
# - Mode selection based on model (Claude models use TOOLS)
```
## Deprecation Notice
> **Deprecation Notice:**
>
> The `_async` argument to `instructor.from_bedrock` is deprecated. Please use `async_client=True` for async clients instead. Support for `_async` may be removed in a future release. All new code and examples should use `async_client`.
### Environment Configuration
Set your AWS credentials and region:
```bash
export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key
export AWS_DEFAULT_REGION=us-east-1
```
Or configure using AWS CLI:
```bash
aws configure
```
## Sync Example
```python
import boto3
import instructor
from pydantic import BaseModel
bedrock_client = boto3.client('bedrock-runtime')
client = instructor.from_provider("bedrock/claude-3-5-sonnet-20241022")
class User(BaseModel):
name: str
age: int
user = client.create(
modelId="anthropic.claude-3-sonnet-20240229-v1:0",
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(user)
# > User(name='Jason', age=25)
```
## Async Example
> **Warning:**
> AWS Bedrock's official SDK (`boto3`) does not support async natively. If you need to call Bedrock from async code, you can use `asyncio.to_thread` to run synchronous Bedrock calls in a non-blocking way.
```python
import instructor
from pydantic import BaseModel
import asyncio
client = instructor.from_provider("bedrock/anthropic.claude-3-sonnet-20240229-v1:0")
class User(BaseModel):
name: str
age: int
def get_user():
return client.create(
modelId="anthropic.claude-3-sonnet-20240229-v1:0",
messages=[{"role": "user", "content": "Extract Jason is 25 years old"}],
response_model=User,
)
async def get_user_async():
return await asyncio.to_thread(get_user)
user = asyncio.run(get_user_async())
print(user)
```
## Supported Modes
AWS Bedrock supports the following **core** modes:
- `TOOLS`: Uses function calling for models that support it (like Claude models)
- `MD_JSON`: Direct JSON response generation (text extraction fallback)
> Legacy modes (`BEDROCK_TOOLS`, `BEDROCK_JSON`) are deprecated and map to `Mode.TOOLS` and `Mode.MD_JSON`.
> modes above. Use `TOOLS` or `MD_JSON` in new code.
```python
import boto3
import instructor
from instructor import Mode
from pydantic import BaseModel
# Use from_provider for simplified setup
client = instructor.from_provider("bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0", mode=Mode.TOOLS)
# Or if you need to use a custom boto3 client:
# bedrock_client = boto3.client('bedrock-runtime')
# client = instructor.from_provider("bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0", client=bedrock_client, mode=Mode.TOOLS)
class User(BaseModel):
name: str
age: int
```
## OpenAI Compatibility: Flexible Input Format and Model Parameter
Instructor’s Bedrock integration supports both OpenAI-style and Bedrock-native message formats, as well as any mix of the two. You can use either:
- **OpenAI-style**:
`{"role": "user", "content": "Extract: Jason is 25 years old"}`
- **Bedrock-native**:
`{"role": "user", "content": [{"text": "Extract: Jason is 25 years old"}]}`
- **Mixed**:
You can freely mix OpenAI-style and Bedrock-native messages in the same request. The integration will automatically convert OpenAI-style messages to the correct Bedrock format, while preserving any Bedrock-native fields you provide.
This flexibility also applies to other keyword arguments, such as the model name:
- You can use either `model` (OpenAI-style) or `modelId` (Bedrock-native) as a keyword argument.
- If you provide `model`, Instructor will automatically convert it to `modelId` for Bedrock.
- If you provide both, `modelId` takes precedence.
**Example:**
```python
import instructor
messages = [
{"role": "system", "content": "Extract the name and age."}, # OpenAI-style
{"role": "user", "content": [{"text": "Extract: Jason is 25 years old"}]}, # Bedrock-native
{"role": "assistant", "content": "Sure! Jason is 25."}, # OpenAI-style
]
# Both of these are valid:
user = client.create(
model="anthropic.claude-3-sonnet-20240229-v1:0", # OpenAI-style
messages=messages,
response_model=User,
)
user = client.create(
modelId="anthropic.claude-3-sonnet-20240229-v1:0", # Bedrock-native
messages=messages,
response_model=User,
)
```
All of the above will work seamlessly with Instructor’s Bedrock integration.
## Multimodal: Images and Documents
Instructor will convert OpenAI-style image parts into Bedrock image blocks automatically. For documents (PDFs), Bedrock expects a native `document` block, so you should either pass a Bedrock-native document dict directly or build one with the `PDF` helper.
```python
import instructor
from instructor.processing.multimodal import PDF
client = instructor.from_provider("bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0")
pdf = PDF.from_url("https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf")
response = client.create(
modelId="anthropic.claude-3-sonnet-20240229-v1:0",
messages=[
{
"role": "user",
"content": [
"Analyze this document",
pdf.to_bedrock(),
],
}
],
)
```
Bedrock document blocks also support S3 URIs (for example, `s3://bucket/key.pdf`) and local files; `PDF.to_bedrock()` will load the bytes and sanitize the document name for you.
## Nested Objects
```python
import boto3
import instructor
from pydantic import BaseModel
# Initialize the Bedrock client
bedrock_client = boto3.client('bedrock-runtime')
# Enable instructor patches for Bedrock client
client = instructor.from_provider("bedrock/claude-3-5-sonnet-20241022")
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Create structured output with nested objects
user = client.create(
modelId="anthropic.claude-3-sonnet-20240229-v1:0",
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> User(
#> name='Jason',
#> age=25,
#> addresses=[
#> Address(street='123 Main St', city='New York', country='USA'),
#> Address(street='456 Beach Rd', city='Miami', country='USA')
#> ]
#> )
```
## Modern Models and Features
### Latest Model Support
AWS Bedrock supports many modern foundation models:
```python
import instructor
# Claude 3.5 models (latest)
client = instructor.from_provider("bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0")
# or
client = instructor.from_provider("bedrock/anthropic.claude-3-5-haiku-20241022-v1:0")
# Amazon Nova models (multimodal)
client = instructor.from_provider("bedrock/amazon.nova-micro-v1:0")
# Meta Llama 3 models
client = instructor.from_provider("bedrock/meta.llama3-70b-instruct-v1:0")
# Mistral models
client = instructor.from_provider("bedrock/mistral.mistral-large-2402-v1:0")
```
### Advanced Configuration
```python
import boto3
import instructor
# Custom AWS configuration
bedrock_client = boto3.client(
'bedrock-runtime',
region_name='us-west-2',
aws_access_key_id='your_key',
aws_secret_access_key='your_secret'
)
# Use from_provider with custom client
client = instructor.from_provider(
"bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
client=bedrock_client,
mode=instructor.Mode.TOOLS
)
# Advanced inference configuration
user = client.create(
modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
messages=[{"role": "user", "content": "Extract user info"}],
response_model=User,
inferenceConfig={
"maxTokens": 2048,
"temperature": 0.1,
"topP": 0.9,
"stopSequences": ["STOP"]
}
)
```

242
참고/instructor-main/docs/integrations/cerebras.md Normal file
View File

@@ -0,0 +1,242 @@
---
title: "Structured outputs with Cerebras, a complete guide w/ instructor"
description: "Complete guide to using Instructor with Cerebras's hardware-accelerated AI models. Learn how to generate structured, type-safe outputs with high-performance computing."
---
# Structured outputs with Cerebras, a complete guide w/ instructor
Cerebras provides hardware-accelerated AI models optimized for high-performance computing environments. This guide shows you how to use Instructor with Cerebras's models for type-safe, validated responses.
## Quick Start
Install Instructor with Cerebras support:
```bash
pip install "instructor[cerebras_cloud_sdk]"
```
## Simple User Example (Sync)
```python
import instructor
from cerebras.cloud.sdk import Cerebras
from pydantic import BaseModel
client = instructor.from_provider("cerebras/llama3.1-70b")
class User(BaseModel):
name: str
age: int
# Create structured output
resp = client.create(
messages=[
{
"role": "user",
"content": "Extract the name and age of the person in this sentence: John Smith is 29 years old.",
}
],
response_model=User,
)
print(resp)
#> User(name='John Smith', age=29)
```
## Simple User Example (Async)
```python
import instructor
from pydantic import BaseModel
import asyncio
client = instructor.from_provider(
"cerebras/llama3.1-70b",
async_client=True,
)
class User(BaseModel):
name: str
age: int
async def extract_user():
resp = await client.create(
messages=[
{
"role": "user",
"content": "Extract the name and age of the person in this sentence: John Smith is 29 years old.",
}
],
response_model=User,
)
return resp
# Run async function
resp = asyncio.run(extract_user())
print(resp)
#> User(name='John Smith', age=29)
```
## Nested Example
```python
from pydantic import BaseModel
import instructor
from cerebras.cloud.sdk import Cerebras
client = instructor.from_provider("cerebras/llama3.1-70b")
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
}
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
We currently support partial streaming for Cerebras by parsing the raw text completion. We have not implemented streaming for function calling at this point in time yet. Please make sure you have `mode=instructor.Mode.MD_JSON` set when using partial streaming.
```python
import instructor
from cerebras.cloud.sdk import Cerebras
from pydantic import BaseModel
from typing import Iterable
client = instructor.from_provider(
"cerebras/llama3.1-70b",
mode=instructor.Mode.MD_JSON,
)
class Person(BaseModel):
name: str
age: int
resp = client.create_partial(
messages=[
{
"role": "user",
"content": "Ivan is 27 and lives in Singapore",
}
],
response_model=Person,
stream=True,
)
for person in resp:
print(person)
# > name=None age=None
# > name='Ivan' age=None
# > name='Ivan' age=27
```
## Iterable Example
```python
import instructor
from cerebras.cloud.sdk import Cerebras
from pydantic import BaseModel
from typing import Iterable
client = instructor.from_provider(
"cerebras/llama3.1-70b",
mode=instructor.Mode.MD_JSON,
)
class Person(BaseModel):
name: str
age: int
resp = client.create_iterable(
messages=[
{
"role": "user",
"content": "Extract all users from this sentence : Chris is 27 and lives in San Francisco, John is 30 and lives in New York while their college roomate Jessica is 26 and lives in London",
}
],
response_model=Person,
stream=True,
)
for person in resp:
print(person)
# > Person(name='Chris', age=27)
# > Person(name='John', age=30)
# > Person(name='Jessica', age=26)
```
## Instructor Hooks
Instructor provides several hooks to customize behavior:
### Validation Hook
```python
from instructor import Instructor
def validation_hook(value, retry_count, exception):
print(f"Validation failed {retry_count} times: {exception}")
return retry_count < 3 # Retry up to 3 times
instructor.patch(client, validation_hook=validation_hook)
```
## Instructor Modes
We provide serveral modes to make it easy to work with the different response models that Cerebras Supports
1. `instructor.Mode.MD_JSON` : This parses the raw completions as a valid JSON object.
2. `instructor.Mode.TOOLS` : This uses Cerebras's tool calling mode to return structured outputs to the client.
In general, we recommend using `Mode.TOOLS` because it's the most flexible and future-proof mode. It has the largest set of features that you can specify your schema in and makes things significantly easier to work with.

163
참고/instructor-main/docs/integrations/cohere.md Normal file
View File

@@ -0,0 +1,163 @@
---
title: Structured outputs with Cohere, a complete guide w/ instructor
description: Learn how to leverage Cohere's command models with Python's instructor library for structured data outputs.
---
# Structured outputs with Cohere, a complete guide w/ instructor
This guide demonstrates how to use Cohere with Instructor to generate structured outputs. You'll learn how to use Cohere's command models to create type-safe responses.
You can now use any of the Cohere's [command models](https://docs.cohere.com/docs/models) with the `instructor` library to get structured outputs.
You'll need a cohere API key which can be obtained by signing up [here](https://dashboard.cohere.com/) and gives you [free](https://cohere.com/pricing), rate-limited usage for learning and prototyping.
### See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Document Segmentation](../examples/document_segmentation.md) - Cohere example for document processing
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
# Cohere V2 API Support
As of version 1.12.0, Instructor supports both Cohere V1 and V2 SDK clients. The V2 API provides an OpenAI-compatible interface with support for the latest Cohere models.
**Key differences:**
- **V2 API** (recommended): Uses `cohere.ClientV2` / `cohere.AsyncClientV2` with OpenAI-compatible message format
- **V1 API** (legacy): Uses `cohere.Client` / `cohere.AsyncClient` with Cohere-specific message format
The V2 API is recommended for new projects as it provides better compatibility with the OpenAI SDK interface and supports the latest models like `command-a-03-2025`.
## Setup
```
pip install "instructor[cohere]"
```
This installs `cohere>=5.1.8`, which includes both V1 and V2 client support.
Export your key:
```
export CO_API_KEY=<YOUR_COHERE_API_KEY>
```
## Example (V2 API - Recommended)
The easiest way to use Cohere with Instructor is through the `from_provider` factory, which automatically uses the V2 API:
```python
from pydantic import BaseModel, Field
from typing import List
import instructor
# Using from_provider automatically uses Cohere V2 API
client = instructor.from_provider(
"cohere/command-a-03-2025",
max_tokens=1000,
)
class Person(BaseModel):
name: str = Field(description="name of the person")
country_of_origin: str = Field(description="country of origin of the person")
class Group(BaseModel):
group_name: str = Field(description="name of the group")
members: List[Person] = Field(description="list of members in the group")
task = """\
Given the following text, create a Group object for 'The Beatles' band
Text:
The Beatles were an English rock band formed in Liverpool in 1960. With a line-up comprising John Lennon, Paul McCartney, George Harrison and Ringo Starr, they are regarded as the most influential band of all time. The group were integral to the development of 1960s counterculture and popular music's recognition as an art form.
"""
group = client.create(
response_model=Group,
messages=[{"role": "user", "content": task}],
temperature=0,
)
print(group.model_dump_json(indent=2))
"""
{
"group_name": "The Beatles",
"members": [
{
"name": "John Lennon",
"country_of_origin": "England"
},
{
"name": "Paul McCartney",
"country_of_origin": "England"
},
{
"name": "George Harrison",
"country_of_origin": "England"
},
{
"name": "Ringo Starr",
"country_of_origin": "England"
}
]
}
"""
```
### Async Example
```python
import instructor
async_client = instructor.from_provider(
"cohere/command-a-03-2025",
async_client=True,
max_tokens=1000,
)
```
## Using Cohere SDK Directly
You can also explicitly create a Cohere client and patch it with Instructor:
### V2 API (Recommended)
```python
import cohere
import instructor
# Use from_provider for simplified setup
client = instructor.from_provider("cohere/command-a-03-2025", mode=instructor.Mode.TOOLS)
# Now use it with structured outputs
response = client.create(
response_model=YourModel,
model="command-a-03-2025",
messages=[{"role": "user", "content": "Extract..."}],
)
```
### V1 API (Legacy Support)
The V1 API is still supported for backward compatibility:
```python
import cohere
import instructor
# Use from_provider for simplified setup (works with both V1 and V2)
client = instructor.from_provider("cohere/command-a-03-2025", mode=instructor.Mode.TOOLS)
# V1 uses different message format internally but instructor handles the conversion
response = client.create(
response_model=YourModel,
model="command-r-plus",
messages=[{"role": "user", "content": "Extract..."}],
)
```
**Note**: Instructor automatically detects whether you're using V1 or V2 client and handles message format conversion accordingly. The V2 API uses OpenAI-compatible message format (`messages`), while V1 uses Cohere's legacy format (`message` + `chat_history`).

169
참고/instructor-main/docs/integrations/cortex.md Normal file
View File

@@ -0,0 +1,169 @@
---
title: "Structured outputs with Cortex, a complete guide w/ instructor"
description: "Learn how to use Cortex with Instructor for structured outputs. Complete guide with examples and best practices."
---
# Structured outputs with Cortex
Cortex.cpp is a runtime that helps you run open source LLMs out of the box. It supports a wide variety of models and powers their [Jan](https://jan.ai) platform. This guide provides a quickstart on how to use Cortex with instructor for structured outputs.
## Quick Start
Instructor comes with support for the OpenAI client out of the box, so you don't need to install anything extra.
```bash
pip install "instructor"
```
Once you've done so, make sure to pull the model that you'd like to use. In this example, we'll be using a quantized llama3.2 model.
```bash
cortex run llama3.2:3b-gguf-q4-km
```
Let's start by initializing the client below - note that we need to provide a base URL and an API key here. The API key isn't important, it's just so the OpenAI client doesn't throw an error.
```python
import instructor
client = instructor.from_provider(
"cortex/llama3.2:3b-gguf-q4-km",
base_url="http://localhost:39281/v1",
api_key="this is a fake api key that doesn't matter",
)
```
## Simple User Example (Sync)
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider(
"cortex/llama3.2:3b-gguf-q4-km",
base_url="http://localhost:39281/v1",
api_key="this is a fake api key that doesn't matter",
)
class User(BaseModel):
name: str
age: int
resp = client.create(
messages=[{"role": "user", "content": "Ivan is 27 and lives in Singapore"}],
response_model=User,
)
print(resp)
# > name='Ivan', age=27
```
## Simple User Example (Async)
```python
import instructor
from pydantic import BaseModel
import asyncio
# Initialize with API key
client = instructor.from_provider(
"cortex/llama3.2:3b-gguf-q4-km",
async_client=True,
base_url="http://localhost:39281/v1",
api_key="this is a fake api key that doesn't matter",
)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
#> User(name='Jason', age=25)
```
## Nested Example
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider(
"cortex/llama3.2:3b-gguf-q4-km",
base_url="http://localhost:39281/v1",
api_key="this is a fake api key that doesn't matter",
)
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
In this tutorial we've seen how we can run local models with Cortex while simplifying a lot of the logic around managing retries and function calling with our simple interface.
We'll be publishing a lot more content on Cortex and how to work with local models moving forward so do keep an eye out for that.
## Related Resources
- [Cortex Documentation](https://cortex.so/docs/)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Updates and Compatibility
Instructor maintains compatibility with the latest OpenAI API versions and models. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.

89
참고/instructor-main/docs/integrations/databricks.md Normal file
View File

@@ -0,0 +1,89 @@
---
title: Databricks
description: Guide to using instructor with Databricks models
---
# Structured outputs with Databricks, a complete guide w/ instructor
[Databricks](https://www.databricks.com/) provides an AI platform with access to various models. This guide shows how to use instructor with Databricks to get structured outputs.
## Quick Start
First, install the required packages:
```bash
uv pip install instructor openai
```
Set your Databricks workspace URL and token as environment variables:
```bash
export DATABRICKS_TOKEN="your_personal_access_token"
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
```
`DATABRICKS_API_KEY` and `DATABRICKS_WORKSPACE_URL` are also supported if you prefer those names. The provider appends `/serving-endpoints` automatically, so the host only needs the base workspace URL.
## Basic Example
Here's how to extract structured data from Databricks models:
```python
import instructor
from pydantic import BaseModel
# Initialize the client; host and token are read from the environment
client = instructor.from_provider(
"databricks/dbrx-instruct",
mode=instructor.Mode.TOOLS,
)
# Define your data structure
class UserExtract(BaseModel):
name: str
age: int
# Extract structured data
user = client.create(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
print(user)
# Output: UserExtract(name='Jason', age=25)
```
If you need to point at a different workspace or testing endpoint, pass `base_url="https://alt-workspace.cloud.databricks.com/serving-endpoints"`. The helper will use that value as-is without adding another suffix.
### Async Example
```python
async_client = instructor.from_provider(
"databricks/dbrx-instruct",
async_client=True,
mode=instructor.Mode.TOOLS,
)
```
## Supported Modes
Databricks supports the same modes as OpenAI:
- `Mode.TOOLS`
- `Mode.JSON`
- `Mode.FUNCTIONS`
- `Mode.PARALLEL_TOOLS`
- `Mode.MD_JSON`
- `Mode.TOOLS_STRICT`
- `Mode.JSON_O1`
## Models
Databricks provides access to various models depending on your setup, including:
- Foundation models hosted on Databricks
- Custom fine-tuned models
- Open source models deployed on Databricks

323
참고/instructor-main/docs/integrations/deepseek.md Normal file
View File

@@ -0,0 +1,323 @@
---
title: "Structured outputs with DeepSeek, a complete guide with instructor"
description: "Learn how to use Instructor with DeepSeek's models for type-safe, structured outputs."
---
# Structured outputs with DeepSeek, a complete guide with instructor
DeepSeek is a Chinese company that provides AI models and services. They're most notable for the deepseek coder and chat model and most recently, the R1 reasoning model.
This guide covers everything you need to know about using DeepSeek with Instructor for type-safe, validated responses.
## Quick Start
Instructor comes with support for the OpenAI Client out of the box, so you don't need to install anything extra.
```bash
pip install "instructor"
```
⚠️ **Important**: You must set your DeepSeek API key before using the client. You can do this in two ways:
1. Set the environment variable:
```bash
export DEEPSEEK_API_KEY='your-api-key-here'
```
2. Or provide it directly to the client:
```python
import os
from openai import OpenAI
client = OpenAI(api_key=os.getenv('DEEPSEEK_API_KEY'), base_url="https://api.deepseek.com")
```
## Simple User Example (Sync)
```python
import os
from openai import OpenAI
from pydantic import BaseModel
import instructor
client = instructor.from_provider(
"deepseek/deepseek-chat",
base_url="https://api.deepseek.com",
)
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(user)
# > name='Jason' age=25
```
## Simple User Example (Async)
```python
import os
import asyncio
from pydantic import BaseModel
import instructor
client = instructor.from_provider(
"deepseek/deepseek-chat",
async_client=True,
base_url="https://api.deepseek.com",
)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
# > name='Jason' age=25
```
## Nested Example
```python
from pydantic import BaseModel
import os
from openai import OpenAI
import instructor
from pydantic import BaseModel
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Initialize with API key
client = instructor.from_provider(
"deepseek/deepseek-chat",
base_url="https://api.deepseek.com",
)
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
### Partials
```python
from pydantic import BaseModel
import os
from openai import OpenAI
import instructor
from pydantic import BaseModel
# Initialize with API key
client = instructor.from_provider(
"deepseek/deepseek-chat",
base_url="https://api.deepseek.com",
)
class User(BaseModel):
name: str
age: int
bio: str
user = client.create_partial(
messages=[
{
"role": "user",
"content": "Create a user profile for Jason and a one sentence bio, age 25",
},
],
response_model=User,
)
for user_partial in user:
print(user_partial)
# > name='Jason' age=None bio='None'
# > name='Jason' age=25 bio='A tech'
# > name='Jason' age=25 bio='A tech enthusiast'
# > name='Jason' age=25 bio='A tech enthusiast who loves coding, gaming, and exploring new'
# > name='Jason' age=25 bio='A tech enthusiast who loves coding, gaming, and exploring new technologies'
```
### Iterable Example
```python
from pydantic import BaseModel
import os
from openai import OpenAI
import instructor
from pydantic import BaseModel
# Initialize with API key
client = instructor.from_provider(
"deepseek/deepseek-chat",
base_url="https://api.deepseek.com",
)
class User(BaseModel):
name: str
age: int
# Extract multiple users from text
users = client.create_iterable(
messages=[
{
"role": "user",
"content": """
Extract users:
1. Jason is 25 years old
2. Sarah is 30 years old
3. Mike is 28 years old
""",
},
],
response_model=User,
)
for user in users:
print(user)
#> name='Jason' age=25
#> name='Sarah' age=30
#> name='Mike' age=28
```
## Reasoning Models
Because Instructor is built on top of the OpenAI API, we can get our reasoning traces from the `deepseek-reasoner` model. Make sure to configure the `MD_JSON` mode here to get the best experience.
```python
import os
from openai import OpenAI
from pydantic import BaseModel
import instructor
from rich import print
client = instructor.from_provider(
"deepseek/deepseek-chat",
base_url="https://api.deepseek.com",
mode=instructor.Mode.MD_JSON,
)
class User(BaseModel):
name: str
age: int
# Create structured output
completion, raw_completion = client.create_with_completion(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(completion)
# > User(name='Jason', age=25)
print(raw_completion.choices[0].message.reasoning_content)
# > Okay, let's see. The user wants me to extract information from the sentence "Jason is 25 years old" and format it into a JSON object that matches the given schema. The schema requires a "name" and an "age", both of which are required.
# >
# > First, I need to identify the name. The sentence starts with "Jason", so that's the name. Then the age is given as "25 years old". The age should be an integer, so I need to convert "25" from a string to a number.
# >
# > So putting that together, the JSON should have "name": "Jason" and "age": 25. Let me double-check the schema to make sure there are no other requirements. The properties are "name" (string) and "age" (integer), both required. Yep, that's all.
# >
# > I need to make sure the JSON is correctly formatted, with commas and braces. Also, the user specified to return it in a json codeblock, not the schema itself. So the final answer should be a JSON object with those key-value pairs.
```
## Instructor Modes
We suggest using the `Mode.Tools` mode for Deepseek which is the default when initializing via `from_provider`.
## Related Resources
- [DeepSeek Documentation](https://api-docs.deepseek.com/)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Updates and Compatibility
Instructor maintains compatibility with the latest OpenAI API versions and models. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.

251
참고/instructor-main/docs/integrations/fireworks.md Normal file
View File

@@ -0,0 +1,251 @@
---
title: "Structured outputs with Fireworks, a complete guide w/ instructor"
description: "Complete guide to using Instructor with Fireworks AI models. Learn how to generate structured, type-safe outputs with high-performance, cost-effective AI capabilities."
---
# Structured outputs with Fireworks, a complete guide w/ instructor
Fireworks provides efficient and cost-effective AI models with enterprise-grade reliability. This guide shows you how to use Instructor with Fireworks's models for type-safe, validated responses.
## Quick Start
Install Instructor with Fireworks support:
```bash
pip install "instructor[fireworks-ai]"
```
## Simple User Example (Sync)
```python
from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel
# Initialize the client
client = Fireworks()
# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
messages=[
{
"role": "user",
"content": "Extract: Jason is 25 years old",
}
],
response_model=User,
)
print(user)
# > User(name='Jason', age=25)
```
## Simple User Example (Async)
```python
import instructor
from pydantic import BaseModel
import asyncio
client = instructor.from_provider(
"fireworks/llama-v3-70b-instruct",
async_client=True,
)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
messages=[
{
"role": "user",
"content": "Extract: Jason is 25 years old",
}
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user) # User(name='Jason', age=25)
```
## Nested Example
```python
from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel
# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
}
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
### Partial Streaming Example
```python
from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel
# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")
class User(BaseModel):
name: str
age: int
bio: str
user = client.create_partial(
messages=[
{
"role": "user",
"content": "Create a user profile for Jason + 1 sentence bio, age 25",
},
],
response_model=User,
)
for user_partial in user:
print(user_partial)
# name=None age=None bio=None
# name='Jason' age=None bio=None
# name='Jason' age=25 bio="When he's"
# name='Jason' age=25 bio="When he's not working as a graphic designer, Jason can usually be found trying out new craft beers or attempting to cook something other than ramen noodles."
```
## Iterable Example
```python
from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel
# Enable instructor patches
client = instructor.from_provider("fireworks/llama-v3-70b-instruct")
class User(BaseModel):
name: str
age: int
# Extract multiple users from text
users = client.create_iterable(
messages=[
{
"role": "user",
"content": """
Extract users:
1. Jason is 25 years old
2. Sarah is 30 years old
3. Mike is 28 years old
""",
},
],
response_model=User,
)
for user in users:
print(user)
# name='Jason' age=25
# name='Sarah' age=30
# name='Mike' age=28
```
## Instructor Modes
We provide several modes to make it easy to work with the different response models that Fireworks supports
1. `instructor.Mode.MD_JSON` : This parses the raw text completion into a pydantic object
2. `instructor.Mode.TOOLS` : This uses Fireworks's tool calling API to return structured outputs to the client
## Related Resources
- [Fireworks Documentation](https://docs.fireworks.ai/)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Updates and Compatibility
Instructor maintains compatibility with Fireworks's latest API versions. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.
Note: Always verify model-specific features and limitations before implementing streaming functionality in production environments.

717
참고/instructor-main/docs/integrations/genai.md Normal file
View File

@@ -0,0 +1,717 @@
---
draft: False
date: 2025-03-15
title: "Structured outputs with Google's genai SDK"
description: "Learn how to use Instructor with Google's Generative AI SDK to extract structured data from Gemini models."
slug: genai
tags:
- patching
authors:
- instructor
---
# Structured Outputs with Google's genai SDK
!!! info "Recommended SDK"
The `genai` SDK is Google's recommended Python client for working with Gemini models. It provides a unified interface for both the Gemini API and Vertex AI. For detailed setup instructions, including how to use it with Vertex AI, please refer to the [official Google AI documentation for the GenAI SDK](https://googleapis.github.io/python-genai/).
This guide demonstrates how to use Instructor with Google's `genai` SDK to extract structured data from Gemini models.
We currently have two modes for Gemini
- `Mode.TOOLS` : This leverages function calling under the hood and returns a structured response
- `Mode.JSON` : This provides Gemini with a JSON Schema that it will use to respond in a structured format with
!!! info "Gemini Thought Parts Filtering"
When using `Mode.TOOLS`, Instructor automatically filters out thought parts from Gemini responses. Gemini 2.5 models include internal reasoning parts with `thought: true` by default, which cannot be disabled. Instructor removes these thought parts before processing the structured output to prevent runtime errors.
This filtering happens automatically and requires no additional configuration. For more information about Gemini's thinking feature, see the [official documentation](https://ai.google.dev/gemini-api/docs/thinking).
!!! note "Backwards Compatibility"
The provider-specific modes (`Mode.TOOLS`, `Mode.JSON`, `Mode.JSON`) are still supported but emit deprecation warnings and map to the generic modes (`Mode.TOOLS`, `Mode.JSON`).
## Installation
```bash
pip install "instructor[google-genai]"
```
## Basic Usage
!!! warning "Unions and Optionals"
Gemini doesn't have support for Union and Optional types in the structured outputs and tool calling integrations. We currently throw an error when we detect these in your response model.
Getting started with Instructor and the genai SDK is straightforward. Just create a Pydantic model defining your output structure, patch the genai client, and make your request with a response_model parameter:
```python
from google import genai
import instructor
from pydantic import BaseModel
# Define your Pydantic model
class User(BaseModel):
name: str
age: int
# Initialize and patch the client
client = instructor.from_provider("google/gemini-2.5-flash")
# Extract structured data
response = client.create(
messages=[{"role": "user", "content": "Extract: Jason is 25 years old"}],
response_model=User,
)
print(response) # User(name='Jason', age=25)
```
## Alternative: Using the v2 GenAI client
!!! note "Recommended: Use `from_provider`"
The `from_provider` approach shown above is recommended for most use cases. The `from_genai` helper below is available if you need to work directly with the native `google.genai.Client` and keep the Google request format intact.
```python
from google.genai import Client
from instructor import Mode
from instructor.v2 import from_genai
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
raw_client = Client(api_key="YOUR_KEY")
client = from_genai(raw_client, mode=Mode.TOOLS)
result = client.chat.completions.create(
messages=[{"role": "user", "content": "Extract: Jason is 25 years old"}],
response_model=User,
)
print(result)
```
Behind the scenes the v2 client registers the correct mode handler, converts OpenAI-style messages to the GenAI `contents` format, and parses the response while filtering Gemini thought parts.
## Message Formatting
Genai supports multiple message formats, and Instructor seamlessly works with all of them. This flexibility allows you to use whichever format is most convenient for your application:
```python
from google import genai
import instructor
from pydantic import BaseModel
from google.genai import types
# Define your Pydantic model
class User(BaseModel):
name: str
age: int
# Initialize and patch the client
client = instructor.from_provider("google/gemini-2.5-flash")
# Single string (converted to user message)
response = client.create(
messages="Jason is 25 years old",
response_model=User,
)
print(response)
# > name='Jason' age=25
# Standard format
response = client.create(
messages=[
{"role": "user", "content": "Jason is 25 years old"}
],
response_model=User,
)
print(response)
# > name='Jason' age=25
# Using genai's Content type
response = client.create(
messages=[
genai.types.Content(
role="user",
parts=[genai.types.Part.from_text(text="Jason is 25 years old")]
)
],
response_model=User,
)
print(response)
# > name='Jason' age=25
```
### System Messages
System messages help set context and instructions for the model. With Gemini models, you can provide system messages in two different ways:
```python
from google import genai
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_provider("google/gemini-2.5-flash")
# As a parameter
response = client.create(
system="Jason is 25 years old",
messages=[{"role": "user", "content": "You are a data extraction assistant"}],
response_model=User,
)
print(response)
# > name='Jason' age=25
# Or as a message with role "system"
response = client.create(
messages=[
{"role": "system", "content": "Jason is 25 years old"},
{"role": "user", "content": "You are a data extraction assistant"},
],
response_model=User,
)
print(response)
# > name='Jason' age=25
```
## Template Variables
Template variables make it easy to reuse prompts with different values. This is particularly useful for dynamic content or when testing different inputs:
```python
from google import genai
import instructor
from pydantic import BaseModel
from google.genai import types
# Define your Pydantic model
class User(BaseModel):
name: str
age: int
# Initialize and patch the client
client = instructor.from_provider("google/gemini-2.5-flash")
# Single string (converted to user message)
response = client.create(
messages=[{"role": "user", "content": "{{ name }} is {{ age }} years old"}],
response_model=User,
context={
"name": "Jason",
"age": 25,
},
)
print(response)
# > name='Jason' age=25
# Standard format
response = client.create(
messages=[{"role": "user", "content": "{{ name }} is {{ age }} years old"}],
response_model=User,
context={
"name": "Jason",
"age": 25,
},
)
print(response)
# > name='Jason' age=25
# Using genai's Content type
response = client.create(
messages=[
genai.types.Content(
role="user",
parts=[genai.types.Part.from_text(text="{{name}} is {{age}} years old")],
)
],
response_model=User,
context={
"name": "Jason",
"age": 25,
},
)
print(response)
# > name='Jason' age=25
```
## Validation and Retries
Instructor can automatically retry requests when validation fails, ensuring you get properly formatted data. This is especially helpful when enforcing specific data requirements:
```python
from typing import Annotated
from pydantic import AfterValidator, BaseModel
import instructor
from google import genai
def uppercase_validator(v: str) -> str:
if v.islower():
raise ValueError("Name must be ALL CAPS")
return v
class UserDetail(BaseModel):
name: Annotated[str, AfterValidator(uppercase_validator)]
age: int
client = instructor.from_provider("google/gemini-2.5-flash")
response = client.create(
messages=[{"role": "user", "content": "Extract: jason is 25 years old"}],
response_model=UserDetail,
max_retries=3,
)
print(response) # UserDetail(name='JASON', age=25)
```
## Multimodal Capabilities
> We've provided a few different sample files for you to use to test out these new features. All examples below use these files.
>
> - (Audio) : A Recording of the Original Gettysburg Address : [gettysburg.wav](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/gettysburg.wav)
> - (Image) : An image of some blueberry plants [image.jpg](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg)
> - (PDF) : A sample PDF file which contains a fake invoice [invoice.pdf](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf)
Instructor provides a unified, provider-agnostic interface for working with multimodal inputs like images, PDFs, and audio files. With Instructor's multimodal objects, you can easily load media from URLs, local files, or base64 strings using a consistent API that works across different AI providers (OpenAI, Anthropic, Mistral, etc.).
Instructor handles all the provider-specific formatting requirements behind the scenes, ensuring your code remains clean and future-proof as provider APIs evolve.
Let's see how to use the Image, Audio and PDF classes.
### Image Processing
!!! info "Autodetect Images"
For convenient handling of images, you can enable automatic image conversion using the `autodetect_images` parameter. When enabled, Instructor will automatically detect and convert file paths and HTTP URLs provided as strings into the appropriate format required by the Google GenAI SDK. This makes working with images seamless and straightforward. ( see examples below )
Instructor makes it easy to analyse and extract semantic information from images using the Gemini series of models. [Click here](https://ai.google.dev/gemini-api/docs/models) to check if the model you'd like to use has vison capabilities.
Let's see an example below with the sample image above where we'll load it in using our `from_url` method.
Note that we support local files and base64 strings too with the `from_path` and the `from_base64` class methods.
```python
from instructor.processing.multimodal import Image
from pydantic import BaseModel, Field
import instructor
from google.genai import Client
class ImageDescription(BaseModel):
objects: list[str] = Field(..., description="The objects in the image")
scene: str = Field(..., description="The scene of the image")
colors: list[str] = Field(..., description="The colors in the image")
client = instructor.from_provider("google/gemini-2.5-flash")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg"
# Multiple ways to load an image:
response = client.create(
response_model=ImageDescription,
messages=[
{
"role": "user",
"content": [
"What is in this image?",
# Option 1: Direct URL with autodetection
Image.from_url(url),
# Option 2: Local file
# Image.from_path("path/to/local/image.jpg")
# Option 3: Base64 string
# Image.from_base64("base64_encoded_string_here")
# Option 4: Autodetect
# Image.autodetect(<url|path|base64>)
],
},
],
)
print(response)
# Example output:
# ImageDescription(
# objects=['blueberries', 'leaves'],
# scene='A blueberry bush with clusters of ripe blueberries and some unripe ones against a cloudy sky',
# colors=['green', 'blue', 'purple', 'white']
# )
```
### Audio Processing
Instructor makes it easy to analyse and extract semantic information from Audio files using the Gemini series of models. Let's see an example below with the sample Audio file above where we'll load it in using our `from_url` method.
Note that we support local files and base64 strings too with the `from_path`
```python
from instructor.processing.multimodal import Audio
from pydantic import BaseModel
import instructor
from google.genai import Client
class AudioDescription(BaseModel):
transcript: str
summary: str
speakers: list[str]
key_points: list[str]
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/gettysburg.wav"
client = instructor.from_provider("google/gemini-2.5-flash")
response = client.create(
response_model=AudioDescription,
messages=[
{
"role": "user",
"content": [
"Please transcribe and analyze this audio:",
# Multiple loading options:
Audio.from_url(url),
# Option 2: Local file
# Audio.from_path("path/to/local/audio.mp3")
],
},
],
)
print(response)
# > transcript='Four score and seven years ago our fathers..."]
```
### PDF
Instructor makes it easy to analyse and extract semantic information from PDFs using Gemini's new models.
Let's see an example below with the sample PDF above where we'll load it in using our `from_url` method. With this integration that we're passing in the raw bytes to gemini itself, we also support using the Files api with the `PDFWithGenaiFile` class.
Note that we support local files and base64 strings using this method too with the `from_path` and the `from_base64` class methods.
```python
from instructor.processing.multimodal import PDF
from pydantic import BaseModel
import instructor
from google.genai import Client
class Receipt(BaseModel):
total: int
items: list[str]
client = instructor.from_provider("google/gemini-2.5-flash")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
# Multiple ways to load an PDF:
response = client.create(
response_model=Receipt,
messages=[
{
"role": "user",
"content": [
"Extract out the total and line items from the invoice",
# Option 1: Direct URL
PDF.from_url(url),
# Option 2: Local file
# PDF.from_path("path/to/local/invoice.pdf"),
# Option 3: Base64 string
# PDF.from_base64("base64_encoded_string_here")
# Option 4: Autodetect
# PDF.autodetect(<url|path|base64>)
],
},
],
)
print(response)
# > Receipt(total=220, items=['English Tea', 'Tofu'])
```
We also support the use of PDFs with the Gemini `Files` api with the `PDFWithGenaiFile` that allows you to use existing uploaded files or local files.
Note that the `PdfWithGenaiFile.from_new_genai_file` operation is blocking and you can set the timeout and retry delay that we'll call while we await the upload to be registered as completed.
```python
PDFWithGenaiFile.from_new_genai_file(
"./invoice.pdf",
retry_delay=1, # Time to wait before checking if file is ready to use
max_retries=20 # Number of times to check before throwing an error
),
```
This makes it easier for you to work with the Gemini files API. You can use this in a normal chat completion as seen below
```python
from instructor.processing.multimodal import PDFWithGenaiFile
from pydantic import BaseModel
import instructor
from google.genai import Client
class Receipt(BaseModel):
total: int
items: list[str]
client = instructor.from_provider("google/gemini-2.5-flash")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
# Multiple ways to load an PDF:
response = client.create(
response_model=Receipt,
messages=[
{
"role": "user",
"content": [
"Extract out the total and line items from the invoice",
# Option 1: Direct URL
PDFWithGenaiFile.from_new_genai_file("./invoice.pdf"),
# Option 2 : Existing Genai File
# PDFWithGenaiFile.from_existing_genai_file("invoice.pdf"),
],
},
],
)
print(response)
```
If you'd like more fine-grained control over the files used, you can also use the `Files` api directly as seen below.
## Using Files
Our API integration also supports the use of files
```python
import instructor
from pydantic import BaseModel
class Summary(BaseModel):
summary: str
client = instructor.from_provider("google/gemini-2.5-flash")
file1 = client.files.upload(
file="./gettysburg.wav",
)
# As a parameter
response = client.create(
messages=[
{
"role": "user",
"content": [
"Summarise the audio file.",
file1,
]
}
],
response_model=Summary,
)
print(response)
# > summary="Abraham Lincoln's Gettysburg Address commences by stating that 87 years prior, the founding fathers created a new nation based on liberty and equality. It goes on to say that the Civil War is testing whether a nation so conceived can survive."
```
## Streaming Responses
!!! warning "Streaming Limitations"
**As of July 11, 2025, Google GenAI does not support streaming with tool/function calling or structured outputs for regular models.**
- `Mode.TOOLS` and `Mode.JSON` do not support streaming with regular models
- To use streaming, you must use `Partial[YourModel]` explicitly or switch to other modes like `Mode.JSON`
- Alternatively, set `stream=False` to disable streaming
Streaming allows you to process responses incrementally rather than waiting for the complete result. This is extremely useful for making UI changes feel instant and responsive.
### Partial Streaming
Receive a stream of complete, validated objects as they're generated:
```python
from pydantic import BaseModel
import instructor
client = instructor.from_provider(
"google/gemini-2.5-flash",
mode=instructor.Mode.JSON,
)
class Person(BaseModel):
name: str
age: int
class PersonList(BaseModel):
people: list[Person]
stream = client.create_partial(
model="gemini-2.5-flash",
response_model=PersonList,
stream=True,
messages=[
{
"role": "user",
"content": "Ivan is 20 years old, Jason is 25 years old, and John is 30 years old",
}
],
)
for extraction in stream:
print(extraction)
# > people=[PartialPerson(name='Ivan', age=None)]
# > people=[PartialPerson(name='Ivan', age=20), PartialPerson(name='Jason', age=25), PartialPerson(name='John', age=None)]
# > people=[PartialPerson(name='Ivan', age=20), PartialPerson(name='Jason', age=25), PartialPerson(name='John', age=30)]
```
### Iterable Streaming
For extracting multiple objects from a single response, use `create_iterable`:
```python
from pydantic import BaseModel
import instructor
client = instructor.from_provider("google/gemini-2.5-flash")
class User(BaseModel):
name: str
age: int
# Extract multiple users from a single response
stream = client.create_iterable(
model="gemini-2.5-flash",
response_model=User,
stream=True,
messages=[
{
"role": "user",
"content": "Jason is 25 years old, Sarah is 30 years old, and Mike is 28 years old",
}
],
)
for user in stream:
print(user)
# > User(name='Jason', age=25)
# > User(name='Sarah', age=30)
# > User(name='Mike', age=28)
```
### Async Streaming
Both partial and iterable streaming work with async clients:
```python
import asyncio
from pydantic import BaseModel
import instructor
class User(BaseModel):
name: str
age: int
async def async_partial_example():
client = instructor.from_provider("google/gemini-2.5-flash", async_client=True)
stream = client.create_partial(
model="gemini-2.5-flash",
response_model=User,
stream=True,
messages=[
{"role": "user", "content": "Jason is 25 years old"}
],
)
async for chunk in stream:
print(chunk)
async def async_iterable_example():
client = instructor.from_provider("google/gemini-2.5-flash", async_client=True)
stream = client.create_iterable(
model="gemini-2.5-flash",
response_model=User,
stream=True,
messages=[
{
"role": "user",
"content": "Jason is 25, Sarah is 30, Mike is 28"
}
],
)
async for user in stream:
print(user)
# Run async examples
asyncio.run(async_partial_example())
asyncio.run(async_iterable_example())
```
## Async Support
Instructor provides full async support for the genai SDK, allowing you to make non-blocking requests in async applications:
```python
import asyncio
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
async def extract_user():
client = instructor.from_provider(
"google/gemini-2.5-flash",
async_client=True,
)
response = await client.create(
messages=[{"role": "user", "content": "Extract: Jason is 25 years old"}],
response_model=User,
)
return response
print(asyncio.run(extract_user()))
#> name = Jason age= 25
```

444
참고/instructor-main/docs/integrations/google.md Normal file
View File

@@ -0,0 +1,444 @@
---
title: "Google Gemini Tutorial: Structured Outputs with Instructor"
description: "Learn how to use Google's Gemini models (Pro, Flash, Ultra) with Instructor for structured data extraction. Complete tutorial with examples for multimodal AI and type-safe outputs."
---
## See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Multi-Modal Examples](../examples/multi_modal_gemini.md) - Vision and multi-modal processing
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
# Google Gemini Tutorial: Structured Outputs with Instructor
Master structured data extraction using Google's Gemini models with Instructor. This comprehensive tutorial covers Gemini Pro, Flash, and Ultra models, including multimodal capabilities for processing text, images, and more.
## Google GenAI SDK
Google's GenAI SDK is the recommended way to access Gemini models. It provides a unified interface for both the Gemini API and Vertex AI. This guide shows you how to use Instructor with Google's GenAI SDK for type-safe, validated responses.
```bash
pip install "instructor[google-genai]"
```
## Simple User Example (Sync)
```python
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
# Using from_provider (recommended)
client = instructor.from_provider(
"google/gemini-3-flash",
)
resp = client.create(
response_model=User,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
)
print(resp) # User(name='Jason', age=25)
```
## Simple User Example (Async)
!!! info "Async Support"
Instructor supports async mode for the Google GenAI SDK. If you're using the async client, make sure that your client is declared within the same event loop as the function that calls it. If not you'll get a bunch of errors.
```python
import instructor
from pydantic import BaseModel
import asyncio
class User(BaseModel):
name: str
age: int
async def extract_user():
client = instructor.from_provider(
"google/gemini-3-flash",
async_client=True,
)
user = await client.create(
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user) # User(name='Jason', age=25)
```
## Configuration Options
You can customize the model's behavior using generation configuration parameters. These parameters control aspects like temperature, token limits, and sampling methods. Pass these parameters as a dictionary to the `generation_config` parameter when creating the response.
The most common parameters include:
- `temperature`: Controls randomness in the output (0.0 to 1.0)
- `max_tokens`: Maximum number of tokens to generate
- `top_p`: Nucleus sampling parameter
- `top_k`: Number of highest probability tokens to consider
For more details on configuration options, see [Google's documentation on Gemini configuration parameters](https://cloud.google.com/vertex-ai/generative-ai/docs/samples/generativeaionvertexai-gemini-pro-config-example){target="_blank"}.
```python
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"google/gemini-3-flash",
mode=instructor.Mode.JSON,
)
resp = client.create(
response_model=User,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
},
],
generation_config={
"temperature": 0.5,
"max_tokens": 1000,
"top_p": 1,
"top_k": 32,
},
)
print(resp)
```
## Safety settings with images
Google GenAI uses a different set of harm categories for image inputs (for example, `HARM_CATEGORY_IMAGE_HATE`).
When your request includes image content, Instructor will:
- Use the image-specific categories in the request config
- Map thresholds you pass for text categories (like `HARM_CATEGORY_HATE_SPEECH`) to the matching image category (like `HARM_CATEGORY_IMAGE_HATE`)
This avoids `400 INVALID_ARGUMENT` errors when you combine `safety_settings` with images.
```python
import instructor
from google.genai.types import HarmBlockThreshold, HarmCategory
from instructor.processing.multimodal import Image
from pydantic import BaseModel
class Result(BaseModel):
summary: str
client = instructor.from_provider("google/gemini-3-flash")
result = client.create(
response_model=Result,
messages=[
{
"role": "user",
"content": [
"Describe the image in one sentence.",
Image.autodetect("path/to/image.png"),
],
}
],
# You can still pass text categories. Instructor will map them for image inputs.
safety_settings={
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
},
)
print(result)
```
## Nested Example
```python
import instructor
from pydantic import BaseModel
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
client = instructor.from_provider(
"google/gemini-3-flash",
)
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
### Partials
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider(
"google/gemini-3-flash",
)
class User(BaseModel):
name: str
age: int
bio: str
user = client.create_partial(
messages=[
{
"role": "user",
"content": "Create a user profile for Jason and 1 sentence bio, age 25",
},
],
response_model=User,
)
for user_partial in user:
print(user_partial)
# > name=None age=None bio=None
# > name=None age=25 bio='Jason is a great guy'
# > name='Jason' age=25 bio='Jason is a great guy'
```
### Iterable Example
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider(
"google/gemini-3-flash",
)
class User(BaseModel):
name: str
age: int
# Extract multiple users from text
users = client.create_iterable(
messages=[
{
"role": "user",
"content": """
Extract users:
1. Jason is 25 years old
2. Sarah is 30 years old
3. Mike is 28 years old
""",
},
],
response_model=User,
)
for user in users:
print(user)
#> name='Jason' age=25
#> name='Sarah' age=30
#> name='Mike' age=28
```
## Known Limitations (as of Nov 12, 2024)
Google Gemini has the following known limitations when used with Instructor:
1. **Union Types**: Gemini does not support Union types (except for Optional). Use separate response models or Literal types instead.
2. **Enum Types**: Gemini returns string values instead of properly typed Enum instances. You may need to manually convert strings to enums after extraction.
3. **Union Streaming**: Streaming is not supported for Union types with Iterable.
These limitations are specific to Google Gemini and do not affect other providers like OpenAI or Anthropic. Tests automatically skip these features for Google to prevent failures.
## Instructor Modes
We provide several modes to make it easy to work with the different response models that Gemini supports:
1. `instructor.Mode.TOOLS` : This uses Gemini's tool calling API to return structured outputs (default)
2. `instructor.Mode.JSON` : This uses Gemini's JSON schema mode for structured outputs
!!! note "Backwards Compatibility"
Legacy provider-specific modes (for example `Mode.TOOLS`, `Mode.JSON`, `Mode.JSON`, `Mode.TOOLS`) are deprecated. They emit warnings and map to the generic modes.
!!! info "Mode Selection"
When using `from_provider`, the appropriate mode is automatically selected based on the provider and model capabilities.
## Available Models
Google offers several Gemini models:
- Gemini Flash (General purpose)
- Gemini Pro (Multimodal)
- Gemini Flash-8b (Coming soon)
## Using Gemini's Multimodal Capabilities
We've written an extensive list of guides on how to use gemini's multimodal capabilities with instructor.
- [Using Geminin To Extract Travel Video Recomendations](../blog/posts/multimodal-gemini.md)
- [Parsing PDFs with Gemini](../blog/posts/chat-with-your-pdf-with-gemini.md)
- [Generating Citations with Gemini](../blog/posts/generating-pdf-citations.md)
Stay tuned to the blog for more guides on using Gemini with instructor.
## Related Resources
- [Google AI Documentation](https://ai.google.dev/)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Migration from google-generativeai
If you're currently using the legacy `google-generativeai` package with Instructor, here's how to migrate:
### Old Way (Deprecated)
```python
import instructor
import google.generativeai as genai
client = instructor.from_provider(
"google/gemini-2.5-flash",
mode=instructor.Mode.JSON,
)
```
### New Way (Recommended)
```python
import instructor
# Option 1: Using from_provider (recommended)
client = instructor.from_provider("google/gemini-2.5-flash")
# Option 2: Using from_genai directly (legacy/advanced)
from google import genai
from instructor import from_genai
client = from_genai(genai.Client())
```
### Vertex AI Migration
For Vertex AI users, the migration is similar:
#### Old Way (Deprecated)
```python
import instructor
import vertexai
from vertexai.generative_models import GenerativeModel
vertexai.init(project="your-project", location="us-central1")
client = instructor.from_provider("google/gemini-2.5-flash", vertexai=True),
mode=instructor.Mode.TOOLS,
)
```
#### New Way (Recommended)
```python
import instructor
# Option 1: Using from_provider (recommended)
client = instructor.from_provider(
"vertexai/gemini-3-flash",
project="your-project",
location="us-central1"
)
# Option 2: Using from_genai with vertexai=True (legacy/advanced)
from google import genai
from instructor import from_genai
client = from_genai(
genai.Client(
vertexai=True,
project="your-project",
location="us-central1"
)
)
```
## Updates and Compatibility
Instructor maintains compatibility with Google's latest API versions. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.

155
참고/instructor-main/docs/integrations/groq.md Normal file
View File

@@ -0,0 +1,155 @@
---
title: Structured Outputs with Groq AI and Pydantic
description: Learn how to use Groq AI for structured outputs with Pydantic in Python and enhance API interactions.
---
# Structured Outputs with Groq AI
This guide demonstrates how to use Groq AI with Instructor to generate structured outputs. You'll learn how to use Groq's LLM models to create type-safe responses.
you'll need to sign up for an account and get an API key. You can do that [here](https://console.groq.com/docs/quickstart).
```bash
export GROQ_API_KEY=<your-api-key-here>
pip install "instructor[groq]"
```
### See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [Groq Examples](../examples/groq.md) - Practical Groq examples
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
# Groq AI
Groq supports structured outputs with their new `llama-3-groq-70b-8192-tool-use-preview` model.
### Sync Example
```python
import os
from groq import Groq
import instructor
from pydantic import BaseModel
# Initialize with API key
client = Groq(api_key=os.getenv("GROQ_API_KEY"))
# Enable instructor patches for Groq client
client = instructor.from_provider("groq/llama3-8b-8192")
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(user)
# > User(name='Jason', age=25)
```
### Async Example
```python
import instructor
from pydantic import BaseModel
import asyncio
# Initialize async client using provider string
client = instructor.from_provider(
"groq/llama3-8b-8192",
async_client=True,
)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
# > User(name='Jason', age=25)
```
### Nested Object
```python
import os
from groq import Groq
import instructor
from pydantic import BaseModel
# Initialize with API key
client = Groq(api_key=os.getenv("GROQ_API_KEY"))
# Enable instructor patches for Groq client
client = instructor.from_provider("groq/llama3-8b-8192")
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```

181
참고/instructor-main/docs/integrations/index.md Normal file
View File

@@ -0,0 +1,181 @@
---
title: "LLM Provider Integration Tutorials - Instructor"
description: "Complete tutorials for integrating Instructor with 15+ LLM providers. Learn structured data extraction with OpenAI, Anthropic Claude, Google Gemini, local models with Ollama, and more."
---
# LLM Provider Integration Tutorials
Learn how to integrate Instructor with various AI model providers. These comprehensive tutorials cover everything from cloud-based services like OpenAI and Anthropic to local open-source models, helping you extract structured outputs from any LLM.
<div class="grid cards" markdown>
- :material-cloud: **Major Cloud Providers**
Leading AI providers with comprehensive features
[:octicons-arrow-right-16: OpenAI](./openai.md) ·
[:octicons-arrow-right-16: OpenAI Responses](./openai-responses.md) ·
[:octicons-arrow-right-16: Azure](./azure.md) ·
[:octicons-arrow-right-16: Anthropic](./anthropic.md) ·
[:octicons-arrow-right-16: Google.GenerativeAI](./google.md) ·
[:octicons-arrow-right-16: Vertex AI](./vertex.md) ·
[:octicons-arrow-right-16: AWS Bedrock](./bedrock.md) ·
[:octicons-arrow-right-16: Google.GenAI](./genai.md) ·
[:octicons-arrow-right-16: xAI](./xai.md)
- :material-cloud-outline: **Additional Cloud Providers**
Other commercial AI providers with specialized offerings
[:octicons-arrow-right-16: Cohere](./cohere.md) ·
[:octicons-arrow-right-16: Mistral](./mistral.md) ·
[:octicons-arrow-right-16: DeepSeek](./deepseek.md) ·
[:octicons-arrow-right-16: Together AI](./together.md) ·
[:octicons-arrow-right-16: Groq](./groq.md) ·
[:octicons-arrow-right-16: Fireworks](./fireworks.md) ·
[:octicons-arrow-right-16: Cerebras](./cerebras.md) ·
[:octicons-arrow-right-16: Writer](./writer.md) ·
[:octicons-arrow-right-16: Perplexity](./perplexity.md)
[:octicons-arrow-right-16: SambaNova](./sambanova.md)
- :material-open-source-initiative: **Open Source**
Run open-source models locally or in the cloud
[:octicons-arrow-right-16: Ollama](./ollama.md) ·
[:octicons-arrow-right-16: llama-cpp-python](./llama-cpp-python.md)
- :material-router-wireless: **Routing**
Unified interfaces for multiple providers
[:octicons-arrow-right-16: LiteLLM](./litellm.md)
[:octicons-arrow-right-16: OpenRouter](./openrouter.md)
</div>
## Common Features
All integrations support these core features:
| Feature | Description | Documentation |
|---------|-------------|---------------|
| **Model Patching** | Enhance provider clients with structured output capabilities | [Patching](../concepts/patching.md) |
| **Response Models** | Define expected response schema with Pydantic | [Models](../concepts/models.md) |
| **Validation** | Ensure responses match your schema definition | [Validation](../concepts/validation.md) |
| **Streaming** | Stream partial or iterative responses | [Partial](../concepts/partial.md), [Iterable](../concepts/iterable.md) |
| **Hooks** | Add callbacks for monitoring and debugging | [Hooks](../concepts/hooks.md) |
However, each provider has different capabilities and limitations. Refer to the specific provider documentation for details.
## Provider Modes
Providers support different methods for generating structured outputs:
| Mode | Description | Providers |
|------|-------------|-----------|
| `TOOLS` | Uses OpenAI-style tools/function calling | OpenAI, Anthropic, Mistral |
| `PARALLEL_TOOLS` | Multiple simultaneous tool calls | OpenAI |
| `JSON` | Direct JSON response generation | OpenAI, Gemini, Cohere, GenAI |
| `MD_JSON` | JSON embedded in markdown | Most providers |
See the [Modes Comparison](../modes-comparison.md) guide for details.
## Getting Started
There are two ways to use providers with Instructor:
### 1. Using Provider Initialization (Recommended)
The simplest way to get started is using the provider initialization:
```python
import instructor
from pydantic import BaseModel
class UserInfo(BaseModel):
name: str
age: int
# Initialize any provider with a simple string
client = instructor.from_provider("openai/gpt-4")
# Or use async client
async_client = instructor.from_provider("anthropic/claude-3-sonnet", async_client=True)
# Use the same interface for all providers
response = client.create(
response_model=UserInfo,
messages=[{"role": "user", "content": "Your prompt"}]
)
```
Supported provider strings:
- `openai/model-name`: OpenAI models
- `anthropic/model-name`: Anthropic models
- `google/model-name`: Google models
- `mistral/model-name`: Mistral models
- `cohere/model-name`: Cohere models
- `perplexity/model-name`: Perplexity models
- `groq/model-name`: Groq models
- `writer/model-name`: Writer models
- `bedrock/model-name`: AWS Bedrock models
- `cerebras/model-name`: Cerebras models
- `fireworks/model-name`: Fireworks models
- `vertexai/model-name`: Vertex AI models
- `genai/model-name`: Google GenAI models
- `ollama/model-name`: Ollama models
### Provider Checklist
Use these example strings with `from_provider` to quickly get started:
- [x] `instructor.from_provider("openai/gpt-5-nano")`
- [x] `instructor.from_provider("anthropic/claude-3-sonnet")`
- [x] `instructor.from_provider("google/gemini-2.5-flash")`
- [x] `instructor.from_provider("mistral/mistral-large-latest")`
- [x] `instructor.from_provider("cohere/command-r")`
- [x] `instructor.from_provider("perplexity/sonar-small")`
- [x] `instructor.from_provider("groq/llama3-8b-8192")`
- [x] `instructor.from_provider("writer/palmyra-x-004")`
- [x] `instructor.from_provider("bedrock/anthropic.claude-3-sonnet-20240229-v1:0")`
- [x] `instructor.from_provider("cerebras/llama3.1-70b")`
- [x] `instructor.from_provider("fireworks/llama-v3-70b-instruct")`
- [x] `instructor.from_provider("vertexai/gemini-3-flash")`
- [x] `instructor.from_provider("genai/gemini-3-flash")`
- [x] `instructor.from_provider("ollama/llama3")`
### 2. Manual Client Setup
Alternatively, you can manually set up the client:
1. Install the required dependencies:
```bash
pip install "instructor[provider]" # e.g., instructor[anthropic]
```
2. Import the provider client and patch it with Instructor:
```python
import instructor
from provider_package import Client
client = instructor.from_provider(Client())
```
3. Use the patched client with your Pydantic model:
```python
response = client.create(
response_model=YourModel,
messages=[{"role": "user", "content": "Your prompt"}]
)
```
For provider-specific setup and examples, visit each provider's documentation page.
## Need Help?
If you need assistance with a specific integration:
1. Check the provider-specific documentation
2. Browse the [examples](../examples/index.md) and [cookbooks](../examples/index.md)
3. Search existing [GitHub issues](https://github.com/jxnl/instructor/issues)
4. Join our [Discord community](https://discord.gg/bD9YE9JArw)

121
참고/instructor-main/docs/integrations/litellm.md Normal file
View File

@@ -0,0 +1,121 @@
---
title: "Structured outputs with LiteLLM, a complete guide w/ instructor"
description: "Complete guide to using Instructor with LiteLLM's unified interface. Learn how to generate structured, type-safe outputs across multiple LLM providers."
---
# Structured outputs with LiteLLM, a complete guide w/ instructor
LiteLLM provides a unified interface for multiple LLM providers, making it easy to switch between different models and providers. This guide shows you how to use Instructor with LiteLLM for type-safe, validated responses across various LLM providers.
## Quick Start
Install Instructor with LiteLLM support:
```bash
pip install "instructor[litellm]"
```
## Simple User Example (Sync)
```python
from litellm import completion
import instructor
from pydantic import BaseModel
# Enable instructor patches
client = instructor.from_provider("litellm/gpt-3.5-turbo")
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(user) # User(name='Jason', age=25)
```
## Simple User Example (Async)
```python
import instructor
from pydantic import BaseModel
import asyncio
client = instructor.from_provider(
"litellm/gpt-3.5-turbo",
async_client=True,
)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user) # User(name='Jason', age=25)
```
## Cost Calculation
In order to calculate the cost of the response, LiteLLM provides a simple `response_cost` attribute on the response object's `_hidden_params` attribute. This is recorded in their documentation [here](https://docs.litellm.ai/docs/completion/token_usage#6-completion_cost).
Here is a code snippet using instructor to calculate the cost of the response:
```python
import instructor
from litellm import completion
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_provider("litellm/gpt-3.5-turbo")
instructor_resp, raw_completion = client.create_with_completion(
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
print(raw_completion._hidden_params["response_cost"])
#> 0.00189
```
## Related Resources
- [LiteLLM Documentation](https://docs.litellm.ai/)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Updates and Compatibility
Instructor maintains compatibility with LiteLLM's latest releases. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.
Note: Always verify provider-specific features and limitations in their respective documentation before implementation.

108
참고/instructor-main/docs/integrations/llama-cpp-python.md Normal file
View File

@@ -0,0 +1,108 @@
---
draft: False
date: 2024-02-12
title: "Structured outputs with llama-cpp-python, a complete guide w/ instructor"
description: "Complete guide to using Instructor with llama-cpp-python. Learn how to generate structured, type-safe outputs with llama-cpp-python."
slug: llama-cpp-python
tags:
- patching
authors:
- jxnl
---
# Structured outputs with llama-cpp-python, a complete guide w/ instructor
This guide demonstrates how to use llama-cpp-python with Instructor to generate structured outputs. You'll learn how to use JSON schema mode and speculative decoding to create type-safe responses from local LLMs.
Open-source LLMS are gaining popularity, and llama-cpp-python has made the `llama-cpp` model available to obtain structured outputs using JSON schema via a mixture of [constrained sampling](https://llama-cpp-python.readthedocs.io/en/latest/#json-schema-mode) and [speculative decoding](https://llama-cpp-python.readthedocs.io/en/latest/#speculative-decoding).
They also support a [OpenAI compatible client](https://llama-cpp-python.readthedocs.io/en/latest/#openai-compatible-web-server), which can be used to obtain structured output as a in process mechanism to avoid any network dependency.
<!-- more -->
## Patching
Instructor's patch enhances an create call it with the following features:
- `response_model` in `create` calls that returns a pydantic model
- `max_retries` in `create` calls that retries the call if it fails by using a backoff strategy
!!! note "Learn More"
To learn more, please refer to the [docs](../index.md). To understand the benefits of using Pydantic with Instructor, visit the tips and tricks section of the [why use Pydantic](../why.md) page. If you want to check out examples of using Pydantic with Instructor, visit the [examples](../examples/index.md) page.
### See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [Ollama Integration](./ollama.md) - Alternative local model setup
- [Local Classification](../examples/local_classification.md) - Classification with local models
- [Open Source Models](../examples/open_source.md) - More open-source model examples
# llama-cpp-python
Recently llama-cpp-python added support for structured outputs via JSON schema mode. This is a time-saving alternative to extensive prompt engineering and can be used to obtain structured outputs.
In this example we'll cover a more advanced use case of JSON_SCHEMA mode to stream out partial models. To learn more [partial streaming](https://github.com/jxnl/instructor/concepts/partial.md) check out partial streaming.
## Quick Start with `from_provider`
If you run the `llama-cpp-python` server in OpenAI compatible mode, you can use the unified `from_provider` API to patch the client. Simply point the base URL at your local server:
```python
import instructor
# Sync client
client = instructor.from_provider(
"ollama/openhermes", base_url="http://localhost:8080/v1"
)
# Async client
async_client = instructor.from_provider(
"ollama/openhermes", async_client=True, base_url="http://localhost:8080/v1"
)
```
You can then call `chat.completions.create` just like with any other provider.
```python
import llama_cpp
import instructor
from llama_cpp.llama_speculative import LlamaPromptLookupDecoding
from pydantic import BaseModel
llama = llama_cpp.Llama(
model_path="../../models/OpenHermes-2.5-Mistral-7B-GGUF/openhermes-2.5-mistral-7b.Q4_K_M.gguf",
n_gpu_layers=-1,
chat_format="chatml",
n_ctx=2048,
draft_model=LlamaPromptLookupDecoding(num_pred_tokens=2),
logits_all=True,
verbose=False,
)
create = instructor.patch(
create=llama.create_chat_completion_openai_v1,
mode=instructor.Mode.JSON_SCHEMA,
)
class UserDetail(BaseModel):
name: str
age: int
user = create(
messages=[
{
"role": "user",
"content": "Extract `Jason is 30 years old`",
}
],
response_model=UserDetail,
)
print(user)
#> name='Jason' age=30
```

348
참고/instructor-main/docs/integrations/mistral.md Normal file
View File

@@ -0,0 +1,348 @@
---
draft: False
date: 2025-03-11
title: "Structured outputs with Mistral, a complete guide w/ instructor"
description: "Complete guide to using Instructor with Mistral. Learn how to generate structured, type-safe outputs with Mistral."
slug: mistral
tags:
- patching
authors:
- shanktt
- ivanleomk
---
# Structured outputs with Mistral, a complete guide w/ instructor
This guide demonstrates how to use Mistral with Instructor to generate structured outputs. You'll learn how to use function calling with Mistral Large to create type-safe responses.
Mistral Large is the flagship model from Mistral AI, supporting 32k context windows and functional calling abilities. Mistral Large's addition of [function calling](https://docs.mistral.ai/guides/function-calling/) makes it possible to obtain structured outputs using JSON schema.
## Quick Start
To get started with Instructor and Mistral, you'll need to install the required packages:
```bash
pip install "instructor[mistral]"
```
⚠️ **Important**: You must set your Mistral API key by setting it explicitly on the client
```python
import os
from mistralai import Mistral
client = Mistral(api_key='your-api-key-here')
```
## Available Modes
Instructor provides two modes for working with Mistral:
1. `instructor.Mode.TOOLS`: Uses Mistral's function calling API to return structured outputs (default)
2. `instructor.Mode.JSON_SCHEMA`: Uses Mistral's structured output capabilities
To set the mode for your mistral client, simply use the code snippet below
```python
import os
from pydantic import BaseModel
import instructor
# Initialize with API key
instructor_client = instructor.from_provider(
"mistral/mistral-large-latest",
mode=Mode.TOOLS,
)
```
## Simple User Example (Sync)
```python
import os
from pydantic import BaseModel
import instructor
from instructor import Mode
class UserDetails(BaseModel):
name: str
age: int
# Initialize the client
instructor_client = instructor.from_provider(
"mistral/mistral-large-latest",
mode=Mode.TOOLS,
)
# Extract a single user
user = instructor_client.create(
response_model=UserDetails,
messages=[{"role": "user", "content": "Jason is 25 years old"}],
temperature=0,
)
print(user)
# Output: UserDetails(name='Jason', age=25)
```
## Async Example
For asynchronous operations, you can use the `use_async=True` parameter when creating the client:
```python
import os
import asyncio
from pydantic import BaseModel
import instructor
from instructor import Mode
class User(BaseModel):
name: str
age: int
# Initialize the async client
instructor_client = instructor.from_provider(
"mistral/mistral-large-latest",
async_client=True,
mode=Mode.TOOLS,
)
async def extract_user():
user = await instructor_client.create(
response_model=User,
messages=[{"role": "user", "content": "Jack is 28 years old."}],
temperature=0,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
# Output: User(name='Jack', age=28)
```
## Nested Example
You can also work with nested models:
```python
from pydantic import BaseModel
from typing import List
import os
import instructor
from instructor import Mode
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: List[Address]
# Initialize the client
instructor_client = instructor.from_provider(
"mistral/mistral-large-latest",
mode=Mode.TOOLS,
)
# Create structured output with nested objects
user = instructor_client.create(
response_model=User,
messages=[
{"role": "user", "content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
"""}
],
temperature=0,
)
print(user)
# Output:
# User(
# name='Jason',
# age=25,
# addresses=[
# Address(street='123 Main St', city='New York', country='USA'),
# Address(street='456 Beach Rd', city='Miami', country='USA')
# ]
# )
```
## Streaming Support
Instructor now supports streaming capabilities with Mistral! You can use both `create_partial` for incremental model building and `create_iterable` for streaming collections.
### Streaming Partial Responses
```python
from pydantic import BaseModel
import instructor
from mistralai import Mistral
from instructor.dsl.partial import Partial
class UserExtract(BaseModel):
name: str
age: int
# Initialize with API key
client = Mistral(api_key=os.environ.get("MISTRAL_API_KEY"))
# Enable instructor patches for Mistral client
instructor_client = instructor.from_provider("mistral/mistral-small")
# Stream partial responses
model = instructor_client.create(
response_model=Partial[UserExtract],
stream=True,
messages=[
{"role": "user", "content": "Jason Liu is 25 years old"},
],
)
for partial_user in model:
print(f"Received update: {partial_user}")
# Output might show:
# Received update: UserExtract(name='Jason', age=None)
# Received update: UserExtract(name='Jason Liu', age=None)
# Received update: UserExtract(name='Jason Liu', age=25)
```
### Streaming Iterable Collections
```python
from pydantic import BaseModel
import instructor
from mistralai import Mistral
class UserExtract(BaseModel):
name: str
age: int
# Initialize with API key
client = Mistral(api_key=os.environ.get("MISTRAL_API_KEY"))
# Enable instructor patches for Mistral client
instructor_client = instructor.from_provider("mistral/mistral-small")
# Stream iterable responses
users = instructor_client.create_iterable(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Make up two people"},
],
)
for user in users:
print(f"Generated user: {user}")
# Output:
# Generated user: UserExtract(name='Emily Johnson', age=32)
# Generated user: UserExtract(name='Michael Chen', age=28)
```
### Async Streaming
You can also use async versions of both streaming approaches:
```python
import asyncio
from pydantic import BaseModel
import instructor
from mistralai import Mistral
from instructor.dsl.partial import Partial
class UserExtract(BaseModel):
name: str
age: int
# Initialize client with async support
client = Mistral(api_key=os.environ.get("MISTRAL_API_KEY"))
instructor_client = instructor.from_provider("mistral/mistral-small")
async def stream_partial():
model = await instructor_client.create(
response_model=Partial[UserExtract],
stream=True,
messages=[
{"role": "user", "content": "Jason Liu is 25 years old"},
],
)
async for partial_user in model:
print(f"Received update: {partial_user}")
async def stream_iterable():
users = instructor_client.create_iterable(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Make up two people"},
],
)
async for user in users:
print(f"Generated user: {user}")
# Run async functions
asyncio.run(stream_partial())
asyncio.run(stream_iterable())
```
## Related Resources
- [Mistral AI Documentation](https://docs.mistral.ai/)
- [Mistral Function Calling Guide](https://docs.mistral.ai/guides/function-calling/)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Updates and Compatibility
Instructor maintains compatibility with the latest Mistral API versions and models. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates on Mistral integration features.
## Multimodal
Instructor makes it easy to analyse and extract semantic information from PDFs using Mistral's models. Let's see an example below with the sample PDF above where we'll load it in using our `from_url` method. Note that for now Mistral only supports document URLs.
```
from instructor.processing.multimodal import PDF
from pydantic import BaseModel
import instructor
from mistralai import Mistral
import os
class Receipt(BaseModel):
total: int
items: list[str]
client = instructor.from_provider("mistral/mistral-small")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
response = client.create(
response_model=Receipt,
max_tokens=1000,
messages=[
{
"role": "user",
"content": [
"Extract out the total and line items from the invoice",
PDF.from_url(
url
), # Also supports PDF.from_path() and PDF.from_base64()
],
},
],
)
print(response)
# > Receipt(total=220, items=['English Tea', 'Tofu'])
```

214
참고/instructor-main/docs/integrations/ollama.md Normal file
View File

@@ -0,0 +1,214 @@
---
draft: False
date: 2024-02-08
title: "Structured outputs with Ollama, a complete guide w/ instructor"
description: "Complete guide to using Instructor with Ollama. Learn how to generate structured, type-safe outputs with Ollama."
slug: ollama
tags:
- patching
- open source
authors:
- jxnl
---
# Structured outputs with Ollama, a complete guide w/ instructor
This guide demonstrates how to use Ollama with Instructor to generate structured outputs. You'll learn how to use JSON schema mode with local LLMs to create type-safe responses.
Open-source LLMS are gaining popularity, and the release of Ollama's OpenAI compatibility later it has made it possible to obtain structured outputs using JSON schema.
By the end of this blog post, you will learn how to effectively utilize instructor with ollama. But before we proceed, let's first explore the concept of patching.
<!-- more -->
## Patching
Instructor's patch enhances a openai api it with the following features:
- `response_model` in `create` calls that returns a pydantic model
- `max_retries` in `create` calls that retries the call if it fails by using a backoff strategy
- `timeout` parameter for controlling total retry duration (especially important for Ollama)
!!! note "Learn More"
To learn more, please refer to the [docs](../index.md). To understand the benefits of using Pydantic with Instructor, visit the tips and tricks section of the [why use Pydantic](../why.md) page.
## Timeout Handling with Ollama
Ollama integration now properly supports timeout parameters to ensure reliable request handling:
```python
from pydantic import BaseModel
import instructor
class Character(BaseModel):
name: str
age: int
client = instructor.from_provider(
"ollama/llama2",
mode=instructor.Mode.JSON,
)
resp = client.create(
messages=[
{
"role": "user",
"content": "Tell me about Harry Potter",
}
],
response_model=Character,
max_retries=2,
timeout=10.0, # Total timeout across all retry attempts
)
```
The timeout parameter ensures that:
- **Total timeout control**: Limits the total time spent across all retry attempts, not per individual attempt
- **Ollama compatibility**: Prevents timeout issues where retries would multiply the total wait time
- **Predictable behavior**: A 3-second timeout stays 3 seconds total, not 9+ seconds when retrying
!!! tip "Timeout Best Practices"
When using Ollama, especially with larger models, set appropriate timeout values based on your model's response time. The timeout applies to the total retry duration, making response times more predictable.
### See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [Ollama Examples](../examples/ollama.md) - Practical Ollama examples
- [Open Source Models](../examples/open_source.md) - More open-source model examples
- [Local Deployment](../examples/index.md#local-deployment) - Local model deployment guide
# Ollama
Start by downloading [Ollama](https://ollama.ai/download), and then pull a model such as Llama 2 or Mistral.
!!! tip "Make sure you update your `ollama` to the latest version!"
```
ollama pull llama2
```
## Quick Start with Auto Client
You can use Ollama with Instructor's auto client for a simple setup:
```python
import instructor
from pydantic import BaseModel
class Character(BaseModel):
name: str
age: int
# Simple setup - automatically configured for Ollama
client = instructor.from_provider("ollama/llama2")
resp = client.create(
messages=[{"role": "user", "content": "Tell me about Harry Potter"}],
response_model=Character,
)
```
### Async Example
```python
import instructor
from pydantic import BaseModel
import asyncio
async_client = instructor.from_provider(
"ollama/llama2",
async_client=True,
)
class Character(BaseModel):
name: str
age: int
async def get_character():
return await async_client.create(
messages=[{"role": "user", "content": "Tell me about Harry Potter"}],
response_model=Character,
)
print(asyncio.run(get_character()))
```
### Intelligent Mode Selection
The auto client automatically selects the best mode based on your model:
- **Function Calling Models** (llama3.1, llama3.2, llama4, mistral-nemo, qwen2.5, etc.): Uses `TOOLS` mode for enhanced function calling support
- **Other Models**: Uses `JSON` mode for structured output
```python
# These models automatically use TOOLS mode
client = instructor.from_provider("ollama/llama3.1")
client = instructor.from_provider("ollama/qwen2.5")
# Other models use JSON mode
client = instructor.from_provider("ollama/llama2")
```
You can also override the mode manually:
```python
import instructor
# Force JSON mode
client = instructor.from_provider("ollama/llama3.1", mode=instructor.Mode.JSON)
# Force TOOLS mode
client = instructor.from_provider("ollama/llama2", mode=instructor.Mode.TOOLS)
```
## Manual Setup
```python
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import List
import instructor
class Character(BaseModel):
name: str
age: int
fact: List[str] = Field(..., description="A list of facts about the character")
# enables `response_model` in create call
client = instructor.from_provider(
"ollama/llama2",
mode=instructor.Mode.JSON,
)
resp = client.create(
messages=[
{
"role": "user",
"content": "Tell me about the Harry Potter",
}
],
response_model=Character,
)
print(resp.model_dump_json(indent=2))
"""
{
"name": "Harry James Potter",
"age": 37,
"fact": [
"He is the chosen one.",
"He has a lightning-shaped scar on his forehead.",
"He is the son of James and Lily Potter.",
"He attended Hogwarts School of Witchcraft and Wizardry.",
"He is a skilled wizard and sorcerer.",
"He fought against Lord Voldemort and his followers.",
"He has a pet owl named Snowy."
]
}
"""
```

469
참고/instructor-main/docs/integrations/openai-responses.md Normal file
View File

@@ -0,0 +1,469 @@
---
title: "OpenAI Responses API Guide"
description: "Learn how to use Instructor's new Responses API with OpenAI models for structured outputs. Complete guide with examples and best practices."
---
# OpenAI Responses API Guide
The Responses API provides a more streamlined way to work with OpenAI models through Instructor. This guide covers everything you need to know about using the new Responses API for type-safe, validated outputs.
## Quick Start
```python
import instructor
from pydantic import BaseModel
# Initialize the client
client = instructor.from_provider(
"openai/gpt-4.1-mini", mode=instructor.Mode.RESPONSES_TOOLS
)
# Define your response model
class User(BaseModel):
name: str
age: int
# Create structured output
profile = client.responses.create(
input="Extract out Ivan is 28 years old",
response_model=User,
)
print(profile)
#> name='Ivan' age=28
```
## Response Modes
The Responses API supports two main modes:
1. `instructor.Mode.RESPONSES_TOOLS`: Standard mode for structured outputs
2. `instructor.Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS`: Enhanced mode that includes built-in tools like web search and file search
```python
# Initialize the client
client = instructor.from_provider(
"openai/gpt-4.1-mini", mode=instructor.Mode.RESPONSES_TOOLS
)
```
## Core Methods
The Responses API provides several methods for creating structured outputs. Here's how to use each one:
### Basic Creation
The `create` method is the simplest way to get a structured output:
=== "Sync"
```python
from pydantic import BaseModel
import instructor
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS
)
profile = client.responses.create(
input="Extract: Jason is 25 years old",
response_model=User,
)
print(profile) # User(name='Jason', age=25)
```
=== "Async"
```python
from pydantic import BaseModel
import instructor
import asyncio
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS,
async_client=True
)
async def main():
profile = await client.responses.create(
input="Extract: Jason is 25 years old",
response_model=User,
)
print(profile) # User(name='Jason', age=25)
asyncio.run(main())
```
### Create with Completion
If you need the original completion object from OpenAI, you can do so with the `create_with_completion` method. This is useful when you have specific methods and data that you need to work from.
=== "Sync"
```python
from pydantic import BaseModel
import instructor
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS
)
response, completion = client.responses.create_with_completion(
input="Extract: Jason is 25 years old",
response_model=User,
)
print(response) # User(name='Jason', age=25)
print(completion) # Raw completion object
```
=== "Async"
```python
from pydantic import BaseModel
import instructor
import asyncio
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS,
async_client=True
)
async def main():
response, completion = await client.responses.create_with_completion(
input="Extract: Jason is 25 years old",
response_model=User,
)
print(response) # User(name='Jason', age=25)
print(completion) # Raw completion object
asyncio.run(main())
```
### Iterable Creation
If you're interested in extracting multiple instances of the same object, we provide a convinient wrapper to be able to do so.
=== "Sync"
```python
from pydantic import BaseModel
from typing import Iterable
import instructor
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS
)
profiles = client.responses.create(
input="Generate three fake profiles",
response_model=Iterable[User],
)
for profile in profiles:
print(profile)
```
=== "Async"
```python
from pydantic import BaseModel
from typing import Iterable
import instructor
import asyncio
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS,
async_client=True
)
async def main():
profiles = await client.responses.create_iterable(
input="Generate three fake profiles",
response_model=User,
)
async for profile in profiles:
print(profile)
asyncio.run(main())
```
### Partial Creation
We also provide validated outputs that you can stream in real time. This is incredibly useful for working with dynamic generative UI.
=== "Sync"
```python
from pydantic import BaseModel
import instructor
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS
)
resp = client.responses.create_partial(
input="Generate a fake profile",
response_model=User,
)
for user in resp:
print(user) # Will show partial updates as they come in
```
=== "Async"
```python
from pydantic import BaseModel
import instructor
import asyncio
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS,
async_client=True
)
async def main():
resp = client.responses.create_partial(
input="Generate a fake profile",
response_model=User,
)
async for user in resp:
print(user) # Will show partial updates as they come in
asyncio.run(main())
```
## Built-In Tools
The Responses API comes with powerful built-in tools that enhance the model's capabilities. These tools are managed by OpenAI, so you don't need to implement any additional code to use them.
For the most up-to-date documentation on how to use these tools, please refer to the [OpenAI Documentation](https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses)
### Web Search
The web search tool allows models to search the internet for real-time information. This is particularly useful for getting up-to-date information or verifying facts.
Model responses that use the web search tool will include two parts:
- A web_search_call output item with the ID of the search call.
- A message output item containing:
1. The text result in message.content[0].text
2. Annotations message.content[0].annotations for the cited URLs
By default, the model's response will include inline citations for URLs found in the web search results.
In addition to this, the url_citation annotation object will contain the URL, title and location of the cited source. You can extract this information using the `create_with_completion` method.
=== "Sync"
```python
from pydantic import BaseModel
import instructor
class Citation(BaseModel):
id: int
url: str
class Summary(BaseModel):
citations: list[Citation]
summary: str
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS,
async_client=False,
)
response, completion = client.responses.create_with_completion(
input="What are some of the best places to visit in New York for Latin American food?",
tools=[{"type": "web_search_preview"}],
response_model=Summary,
)
print(response)
# > citations=[Citation(id=1,url=....)]
# > summary = New York City offers a rich variety of ...
```
=== "Async"
```python
from pydantic import BaseModel
import instructor
import asyncio
class Citation(BaseModel):
id: int
url: str
class Summary(BaseModel):
citations: list[Citation]
summary: str
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS,
async_client=True,
)
async def main():
response = await client.responses.create(
input="What are some of the best places to visit in New York for Latin American food?",
tools=[{"type": "web_search_preview"}],
response_model=Summary,
)
print(response)
asyncio.run(main())
# > citations=[Citation(id=1,url=....)]
# > summary = New York City offers a rich variety of ...
```
You can customize the web search behavior with additional parameters:
```python
response = client.responses.create(
input="What are the best restaurants around Granary Square?",
tools=[{
"type": "web_search_preview",
"user_location": {
"type": "approximate",
"country": "GB",
"city": "London",
"region": "London",
}
}],
response_model=Summary,
)
```
### File Search
The file search tool enables models to retrieve information from your knowledge base through semantic and keyword search. This is useful for augmenting the model's knowledge with your own documents.
This makes it easy to build RAG applications out of the box
=== "Sync"
```python
from pydantic import BaseModel
import instructor
class Citation(BaseModel):
file_id: int
file_name: str
excerpt: str
class Response(BaseModel):
citations: list[Citation]
response: str
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS
)
response = client.responses.create(
input="How much does the Kyoto itinerary cost?",
tools=[{
"type": "file_search",
"vector_store_ids": ["your_vector_store_id"],
"max_num_results": 2,
}],
response_model=Response,
)
```
=== "Async"
```python
from pydantic import BaseModel
import instructor
import asyncio
class Citation(BaseModel):
file_id: int
file_name: str
excerpt: str
class Response(BaseModel):
citations: list[Citation]
response: str
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS,
async_client=True
)
async def main():
response = await client.responses.create(
input="How much does the Kyoto itinerary cost?",
tools=[{
"type": "file_search",
"vector_store_ids": ["your_vector_store_id"],
"max_num_results": 2,
}],
response_model=Response,
)
asyncio.run(main())
```
## Related Resources
- [OpenAI Documentation](https://platform.openai.com/docs)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)

466
참고/instructor-main/docs/integrations/openai.md Normal file
View File

@@ -0,0 +1,466 @@
---
title: "Structured outputs with OpenAI, a complete guide with instructor"
description: "Learn how to use Instructor with OpenAI's models for type-safe, structured outputs. Complete guide with examples and best practices for GPT-4 and other OpenAI models."
---
# Structured outputs with OpenAI, a complete guide with instructor
OpenAI is the primary integration for Instructor, offering robust support for structured outputs with GPT-3.5, GPT-4, and future models. This guide covers everything you need to know about using OpenAI with Instructor for type-safe, validated responses.
## Quick Start
Instructor comes with support for OpenAI out of the box, so you don't need to install anything extra.
```bash
pip install "instructor"
```
⚠️ **Important**: You must set your OpenAI API key before using the client. You can do this in two ways:
1. Set the environment variable:
```bash
export OPENAI_API_KEY='your-api-key-here'
```
2. Or provide it directly to the client:
```python
import instructor
client = instructor.from_provider(
"openai/gpt-5-nano",
api_key='your-api-key-here',
)
```
## Simple User Example (Sync)
```python
import instructor
from pydantic import BaseModel
# Initialize client using provider string
client = instructor.from_provider("openai/gpt-5-nano")
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(user)
#> User(name='Jason', age=25)
```
## Simple User Example (Async)
```python
import instructor
from pydantic import BaseModel
import asyncio
# Initialize async client using provider string
client = instructor.from_provider("openai/gpt-5-nano", async_client=True)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
#> User(name='Jason', age=25)
```
## Responses API Mode
OpenAI now recommends the Responses API for new builds. Instructor exposes this API through two modes so you can keep the same interface while gaining better caching, stateful context, and optional built-in tools. Pass `mode=instructor.Mode.RESPONSES_TOOLS` when you want Instructor to call the Responses API instead of Chat Completions. Use `instructor.Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS` if you plan to use OpenAI-managed tools like web search or file search.
```python
import asyncio
from pydantic import BaseModel
import instructor
class SupportTicket(BaseModel):
issue: str
priority: str
client = instructor.from_provider(
"openai/gpt-4.1-mini",
mode=instructor.Mode.RESPONSES_TOOLS,
async_client=True,
)
async def create_ticket() -> SupportTicket:
return await client.create(
messages=[
{
"role": "user",
"content": "Log a high priority bug about failed password resets.",
}
],
response_model=SupportTicket,
)
ticket = asyncio.run(create_ticket())
print(ticket)
```
See the [OpenAI Responses API guide](./openai-responses.md) for a deeper walkthrough that includes built-in tool usage, streaming, and best practices.
## Nested Example
```python
from pydantic import BaseModel
from typing import List
import os
from openai import OpenAI
import instructor
from pydantic import BaseModel
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: List[Address]
# Initialize client
client = instructor.from_provider(
"openai/gpt-5-nano",
api_key=os.getenv('OPENAI_API_KEY'),
)
# Create structured output with nested objects
user = client.create(
messages=[
{"role": "user", "content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
"""},
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
## Multimodal
> We've provided a few different sample files for you to use to test out these new features. All examples below use these files.
>
> - (Audio) : A Recording of the Original Gettysburg Address : [gettysburg.wav](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/gettysburg.wav)
> - (Image) : An image of some blueberry plants [image.jpg](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg)
> - (PDF) : A sample PDF file which contains a fake invoice [invoice.pdf](https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf)
Instructor provides a unified, provider-agnostic interface for working with multimodal inputs like images, PDFs, and audio files. With Instructor's multimodal objects, you can easily load media from URLs, local files, or base64 strings using a consistent API that works across different AI providers (OpenAI, Anthropic, Mistral, etc.).
Instructor handles all the provider-specific formatting requirements behind the scenes, ensuring your code remains clean and future-proof as provider APIs evolve.
Let's see how to use the Image, Audio and PDF classes.
### Image
> For a more in-depth walkthrough of the Image component, check out the [docs here](../concepts/multimodal.md)
Instructor makes it easy to analyse and extract semantic information from images using OpenAI's GPT-4o models. [Click here](https://platform.openai.com/docs/models) to check if the model you'd like to use has vision capabilities.
Let's see an example below with the sample image above where we'll load it in using our `from_url` method.
Note that we support local files and base64 strings too with the `from_path` and the `from_base64` class methods.
```python
from instructor.processing.multimodal import Image
from pydantic import BaseModel, Field
import instructor
from openai import OpenAI
class ImageDescription(BaseModel):
objects: list[str] = Field(..., description="The objects in the image")
scene: str = Field(..., description="The scene of the image")
colors: list[str] = Field(..., description="The colors in the image")
client = instructor.from_provider("openai/gpt-5-nano")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/image.jpg"
# Multiple ways to load an image:
response = client.create(
response_model=ImageDescription,
messages=[
{
"role": "user",
"content": [
"What is in this image?",
# Option 1: Direct URL with autodetection
Image.from_url(url),
# Option 2: Local file
# Image.from_path("path/to/local/image.jpg")
# Option 3: Base64 string
# Image.from_base64("base64_encoded_string_here")
# Option 4: Autodetect
# Image.autodetect(<url|path|base64>)
],
},
],
)
print(response)
# Example output:
# ImageDescription(
# objects=['blueberries', 'leaves'],
# scene='A blueberry bush with clusters of ripe blueberries and some unripe ones against a cloudy sky',
# colors=['green', 'blue', 'purple', 'white']
# )
```
### PDF
Instructor makes it easy to analyse and extract semantic information from PDFs using OpenAI's GPT-4o models.
Let's see an example below with the sample PDF above where we'll load it in using our `from_url` method.
Note that we support local files and base64 strings too with the `from_path` and the `from_base64` class methods.
```python
from instructor.processing.multimodal import PDF
from pydantic import BaseModel, Field
import instructor
from openai import OpenAI
class Receipt(BaseModel):
total: int
items: list[str]
client = instructor.from_provider("openai/gpt-5-nano")
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/invoice.pdf"
# Multiple ways to load an PDF:
response = client.create(
response_model=Receipt,
messages=[
{
"role": "user",
"content": [
"Extract out the total and line items from the invoice",
# Option 1: Direct URL
PDF.from_url(url),
# Option 2: Local file
# PDF.from_path("path/to/local/invoice.pdf"),
# Option 3: Base64 string
# PDF.from_base64("base64_encoded_string_here")
# Option 4: Autodetect
# PDF.autodetect(<url|path|base64>)
],
},
],
)
print(response)
# > Receipt(total=220, items=['English Tea', 'Tofu'])
```
### Audio
Instructor makes it easy to analyse and extract semantic information from Audio files using OpenAI's GPT-4o models. Let's see an example below with the sample Audio file above where we'll load it in using our `from_url` method.
Note that we support local files and base64 strings too with the `from_path`
```python
from instructor.processing.multimodal import Audio
from pydantic import BaseModel
import instructor
from openai import OpenAI
class AudioDescription(BaseModel):
transcript: str
summary: str
speakers: list[str]
key_points: list[str]
url = "https://raw.githubusercontent.com/instructor-ai/instructor/main/tests/assets/gettysburg.wav"
client = instructor.from_provider("openai/gpt-5-nano")
response = client.create(
response_model=AudioDescription,
modalities=["text"],
audio={"voice": "alloy", "format": "wav"},
messages=[
{
"role": "user",
"content": [
"Please transcribe and analyze this audio:",
# Multiple loading options:
Audio.from_url(url),
# Option 2: Local file
# Audio.from_path("path/to/local/audio.mp3")
],
},
],
)
print(response)
# > transcript='Four score and seven years ago our fathers..."]
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
### Partials
```python
from pydantic import BaseModel
client = instructor.from_provider("openai/gpt-5-nano")
class User(BaseModel):
name: str
age: int
bio: str
user = client.create_partial(
messages=[
{"role": "user", "content": "Create a user profile for Jason, age 25"},
],
response_model=User,
)
for user_partial in user:
print(user_partial)
# > name='Jason' age=None bio='None'
# > name='Jason' age=25 bio='A tech'
# > name='Jason' age=25 bio='A tech enthusiast'
# > name='Jason' age=25 bio='A tech enthusiast who loves coding, gaming, and exploring new'
# > name='Jason' age=25 bio='A tech enthusiast who loves coding, gaming, and exploring new technologies'
```
### Iterable Example
```python
import os
from openai import OpenAI
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
# Extract multiple users from text
users = client.create_iterable(
messages=[
{"role": "user", "content": """
Extract users:
1. Jason is 25 years old
2. Sarah is 30 years old
3. Mike is 28 years old
"""},
],
response_model=User,
)
for user in users:
print(user)
#> name='Jason' age=25
#> name='Sarah' age=30
#> name='Mike' age=28
```
## Instructor Modes
We provide several modes to make it easy to work with the different response models that OpenAI supports
1. `instructor.Mode.RESPONSES_TOOLS` : Calls the OpenAI Responses API while keeping Instructor's familiar API. Best for new builds that want lower latency, better caching, and the new stateful context features.
2. `instructor.Mode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS` : Same as above, but automatically enables OpenAI's built-in tools (web search, file search, etc.) inside the Responses API.
3. `instructor.Mode.TOOLS` : This uses the [tool calling API](https://platform.openai.com/docs/guides/function-calling) to return structured outputs to the client.
4. `instructor.Mode.JSON` : This forces the model to return JSON by using [OpenAI's JSON mode](https://platform.openai.com/docs/guides/structured-outputs#json-mode).
5. `instructor.Mode.FUNCTIONS` : This uses OpenAI's function calling API to return structured outputs and will be deprecated in the future.
6. `instructor.Mode.PARALLEL_TOOLS` : This uses the [parallel tool calling API](https://platform.openai.com/docs/guides/function-calling#configuring-parallel-function-calling) to return structured outputs to the client. This allows the model to generate multiple calls in a single response.
7. `instructor.Mode.MD_JSON` : This makes a simple call to the OpenAI chat completion API and parses the raw response as JSON.
8. `instructor.Mode.TOOLS_STRICT` : This uses the new Open AI structured outputs API to return structured outputs to the client using constrained grammar sampling. This restricts users to a subset of the JSON schema.
9. `instructor.Mode.JSON_O1` : This is a mode for the `O1` model. We created a new mode because `O1` doesn't support any system messages, tool calling or streaming so you need to use this mode to use Instructor with `O1`.
In general, choose `Mode.RESPONSES_TOOLS` (or the built-in tools variant) when you're targeting the Responses API, and stick with `Mode.TOOLS` for classic Chat Completions integrations. Both modes keep schema handling identical, so switching between them is a single-line change.
## Batch API
We also support batching requests using the `create_batch` method. This is helpful if your request is not time sensitive because you'll get a 50% discount on the token cost.
Read more about how to use it [here](../examples/batch_job_oai.md)
## Best Practices
1. **Model Selection** : We recommend using gpt-4o-mini for simpler use cases because it's cheap and works well with a clearly defined objective for structured outputs. When the task is more ambigious, consider upgrading to `4o` or even `O1` depending on your needs
2. **Performance Optimization** : Streaming a response model is faster and should be done from the get-go. This is especially true if you're using a simple response model.
## Common Use Cases
- Data Extraction
- Form Parsing
- API Response Structuring
- Document Analysis
- Configuration Generation
## Related Resources
- [OpenAI Documentation](https://platform.openai.com/docs)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
- [OpenAI Responses API Guide](./openai-responses.md)
## Updates and Compatibility
Instructor maintains compatibility with the latest OpenAI API versions and models. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.

270
참고/instructor-main/docs/integrations/openrouter.md Normal file
View File

@@ -0,0 +1,270 @@
---
title: "Structured outputs with OpenRouter, a complete guide with instructor"
description: "Learn how to use Instructor with OpenRouter to access multiple LLM providers through a unified API. Get type-safe, structured outputs from various models including Qwen, Gemini, Mistral, and Cohere."
---
# Structured outputs with OpenRouter, a complete guide with instructor
OpenRouter provides a unified API to access multiple LLM providers, allowing you to easily switch between different models. This guide shows you how to use Instructor with OpenRouter for type-safe, validated responses across various LLM providers.
To set Provider specific configuration on the `openai` client, make sure to use the `extra_body` kwarg.
## Quick Start
⚠️ **Important**: Make sure that the model you're using has support for `Tool Calling` and/or `Structured Outputs` in the [OpenRouter models listing](https://openrouter.ai/models)
Instructor works with OpenRouter through the OpenAI client, so you don't need to install anything extra beyond the base package.
## Simple User Example (Sync)
We support simple tool calling with this
```python
from openai import OpenAI
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openrouter/google/gemini-2.0-flash-lite-001",
base_url="https://openrouter.ai/api/v1",
async_client=False
)
resp = client.create(
messages=[
{
"role": "user",
"content": "Ivan is 28 years old",
},
],
response_model=User,
extra_body={"provider": {"require_parameters": True}},
)
print(resp)
#> name='Ivan' age=20
```
## Simple User Example ( Async )
```python
import instructor
from pydantic import BaseModel
import asyncio
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"openrouter/google/gemini-2.0-flash-lite-001",
async_client=True,
)
async def extract_user():
user = await client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
extra_body={"provider": {"require_parameters": True}},
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
```
## Nested Object Example ( Sync )
```python
from pydantic import BaseModel
from openai import OpenAI
import instructor
from pydantic import BaseModel
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Initialize with API key
# Initialize client with base URL
client = instructor.from_provider(
"openrouter/google/gemini-2.0-flash-lite-001",
base_url="https://openrouter.ai/api/v1",
async_client=False
)
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
extra_body={"provider": {"require_parameters": True}},
response_model=User,
)
print(user)
#> name='Jason' age=25 addresses=[Address(street='123 Main St', city='New York', country='USA'), Address(street='456 Beach Rd', city='Miami', country='USA')]
```
## Structured Outputs (Sync)
⚠️ **Important**: Check that your chosen model supports `Structured Outputs` in the [OpenRouter models listing](https://openrouter.ai/models). Structured Outputs is a subset of Tool Calling that constrains the model's output to match your schema in order to produce valid JSON Schema.
Instructor also supports Structured Outputs with OpenRouter as documented in their API [here](https://openrouter.ai/docs/features/structured-outputs). Note that the following User model will throw an error if we use the OpenAI GPT-4o model like `openai/gpt-4o-2024-11-20` because OpenAI does not support using a regex pattern as part of their structured output schema.
```python
from pydantic import BaseModel, Field
from openai import OpenAI
import instructor
class User(BaseModel):
name: str
age: int
phone_number: str = Field(
pattern=r"^\+?1?\s*\(?(\d{3})\)?[-.\s]*(\d{3})[-.\s]*(\d{4})$"
)
# Initialize with API key
# Initialize client with base URL
client = instructor.from_provider(
"openrouter/google/gemini-2.0-flash-lite-001",
base_url="https://openrouter.ai/api/v1",
async_client=False
)
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old and his number is 1-212-456-7890
""",
},
],
response_model=User,
extra_body={"provider": {"require_parameters": True}},
)
print(user)
# > name='Jason' age=25 phone_number='+1 (212) 456-7890'
```
## JSON Mode
In the event that your model doesn't support tool calling, you will see the following error when you try to use `mode.TOOLS`
> instructor.exceptions.InstructorRetryException: Error code: 404 - {'error': {'message': 'No endpoints found that support tool use. To learn more about provider routing, visit: https://openrouter.ai/docs/provider-routing', 'code': 404}}
In this case, we recommend using the `JSON` mode instead as seen below.
```python
from pydantic import BaseModel, Field
from openai import OpenAI
import instructor
class User(BaseModel):
name: str
age: int
phone_number: str = Field(
pattern=r"^\+?1?\s*\(?(\d{3})\)?[-.\s]*(\d{3})[-.\s]*(\d{4})$"
)
# Initialize with API key
# Initialize client with base URL
client = instructor.from_provider(
"openrouter/google/gemini-2.0-flash-lite-001",
base_url="https://openrouter.ai/api/v1",
async_client=False
)
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old and his number is 1-212-456-7890
""",
},
],
response_model=User,
)
print(user)
```
## Streaming
You can also use streaming with as seen below using the `create_partial` method. While we're using JSON mode here, this should work with tool calling and structured outputs too.
```python
from pydantic import BaseModel, Field
from openai import OpenAI
import instructor
class User(BaseModel):
name: str
age: int
# Initialize with API key
# Initialize client with base URL
client = instructor.from_provider(
"openrouter/google/gemini-2.0-flash-lite-001",
base_url="https://openrouter.ai/api/v1",
)
# Create structured output with nested objects
user = client.create_partial(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old and his number is 1-212-456-7890
""",
},
],
response_model=User,
)
for chunk in user:
print(chunk)
# > name=None age=None
# > name='Jason' age=None
# > name='Jason' age=25
```

189
참고/instructor-main/docs/integrations/perplexity.md Normal file
View File

@@ -0,0 +1,189 @@
---
title: Structured Outputs with Perplexity AI and Pydantic
description: Learn how to use Perplexity AI with Instructor for structured JSON outputs using Pydantic models. Create type-safe, validated responses from Perplexity's Sonar models with Python.
---
# Structured Outputs with Perplexity AI
This guide demonstrates how to use Perplexity AI with Instructor to generate structured outputs. You'll learn how to use Perplexity's Sonar models with Pydantic to create type-safe, validated responses.
## Prerequisites
You'll need to sign up for a Perplexity account and get an API key. You can do that [here](https://www.perplexity.ai/).
```bash
export PERPLEXITY_API_KEY=<your-api-key-here>
pip install "instructor[perplexity]"
```
### See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
- [Search Examples](../examples/search.md) - Search query processing examples
# Perplexity AI
Perplexity AI provides access to powerful language models through their API. Instructor supports structured outputs with Perplexity's models using the OpenAI-compatible API.
### Sync Example
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider(
"perplexity/sonar-small-online",
api_key=os.getenv("PERPLEXITY_API_KEY"),
base_url="https://api.perplexity.ai",
)
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(user)
# > User(name='Jason', age=25)
```
### Async Example
```python
import instructor
from pydantic import BaseModel
import asyncio
async_client = instructor.from_provider(
"perplexity/sonar-small-online",
async_client=True,
)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
# > User(name='Jason', age=25)
```
### Nested Objects
```python
import os
from openai import OpenAI
import instructor
from pydantic import BaseModel
# Initialize with API key
client = instructor.from_provider(
"perplexity/sonar-small-online",
api_key=os.getenv("PERPLEXITY_API_KEY"),
base_url="https://api.perplexity.ai",
)
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> User(
#> name='Jason',
#> age=25,
#> addresses=[
#> Address(street='123 Main St', city='New York', country='USA'),
#> Address(street='456 Beach Rd', city='Miami', country='USA')
#> ]
#> )
```
## Supported Modes
Perplexity AI currently supports the following mode with Instructor:
- `PERPLEXITY_JSON`: Direct JSON response generation
```python
import os
from openai import OpenAI
import instructor
from instructor import Mode
from pydantic import BaseModel
# Initialize client with base URL
client = instructor.from_provider(
"perplexity/sonar-small-online",
api_key=os.getenv("PERPLEXITY_API_KEY"),
base_url="https://api.perplexity.ai",
)
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
response_model=User,
)
print(user)
# > User(name='Jason', age=25)
```
## Additional Resources
- [Perplexity API Documentation](https://docs.perplexity.ai/)
- [Perplexity API Reference](https://docs.perplexity.ai/reference/post_chat_completions)

79
참고/instructor-main/docs/integrations/sambanova.md Normal file
View File

@@ -0,0 +1,79 @@
---
title: SambaNova
description: Use Instructor with SambaNova's LLM API for structured outputs.
---
## See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
- [Enterprise Integration](../examples/index.md#enterprise-integration) - More enterprise examples
# SambaNova Integration
Instructor supports SambaNova's LLM API, allowing you to use structured outputs with their models.
## Installation
```bash
pip install "instructor[openai]"
```
## Basic Usage
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider("sambanova/Meta-Llama-3.1-405B-Instruct")
class User(BaseModel):
name: str
age: int
user = client.create(
messages=[
{"role": "user", "content": "Ivan is 28"},
],
response_model=User,
)
print(user)
# > User(name='Ivan', age=28)
```
## Async Usage
```python
import instructor
from pydantic import BaseModel
client = instructor.from_provider(
"sambanova/Meta-Llama-3.1-405B-Instruct",
async_client=True,
)
class User(BaseModel):
name: str
age: int
async def get_user():
user = await client.create(
messages=[
{"role": "user", "content": "Ivan is 28"},
],
response_model=User,
)
return user
# Run with asyncio
import asyncio
user = asyncio.run(get_user())
print(user)
# > User(name='Ivan', age=28)
```
## Available Models
Check the [SambaNova documentation](https://docs.sambanova.ai/cloud/docs/get-started/supported-models) for the latest model offerings and capabilities.

127
참고/instructor-main/docs/integrations/together.md Normal file
View File

@@ -0,0 +1,127 @@
---
draft: False
date: 2024-01-27
slug: together
title: "Structured outputs with Together AI, a complete guide w/ instructor"
description: "Complete guide to using Instructor with Together AI. Learn how to generate structured, type-safe outputs with Together AI."
tags:
- patching
- open source
authors:
- jxnl
---
# Structured outputs with Together AI, a complete guide with instructor
This guide demonstrates how to use Together AI with Instructor to generate structured outputs. You'll learn how to use function calling with Together's models to create type-safe responses.
Open-source LLMS are gaining popularity, and with the release of Together's Function calling models, its been easier than ever to get structured outputs.
By the end of this blog post, you will learn how to effectively utilize instructor with Together AI. But before we proceed, let's first explore the concept of patching.
!!! note "Other Languages"
This blog post is written in Python, but the concepts are applicable to other languages as well, as we currently have support for [Javascript](https://instructor-ai.github.io/instructor-js), [Elixir](https://hexdocs.pm/instructor/Instructor.html) and [PHP](https://github.com/cognesy/instructor-php/).
<!-- more -->
## Patching
Instructor's patch enhances the openai api it with the following features:
- `response_model` in `create` calls that returns a pydantic model
- `max_retries` in `create` calls that retries the call if it fails by using a backoff strategy
!!! note "Learn More"
To learn more, please refer to the [docs](../index.md). To understand the benefits of using Pydantic with Instructor, visit the tips and tricks section of the [why use Pydantic](../why.md) page.
### See Also
- [Getting Started](../getting-started.md) - Quick start guide
- [from_provider Guide](../concepts/from_provider.md) - Detailed client configuration
- [Provider Examples](../index.md#provider-examples) - Quick examples for all providers
- [Open Source Models](../examples/open_source.md) - More open-source model examples
# Together AI
The good news is that Together employs the same OpenAI client, and its models support some of these output modes too!
!!! note "Getting access"
If you want to try this out for yourself check out the [Together AI](https://www.together.ai/) website. You can get started [here](http://api.together.ai/).
```python
import os
from pydantic import BaseModel
import instructor
client = instructor.from_provider(
"together/Mixtral-8x7B-Instruct-v0.1",
api_key=os.environ["TOGETHER_API_KEY"],
base_url="https://api.together.xyz/v1",
)
# By default, the patch function will patch the ChatCompletion.create and ChatCompletion.create methods to support the response_model parameter
# Now, we can use the response_model parameter using only a base model
# rather than having to use the OpenAISchema class
class UserExtract(BaseModel):
name: str
age: int
user: UserExtract = client.create(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
assert isinstance(user, UserExtract), "Should be instance of UserExtract"
assert user.name.lower() == "jason"
assert user.age == 25
print(user.model_dump_json(indent=2))
"""
{
"name": "jason",
"age": 25
}
"""
{
"name": "Jason",
"age": 25,
}
```
### Async Example
```python
import instructor
from pydantic import BaseModel
import os
import asyncio
async_client = instructor.from_provider(
"together/Mixtral-8x7B-Instruct-v0.1",
async_client=True,
api_key=os.environ["TOGETHER_API_KEY"],
base_url="https://api.together.xyz/v1",
)
class UserExtract(BaseModel):
name: str
age: int
async def extract_user():
return await async_client.create(
response_model=UserExtract,
messages=[{"role": "user", "content": "Extract jason is 25 years old"}],
)
print(asyncio.run(extract_user()))
```
You can find more information about Together's function calling support [here](https://docs.together.ai/docs/function-calling).

162
참고/instructor-main/docs/integrations/truefoundry.md Normal file
View File

@@ -0,0 +1,162 @@
---
title: "TrueFoundry"
---
This guide provides instructions for integrating Instructor with the [TrueFoundry AI Gateway](https://www.truefoundry.com/ai-gateway) for structured data extraction from LLMs.
## What is TrueFoundry?
TrueFoundry provides an enterprise-ready [AI Gateway](https://www.truefoundry.com/ai-gateway) and integrates seamlessly with libraries like instructor, providing enterprise-grade AI features including cost tracking, security guardrails, and access controls.
## Prerequisites
Before integrating Instructor with TrueFoundry, ensure you have:
1. **TrueFoundry Account**: Create a [TrueFoundry account](https://www.truefoundry.com/register) with at least one model provider and generate a Personal Access Token by following the instructions in [Generating Tokens](https://docs.truefoundry.com/gateway/authentication). For a quick setup guide, see our [Gateway Quick Start](https://docs.truefoundry.com/gateway/quick-start)
2. **Instructor Installation**: Install Instructor using pip: `pip install instructor`
3. **OpenAI Library**: Install the OpenAI Python library: `pip install openai`
4. **Pydantic**: Install Pydantic for data validation: `pip install pydantic`
## Setup Process
### Step 1: Install Dependencies
```bash
pip install instructor openai pydantic
```
### Step 2: Configure Instructor with TrueFoundry Gateway
Get your TrueFoundry Gateway API key, base URL, and model name from the unified code snippet in your TrueFoundry playground:
<Frame>
<img src="../img/new-code-snippet.png" />
</Frame>
Here's how to configure Instructor to use TrueFoundry's AI Gateway:
```python
import instructor
from pydantic import BaseModel
from openai import OpenAI
# Configure OpenAI client to use TrueFoundry Gateway
client = OpenAI(
api_key="your-truefoundry-api-key", # Your TrueFoundry Personal Access Token
base_url="your-truefoundry-base-url", # Your TrueFoundry Gateway URL
)
# Patch the client with Instructor
instructor_client = instructor.from_provider("openai/gpt-4o")
# Define your Pydantic model for structured output
class User(BaseModel):
name: str
age: int
email: str
# Extract structured data
user_info = instructor_client.create(
model="openai-main/gpt-4o", # Your TrueFoundry model ID
response_model=User,
messages=[
{"role": "user", "content": "Extract user information: John Doe is 30 years old and his email is john@example.com"}
],
)
print(f"Name: {user_info.name}")
print(f"Age: {user_info.age}")
print(f"Email: {user_info.email}")
```
## Usage Examples
### Basic Structured Data Extraction
```python
import instructor
from pydantic import BaseModel
from openai import OpenAI
# Configure TrueFoundry Gateway
client = OpenAI(
api_key="your-truefoundry-api-key",
base_url="your-truefoundry-base-url",
)
instructor_client = instructor.from_provider("openai/gpt-4o")
# Define response structure
class ProductInfo(BaseModel):
name: str
price: float
category: str
in_stock: bool
# Extract product information
product = instructor_client.create(
model="openai-main/gpt-4o",
response_model=ProductInfo,
messages=[
{"role": "user", "content": "Extract product details: The iPhone 15 Pro costs $999, it's in the Electronics category and is currently available in stock."}
],
)
print(f"Product: {product.name}")
print(f"Price: ${product.price}")
print(f"Category: {product.category}")
print(f"In Stock: {product.in_stock}")
```
### Complex Data Structures with Lists
```python
import instructor
from pydantic import BaseModel
from typing import List
from openai import OpenAI
# Configure TrueFoundry Gateway
client = OpenAI(
api_key="your-truefoundry-api-key",
base_url="your-truefoundry-base-url",
)
instructor_client = instructor.from_provider("openai/gpt-4o")
class Task(BaseModel):
title: str
description: str
priority: str
estimated_hours: int
class ProjectPlan(BaseModel):
project_name: str
total_duration_weeks: int
tasks: List[Task]
# Extract complex project structure
project = instructor_client.create(
model="openai-main/gpt-4o",
response_model=ProjectPlan,
messages=[
{"role": "user", "content": """
Create a project plan for building a mobile app:
Project: Food Delivery App (8 weeks total)
Tasks:
1. UI/UX Design - Create user interface mockups and wireframes - High priority - 2 weeks
2. Backend Development - Build API and database - High priority - 3 weeks
3. Frontend Development - Build mobile app frontend - Medium priority - 2 weeks
4. Testing & QA - Test all features and fix bugs - Medium priority - 1 week
"""}
],
)
print(f"Project: {project.project_name}")
print(f"Duration: {project.total_duration_weeks} weeks")
print("\nTasks:")
for task in project.tasks:
print(f"- {task.title}: {task.description} ({task.priority} priority, {task.estimated_hours} weeks)")
```
That's it! You're now ready to use Instructor with TrueFoundry Gateway for robust, production-ready structured data extraction from LLMs.

287
참고/instructor-main/docs/integrations/vertex.md Normal file
View File

@@ -0,0 +1,287 @@
---
title: "Structured outputs with Vertex AI, a complete guide w/ instructor"
description: "Complete guide to using Instructor with Google Cloud's Vertex AI. Learn how to generate structured, type-safe outputs with enterprise-grade AI capabilities."
---
# Structured outputs with Vertex AI, a complete guide w/ instructor
Google Cloud's Vertex AI provides enterprise-grade AI capabilities with robust scaling and security features. This guide shows you how to use Instructor with Vertex AI for type-safe, validated responses.
!!! warning "Migration Notice"
The direct `from_vertexai` integration is being deprecated in favor of the unified `google-genai` SDK.
Please use `from_provider` or `from_genai` with `vertexai=True` for new projects.
See the [migration guide](#migration-to-google-genai) below.
## Quick Start
Install Instructor with Google GenAI support (which includes Vertex AI):
```bash
pip install "instructor[google-genai]"
```
## Simple User Example (Sync)
```python
import instructor
from pydantic import BaseModel
import os
# Set your project ID and location
os.environ["GOOGLE_CLOUD_PROJECT"] = "your-project-id"
os.environ["GOOGLE_CLOUD_LOCATION"] = "us-central1"
class User(BaseModel):
name: str
age: int
# Using from_provider (recommended)
client = instructor.from_provider(
"vertexai/gemini-3-flash",
)
resp = client.create(
response_model=User,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
)
print(resp)
#> User(name='Jason', age=25)
```
## Simple User Example (Async)
```python
import asyncio
import instructor
import vertexai # type: ignore
from vertexai.generative_models import GenerativeModel # type: ignore
from pydantic import BaseModel
vertexai.init()
class User(BaseModel):
name: str
age: int
client = instructor.from_provider(
"vertex_ai/gemini-1.5-pro-preview-0409",
async_client=True,
mode=instructor.Mode.TOOLS,
)
async def extract_user():
user = await client.create(
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user) # User(name='Jason', age=25)
```
## Streaming Support
Instructor now supports streaming capabilities with Vertex AI! You can use both `create_partial` for incremental model building and `create_iterable` for streaming collections.
### Streaming Partial Responses
```python
import vertexai # type: ignore
from vertexai.generative_models import GenerativeModel # type: ignore
import instructor
from pydantic import BaseModel
from instructor.dsl.partial import Partial
vertexai.init()
class UserExtract(BaseModel):
name: str
age: int
client = instructor.from_provider(
"vertex_ai/gemini-1.5-pro-preview-0409",
mode=instructor.Mode.TOOLS,
)
# Stream partial responses
response_stream = client.create(
response_model=Partial[UserExtract],
stream=True,
messages=[
{"role": "user", "content": "Anibal is 23 years old"},
],
)
for partial_user in response_stream:
print(f"Received update: {partial_user}")
# Output might show:
# Received update: UserExtract(name='Anibal', age=None)
# Received update: UserExtract(name='Anibal', age=23)
```
### Streaming Iterable Collections
```python
import vertexai # type: ignore
from vertexai.generative_models import GenerativeModel # type: ignore
import instructor
from pydantic import BaseModel
vertexai.init()
class UserExtract(BaseModel):
name: str
age: int
client = instructor.from_provider(
"vertex_ai/gemini-1.5-pro-preview-0409",
mode=instructor.Mode.TOOLS,
)
# Stream iterable responses
response_stream = client.create_iterable(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Make up two people"},
],
)
for user in response_stream:
print(f"Generated user: {user}")
# Output:
# Generated user: UserExtract(name='Sarah Johnson', age=32)
# Generated user: UserExtract(name='David Chen', age=27)
```
### Async Streaming
You can also use async versions of both streaming approaches:
```python
import asyncio
import vertexai # type: ignore
from vertexai.generative_models import GenerativeModel # type: ignore
import instructor
from pydantic import BaseModel
from instructor.dsl.partial import Partial
vertexai.init()
class UserExtract(BaseModel):
name: str
age: int
client = instructor.from_provider(
"vertex_ai/gemini-1.5-pro-preview-0409",
async_client=True,
mode=instructor.Mode.TOOLS,
)
async def stream_partial():
response_stream = await client.create(
response_model=Partial[UserExtract],
stream=True,
messages=[
{"role": "user", "content": "Anibal is 23 years old"},
],
)
async for partial_user in response_stream:
print(f"Received update: {partial_user}")
async def stream_iterable():
response_stream = client.create_iterable(
response_model=UserExtract,
messages=[
{"role": "user", "content": "Make up two people"},
],
)
async for user in response_stream:
print(f"Generated user: {user}")
# Run async functions
asyncio.run(stream_partial())
asyncio.run(stream_iterable())
```
## Related Resources
- [Vertex AI Documentation](https://cloud.google.com/vertex-ai/docs)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Migration to Google GenAI
The legacy `from_vertexai` method is being deprecated in favor of the unified Google GenAI SDK. Here's how to migrate:
### Old Way (Deprecated)
```python
import instructor
import vertexai
from vertexai.generative_models import GenerativeModel
vertexai.init(project="your-project", location="us-central1")
client = instructor.from_provider("google/gemini-2.5-flash", vertexai=True),
mode=instructor.Mode.TOOLS,
)
```
### New Way (Recommended)
```python
import instructor
# Option 1: Using from_provider (simplest)
client = instructor.from_provider(
"vertexai/gemini-3-flash",
project="your-project", # Optional if set in environment
location="us-central1" # Optional, defaults to us-central1
)
# Option 2: Using from_genai with Google GenAI SDK
from google import genai
from instructor import from_genai
client = from_genai(
genai.Client(
vertexai=True,
project="your-project",
location="us-central1",
model="gemini-3-flash"
)
)
```
### Environment Variables
You can also set these environment variables to avoid passing project/location each time:
```bash
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"
```
## Updates and Compatibility
Instructor maintains compatibility with Vertex AI's latest API versions. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.
Streaming support has been added for both partial responses and iterable collections, with both synchronous and asynchronous interfaces.

179
참고/instructor-main/docs/integrations/writer.md Normal file
View File

@@ -0,0 +1,179 @@
---
title: Structured Outputs with Writer, a complete guide with instructor
description: Learn how to use Writer for structured outputs using their latest Palmyra-X-004 model for more reliable system outputs
---
# Structured Outputs with Writer, a complete guide with instructor
This guide demonstrates how to use Writer for structured outputs using their latest Palmyra-X-004 model for more reliable system outputs.
You'll need to sign up for an account and get an API key. You can do that [here](https://writer.com).
```bash
export WRITER_API_KEY=<your-api-key-here>
pip install "instructor[writer]"
```
## Palmyra-X-004
Writer supports structured outputs with their latest Palmyra-X-004 model that introduces tool calling functionality
### Sync Example
```python
import instructor
from writerai import Writer
from pydantic import BaseModel
# Initialize Writer client
client = instructor.from_provider("writer/palmyra-x-004")
class User(BaseModel):
name: str
age: int
# Extract structured data
user = client.create(
messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
response_model=User,
)
print(user)
#> name='John' age=30
```
### Async Example
```python
import instructor
from pydantic import BaseModel
import asyncio
client = instructor.from_provider(
"writer/palmyra-x-004",
async_client=True,
)
class User(BaseModel):
name: str
age: int
async def extract_user():
# Extract structured data
user = await client.create(
messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
response_model=User,
)
print(user)
# > name='John' age=30
if __name__ == "__main__":
import asyncio
asyncio.run(extract_user())
```
## Nested Objects
Writer also supports nested objects, which is useful for extracting data from more complex responses.
```python
import instructor
from writerai import Writer
from pydantic import BaseModel
# Initialize Writer client
client = instructor.from_provider("writer/palmyra-x-004")
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: list[Address]
# Create structured output with nested objects
user = client.create(
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
## Streaming Support
Instructor has two main ways that you can use to stream responses out
1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.
We currently support streaming for Writer with native tool for both methods listed above.
### Partial Streaming
```python
import instructor
from writerai import Writer
from pydantic import BaseModel
client = instructor.from_provider("writer/palmyra-x-004")
class Person(BaseModel):
name: str
age: int
resp = client.create_partial(
messages=[
{
"role": "user",
"content": "Ivan is 27 and lives in Singapore",
}
],
response_model=Person,
)
for person in resp:
print(person)
# > name=None age=None
# > name='Ivan' age=None
# > name='Ivan' age=27
```

227
참고/instructor-main/docs/integrations/xai.md Normal file
View File

@@ -0,0 +1,227 @@
---
title: "Structured outputs with xAI, a complete guide with instructor"
description: "Learn how to use Instructor with xAI's Grok models for type-safe, structured outputs. Complete guide with examples and best practices."
---
# Structured outputs with xAI, a complete guide with instructor
xAI provides access to Grok models through the `xai-sdk` package, enabling structured outputs with Instructor. This guide covers everything you need to know about using xAI's Grok models with Instructor for type-safe, validated responses.
## Quick Start
Instructor is distributed without xAI dependencies by default. Install xAI support with the optional `xai` extra:
```bash
pip install "instructor[xai]"
```
Or using uv:
```bash
uv pip install "instructor[xai]"
```
⚠️ **Important**: You must set your xAI API key before using the client. You can do this in two ways:
1. Set the environment variable:
```bash
export XAI_API_KEY='your-api-key-here'
```
2. The xAI SDK will use this environment variable automatically.
## Simple User Example (Sync)
```python
import instructor
from pydantic import BaseModel
# Auto-configure xAI client
client = instructor.from_provider("xai/grok-3-mini")
class User(BaseModel):
name: str
age: int
# Create structured output
user = client.create(
response_model=User,
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
)
print(user)
#> User(name='Jason', age=25)
```
## Simple User Example (Async)
```python
import instructor
from pydantic import BaseModel
import asyncio
# Auto-configure async xAI client
client = instructor.from_provider("xai/grok-3-mini", async_client=True)
class User(BaseModel):
name: str
age: int
async def extract_user():
user = await client.create(
response_model=User,
messages=[
{"role": "user", "content": "Extract: Jason is 25 years old"},
],
)
return user
# Run async function
user = asyncio.run(extract_user())
print(user)
#> User(name='Jason', age=25)
```
## Nested Example
```python
from pydantic import BaseModel
from typing import List
import instructor
class Address(BaseModel):
street: str
city: str
country: str
class User(BaseModel):
name: str
age: int
addresses: List[Address]
# Auto-configure xAI client
client = instructor.from_provider("xai/grok-3-mini")
# Create structured output with nested objects
user = client.create(
response_model=User,
messages=[
{"role": "user", "content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
"""},
],
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```
## Instructor Modes
xAI supports the following modes:
1. `instructor.Mode.JSON` : Forces the model to return JSON output (default)
2. `instructor.Mode.TOOLS` : Uses function calling for structured outputs
```python
import instructor
from instructor import Mode
# Using JSON mode (default)
client = instructor.from_provider("xai/grok-3-mini", mode=Mode.JSON)
# Using TOOLS mode
client = instructor.from_provider("xai/grok-3-mini", mode=Mode.TOOLS)
```
## Available Models
xAI provides access to the following models:
- **grok-3** - The most capable Grok model for complex reasoning tasks
- **grok-3-mini** - A smaller, faster version optimized for speed and cost
## Limitations
### Streaming Support
⚠️ **Note**: Streaming responses (`create_iterable` and `create_partial`) are not yet supported due to differences in xAI's streaming API. See [issue #1663](https://github.com/567-labs/instructor/issues/1663) for updates.
### Python Version
⚠️ **Requires Python 3.10+**: The xAI SDK requires Python 3.10 or higher.
## Best Practices
### 1. API Key Management
Store your xAI API key securely using environment variables:
```bash
export XAI_API_KEY="your-api-key-here"
```
### 2. Model Selection
- Use `grok-3-mini` for:
- Simple extraction tasks
- High-volume processing
- Cost-sensitive applications
- Use `grok-3` for:
- Complex reasoning tasks
- Multi-step analysis
- Higher accuracy requirements
### 3. Error Handling
Always handle potential API errors gracefully:
```python
try:
user = client.create(
response_model=User,
messages=[{"role": "user", "content": "Extract user data"}],
)
except Exception as e:
print(f"Error: {e}")
```
## Common Use Cases
- Data Extraction from unstructured text
- Form parsing and validation
- Content classification
- Entity recognition
- Structured data generation
## Related Resources
- [xAI Documentation](https://docs.x.ai/)
- [Instructor Core Concepts](../concepts/index.md)
- [Type Validation Guide](../concepts/validation.md)
- [Advanced Usage Examples](../examples/index.md)
## Updates and Compatibility
Instructor maintains compatibility with the latest xAI SDK versions. Check the [changelog](https://github.com/jxnl/instructor/blob/main/CHANGELOG.md) for updates.