참고소스 수정본
This commit is contained in:
887
참고/instructor-main/NEW_PROVIDER_AGENT_INSTRUCTIONS.md
Normal file
887
참고/instructor-main/NEW_PROVIDER_AGENT_INSTRUCTIONS.md
Normal file
@@ -0,0 +1,887 @@
|
||||
# AI Agent Instructions: Creating a New Instructor Provider
|
||||
|
||||
**Instructions for AI coding agents to create a new provider for the instructor library.**
|
||||
|
||||
Copy these instructions to your AI coding agent when you want to add a new LLM provider to instructor. The agent will have everything needed to implement a complete, working provider.
|
||||
|
||||
**For human contributors:** See the quick reference template in [`instructor/providers/README.md`](instructor/providers/README.md#adding-a-new-provider)
|
||||
|
||||
---
|
||||
|
||||
## Mission
|
||||
|
||||
Create a complete, production-ready provider package for the instructor library that:
|
||||
- Follows the BaseProvider protocol exactly
|
||||
- Includes comprehensive tests using transcript fixtures
|
||||
- Has proper error handling and validation
|
||||
- Provides excellent documentation
|
||||
- Integrates seamlessly with the instructor plugin system
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before starting, ensure you have:
|
||||
- Provider name (e.g., "groq", "perplexity", "fireworks")
|
||||
- Provider's Python SDK package name and version
|
||||
- API documentation URL
|
||||
- Sample API key format (for documentation)
|
||||
- Knowledge of provider's chat completion API structure
|
||||
|
||||
## Step-by-Step Implementation
|
||||
|
||||
### Step 1: Project Structure Setup
|
||||
|
||||
**Note: This creates a new provider integration that follows instructor's existing patterns, not a separate package.**
|
||||
|
||||
Create the following structure in the instructor repository:
|
||||
|
||||
```
|
||||
instructor/providers/{provider}/
|
||||
├── __init__.py # Empty or basic exports
|
||||
├── client.py # from_{provider} function implementation
|
||||
└── utils.py # Provider-specific utilities
|
||||
|
||||
tests/llm/test_{provider}/
|
||||
├── __init__.py # Empty
|
||||
├── conftest.py # Test configuration & API key handling
|
||||
├── util.py # Models and modes configuration
|
||||
├── test_simple.py # Basic functionality tests
|
||||
├── test_stream.py # Streaming tests (if supported)
|
||||
├── test_format.py # Format/structure tests
|
||||
└── test_retries.py # Error handling tests
|
||||
|
||||
docs/integrations/
|
||||
└── {provider}.md # Provider documentation following existing pattern
|
||||
```
|
||||
|
||||
**Important: You're adding to the existing instructor codebase, not creating a separate package.**
|
||||
|
||||
### Step 2: Provider Client Implementation
|
||||
|
||||
#### File: `instructor/providers/{provider}/client.py`
|
||||
|
||||
Follow the exact pattern used by other providers in instructor. This creates a `from_{provider}` function:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, overload
|
||||
|
||||
import instructor
|
||||
from ...core.client import AsyncInstructor, Instructor
|
||||
|
||||
# Import the provider's SDK
|
||||
from {provider_sdk} import {SyncClient}, {AsyncClient} # Replace with actual imports
|
||||
|
||||
|
||||
@overload
|
||||
def from_{provider}(
|
||||
client: {SyncClient},
|
||||
mode: instructor.Mode = instructor.Mode.{PROVIDER}_TOOLS, # Default mode
|
||||
**kwargs: Any,
|
||||
) -> Instructor: ...
|
||||
|
||||
|
||||
@overload
|
||||
def from_{provider}(
|
||||
client: {AsyncClient},
|
||||
mode: instructor.Mode = instructor.Mode.{PROVIDER}_TOOLS, # Default mode
|
||||
**kwargs: Any,
|
||||
) -> AsyncInstructor: ...
|
||||
|
||||
|
||||
def from_{provider}(
|
||||
client: {SyncClient} | {AsyncClient},
|
||||
mode: instructor.Mode = instructor.Mode.{PROVIDER}_TOOLS, # Default mode
|
||||
**kwargs: Any,
|
||||
) -> Instructor | AsyncInstructor:
|
||||
"""
|
||||
Create an instructor client from a {Provider} client
|
||||
|
||||
Args:
|
||||
client: {Provider} sync or async client instance
|
||||
mode: Mode to use for structured outputs
|
||||
**kwargs: Additional arguments passed to instructor client
|
||||
|
||||
Returns:
|
||||
Instructor or AsyncInstructor instance
|
||||
"""
|
||||
# Define valid modes for this provider
|
||||
valid_modes = {
|
||||
instructor.Mode.{PROVIDER}_TOOLS,
|
||||
instructor.Mode.{PROVIDER}_JSON,
|
||||
# Add other modes your provider supports
|
||||
}
|
||||
|
||||
# Validate mode
|
||||
if mode not in valid_modes:
|
||||
from ...core.exceptions import ModeError
|
||||
raise ModeError(
|
||||
mode=str(mode),
|
||||
provider="{Provider}",
|
||||
valid_modes=[str(m) for m in valid_modes],
|
||||
)
|
||||
|
||||
# Validate client type
|
||||
if not isinstance(client, ({AsyncClient}, {SyncClient})):
|
||||
from ...core.exceptions import ClientError
|
||||
raise ClientError(
|
||||
f"Client must be an instance of {SyncClient} or {AsyncClient}. "
|
||||
f"Got: {type(client).__name__}"
|
||||
)
|
||||
|
||||
# Handle async client
|
||||
if isinstance(client, {AsyncClient}):
|
||||
|
||||
async def async_wrapper(*args: Any, **kwargs: Any):
|
||||
"""Wrapper for async client calls"""
|
||||
if "stream" in kwargs and kwargs["stream"] is True:
|
||||
# Handle streaming if supported
|
||||
return client.chat.completions.acreate(*args, **kwargs)
|
||||
return await client.chat.completions.acreate(*args, **kwargs)
|
||||
|
||||
return AsyncInstructor(
|
||||
client=client,
|
||||
create=instructor.patch(create=async_wrapper, mode=mode),
|
||||
provider=instructor.Provider.{PROVIDER}, # Must be defined in Provider enum
|
||||
mode=mode,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
# Handle sync client
|
||||
if isinstance(client, {SyncClient}):
|
||||
return Instructor(
|
||||
client=client,
|
||||
create=instructor.patch(create=client.chat.completions.create, mode=mode),
|
||||
provider=instructor.Provider.{PROVIDER}, # Must be defined in Provider enum
|
||||
mode=mode,
|
||||
**kwargs,
|
||||
)
|
||||
```
|
||||
|
||||
### Step 3: Mode Handlers Implementation
|
||||
|
||||
#### File: `instructor_{provider}/handlers.py`
|
||||
|
||||
```python
|
||||
"""
|
||||
Mode handlers for {Provider} provider
|
||||
|
||||
Each handler knows how to:
|
||||
1. Format requests for the specific mode (TOOLS, JSON, etc.)
|
||||
2. Parse responses back into Pydantic models
|
||||
3. Handle provider-specific response formats
|
||||
"""
|
||||
|
||||
from typing import Dict, Any, Type, Union
|
||||
from pydantic import BaseModel
|
||||
from instructor.mode import Mode
|
||||
from instructor.function_calls import openai_schema
|
||||
import json
|
||||
|
||||
class BaseModeHandler:
|
||||
"""Base class for mode handlers"""
|
||||
|
||||
def __init__(self, provider):
|
||||
self.provider = provider
|
||||
|
||||
def prepare_request(
|
||||
self,
|
||||
response_model: Type[BaseModel],
|
||||
messages: list,
|
||||
model: str,
|
||||
**kwargs
|
||||
) -> Dict[str, Any]:
|
||||
"""Prepare request for this mode"""
|
||||
raise NotImplementedError
|
||||
|
||||
def parse_response(self, response: Any, response_model: Type[BaseModel]) -> BaseModel:
|
||||
"""Parse provider response into Pydantic model"""
|
||||
raise NotImplementedError
|
||||
|
||||
class ToolsHandler(BaseModeHandler):
|
||||
"""Handler for function/tool calling mode"""
|
||||
|
||||
def prepare_request(self, response_model, messages, model, **kwargs):
|
||||
# Convert Pydantic model to function schema
|
||||
schema = openai_schema(response_model)
|
||||
|
||||
return {
|
||||
"model": model,
|
||||
"messages": messages,
|
||||
"tools": [{
|
||||
"type": "function",
|
||||
"function": schema
|
||||
}],
|
||||
"tool_choice": "auto", # or provider-specific equivalent
|
||||
**kwargs
|
||||
}
|
||||
|
||||
def parse_response(self, response, response_model):
|
||||
# Extract function call from response
|
||||
# This is provider-specific - adapt to your provider's response format
|
||||
|
||||
if hasattr(response, 'choices') and response.choices:
|
||||
choice = response.choices[0]
|
||||
if hasattr(choice.message, 'tool_calls') and choice.message.tool_calls:
|
||||
tool_call = choice.message.tool_calls[0]
|
||||
function_args = json.loads(tool_call.function.arguments)
|
||||
return response_model(**function_args)
|
||||
|
||||
raise ValueError("No valid tool call found in response")
|
||||
|
||||
class JSONHandler(BaseModeHandler):
|
||||
"""Handler for JSON mode responses"""
|
||||
|
||||
def prepare_request(self, response_model, messages, model, **kwargs):
|
||||
# Add JSON schema to system message
|
||||
schema_prompt = f"""
|
||||
You must respond with valid JSON that matches this schema:
|
||||
{response_model.model_json_schema()}
|
||||
|
||||
Respond with only the JSON, no additional text.
|
||||
"""
|
||||
|
||||
# Add schema to messages
|
||||
enhanced_messages = [
|
||||
{"role": "system", "content": schema_prompt}
|
||||
] + messages
|
||||
|
||||
return {
|
||||
"model": model,
|
||||
"messages": enhanced_messages,
|
||||
"response_format": {"type": "json_object"}, # if provider supports
|
||||
**kwargs
|
||||
}
|
||||
|
||||
def parse_response(self, response, response_model):
|
||||
# Extract JSON from response content
|
||||
if hasattr(response, 'choices') and response.choices:
|
||||
content = response.choices[0].message.content
|
||||
try:
|
||||
data = json.loads(content)
|
||||
return response_model(**data)
|
||||
except json.JSONDecodeError as e:
|
||||
raise ValueError(f"Invalid JSON in response: {e}")
|
||||
|
||||
raise ValueError("No valid response content found")
|
||||
|
||||
# Handler registry
|
||||
_HANDLERS = {
|
||||
Mode.TOOLS: ToolsHandler,
|
||||
Mode.JSON: JSONHandler,
|
||||
# Add other modes as supported by provider
|
||||
}
|
||||
|
||||
def get_handler(mode: Mode, provider) -> BaseModeHandler:
|
||||
"""Get handler instance for the specified mode"""
|
||||
if mode not in _HANDLERS:
|
||||
supported = ", ".join(h.name for h in _HANDLERS.keys())
|
||||
raise ValueError(f"Mode {mode} not supported. Supported modes: {supported}")
|
||||
|
||||
handler_class = _HANDLERS[mode]
|
||||
return handler_class(provider)
|
||||
```
|
||||
|
||||
### Step 4: Package Configuration
|
||||
|
||||
#### File: `pyproject.toml`
|
||||
|
||||
```toml
|
||||
[project]
|
||||
name = "instructor-{provider}"
|
||||
version = "0.1.0"
|
||||
description = "Instructor provider for {Provider Name}"
|
||||
authors = [
|
||||
{name = "Your Name", email = "your.email@example.com"}
|
||||
]
|
||||
license = {text = "MIT"}
|
||||
requires-python = ">=3.9"
|
||||
dependencies = [
|
||||
"instructor-core>=2.0.0,<3.0.0",
|
||||
"{provider_sdk}>=X.X.X,<Y.0.0", # Replace with actual version constraints
|
||||
"pydantic>=2.8.0,<3.0.0",
|
||||
]
|
||||
|
||||
readme = "README.md"
|
||||
keywords = ["instructor", "llm", "structured-output", "{provider}"]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/instructor-ai/instructor"
|
||||
Documentation = "https://python.useinstructor.com"
|
||||
Repository = "https://github.com/instructor-ai/instructor"
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.3.3,<9.0.0",
|
||||
"pytest-asyncio>=0.24.0,<1.0.0",
|
||||
"pytest-mock>=3.12.0",
|
||||
"responses>=0.24.0", # For HTTP mocking
|
||||
"python-dotenv>=1.0.1",
|
||||
]
|
||||
|
||||
# Register the provider with instructor's plugin system
|
||||
[project.entry-points."instructor.providers"]
|
||||
{provider} = "instructor_{provider}:{Provider}Provider"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
markers = [
|
||||
"unit: Unit tests (fast, no external dependencies)",
|
||||
"integration: Integration tests (may require API keys)",
|
||||
"live: Live API tests (requires valid API key)"
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py39"
|
||||
line-length = 88
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "W", "I", "N", "B", "A", "C4", "T20"]
|
||||
ignore = ["E501"] # Line too long (handled by formatter)
|
||||
```
|
||||
|
||||
### Step 3: Testing Implementation
|
||||
|
||||
#### File: `tests/llm/test_{provider}/conftest.py`
|
||||
|
||||
Follow the exact pattern used by all other providers:
|
||||
|
||||
```python
|
||||
import os
|
||||
import pytest
|
||||
|
||||
# Skip entire test suite if API key is missing
|
||||
if not os.getenv("{PROVIDER}_API_KEY"):
|
||||
pytest.skip(
|
||||
"{PROVIDER}_API_KEY environment variable not set",
|
||||
allow_module_level=True,
|
||||
)
|
||||
|
||||
# Skip if provider package is not installed
|
||||
try:
|
||||
from {provider_sdk} import {SyncClient}, {AsyncClient} # Replace with actual imports
|
||||
except ImportError:
|
||||
pytest.skip("{provider_sdk} package is not installed", allow_module_level=True)
|
||||
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def client():
|
||||
"""Sync client fixture"""
|
||||
yield {SyncClient}()
|
||||
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def aclient():
|
||||
"""Async client fixture"""
|
||||
yield {AsyncClient}()
|
||||
```
|
||||
|
||||
#### File: `tests/llm/test_{provider}/util.py`
|
||||
|
||||
Define supported models and modes:
|
||||
|
||||
```python
|
||||
import instructor
|
||||
|
||||
# Replace with actual model names your provider supports
|
||||
models = ["provider-model-name-1", "provider-model-name-2"]
|
||||
|
||||
# Replace with actual modes your provider supports
|
||||
modes = [
|
||||
instructor.Mode.{PROVIDER}_TOOLS,
|
||||
instructor.Mode.{PROVIDER}_JSON,
|
||||
]
|
||||
```
|
||||
|
||||
#### File: `tests/llm/test_{provider}/test_simple.py`
|
||||
|
||||
Follow the standard pattern for basic functionality tests:
|
||||
|
||||
```python
|
||||
import instructor
|
||||
from {provider_sdk} import {SyncClient}, {AsyncClient} # Replace with actual imports
|
||||
from pydantic import BaseModel, field_validator
|
||||
import pytest
|
||||
from itertools import product
|
||||
from .util import models, modes
|
||||
|
||||
|
||||
class User(BaseModel):
|
||||
"""Standard test model"""
|
||||
name: str
|
||||
age: int
|
||||
|
||||
|
||||
@pytest.mark.parametrize("model, mode", product(models, modes))
|
||||
def test_{provider}_sync(model: str, mode: instructor.Mode, client):
|
||||
"""Test basic sync functionality"""
|
||||
client = instructor.from_{provider}(client, mode=mode)
|
||||
|
||||
resp = client.chat.completions.create(
|
||||
model=model,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Extract a user from this sentence: Ivan is 27 and lives in Singapore",
|
||||
},
|
||||
],
|
||||
response_model=User,
|
||||
)
|
||||
|
||||
assert resp.name.lower() == "ivan"
|
||||
assert resp.age == 27
|
||||
|
||||
|
||||
@pytest.mark.parametrize("model, mode", product(models, modes))
|
||||
def test_{provider}_sync_validated(model: str, mode: instructor.Mode, client):
|
||||
"""Test sync with validation retries"""
|
||||
class ValidatedUser(BaseModel):
|
||||
name: str
|
||||
age: int
|
||||
|
||||
@field_validator("name")
|
||||
def name_validator(cls, v: str) -> str:
|
||||
if not v.isupper():
|
||||
raise ValueError(
|
||||
f"All letters in the name must be uppercase (Eg. JOHN, SMITH) - {v} is not a valid example."
|
||||
)
|
||||
return v
|
||||
|
||||
client = instructor.from_{provider}(client, mode=mode)
|
||||
|
||||
resp = client.chat.completions.create(
|
||||
model=model,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Extract a user from this sentence: Ivan is 27 and lives in Singapore",
|
||||
},
|
||||
],
|
||||
max_retries=5,
|
||||
response_model=ValidatedUser,
|
||||
)
|
||||
|
||||
assert resp.name == "IVAN"
|
||||
assert resp.age == 27
|
||||
|
||||
|
||||
@pytest.mark.parametrize("model, mode", product(models, modes))
|
||||
@pytest.mark.asyncio(scope="session")
|
||||
async def test_{provider}_async(model: str, mode: instructor.Mode, aclient):
|
||||
"""Test async functionality"""
|
||||
client = instructor.from_{provider}(aclient, mode=mode)
|
||||
|
||||
resp = await client.chat.completions.create(
|
||||
model=model,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Extract a user from this sentence: Ivan is 27 and lives in Singapore",
|
||||
},
|
||||
],
|
||||
response_model=User,
|
||||
)
|
||||
|
||||
assert resp.name.lower() == "ivan"
|
||||
assert resp.age == 27
|
||||
|
||||
|
||||
@pytest.mark.parametrize("model, mode", product(models, modes))
|
||||
@pytest.mark.asyncio(scope="session")
|
||||
async def test_{provider}_async_validated(model: str, mode: instructor.Mode, aclient):
|
||||
"""Test async with validation retries"""
|
||||
class ValidatedUser(BaseModel):
|
||||
name: str
|
||||
age: int
|
||||
|
||||
@field_validator("name")
|
||||
def name_validator(cls, v: str) -> str:
|
||||
if not v.isupper():
|
||||
raise ValueError(
|
||||
f"Make sure to uppercase all letters in the name field. Examples include: JOHN, SMITH, etc. {v} is not a valid example."
|
||||
)
|
||||
return v
|
||||
|
||||
client = instructor.from_{provider}(aclient, mode=mode)
|
||||
|
||||
resp = await client.chat.completions.create(
|
||||
model=model,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Extract a user from this sentence: Ivan is 27 and lives in Singapore",
|
||||
},
|
||||
],
|
||||
response_model=ValidatedUser,
|
||||
max_retries=5,
|
||||
)
|
||||
|
||||
assert resp.name == "IVAN"
|
||||
assert resp.age == 27
|
||||
```
|
||||
|
||||
### Step 4: Required Infrastructure Updates
|
||||
|
||||
#### A. Add Mode Constants
|
||||
|
||||
Add your provider's modes to `instructor/mode.py`:
|
||||
|
||||
```python
|
||||
# Add to the Mode enum class
|
||||
{PROVIDER}_TOOLS = "{provider}_tools"
|
||||
{PROVIDER}_JSON = "{provider}_json"
|
||||
# Add other modes as needed
|
||||
```
|
||||
|
||||
#### B. Add Provider to Enum
|
||||
|
||||
Add your provider to `instructor/utils/providers.py`:
|
||||
|
||||
```python
|
||||
# Add to the Provider enum
|
||||
{PROVIDER} = "{provider}"
|
||||
```
|
||||
|
||||
#### C. Update Main __init__.py
|
||||
|
||||
Add conditional import to `instructor/__init__.py`:
|
||||
|
||||
```python
|
||||
# Add this block with the other provider imports
|
||||
if importlib.util.find_spec("{provider_sdk}") is not None:
|
||||
from .providers.{provider}.client import from_{provider}
|
||||
|
||||
__all__ += ["from_{provider}"]
|
||||
```
|
||||
|
||||
#### D. Add to pyproject.toml
|
||||
|
||||
Add your provider to the optional dependencies:
|
||||
|
||||
```toml
|
||||
# In [project.optional-dependencies]
|
||||
{provider} = ["{provider_sdk}>=X.X.X,<Y.0.0"] # Replace with actual version
|
||||
|
||||
# In [dependency-groups]
|
||||
{provider} = ["{provider_sdk}>=X.X.X,<Y.0.0"]
|
||||
```
|
||||
|
||||
### Step 5: Documentation
|
||||
|
||||
#### File: `docs/integrations/{provider}.md`
|
||||
|
||||
Follow the exact pattern of existing provider docs:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: "Structured outputs with {Provider}, a complete guide w/ instructor"
|
||||
description: "Complete guide to using Instructor with {Provider} models. Learn how to generate structured, type-safe outputs with {provider description}."
|
||||
---
|
||||
|
||||
# Structured outputs with {Provider}, a complete guide w/ instructor
|
||||
|
||||
{Provider description and benefits}. This guide shows you how to use Instructor with {Provider}'s models for type-safe, validated responses.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Install Instructor with {Provider} support:
|
||||
|
||||
```bash
|
||||
pip install "instructor[{provider}]"
|
||||
```
|
||||
|
||||
## Simple User Example (Sync)
|
||||
|
||||
```python
|
||||
from {provider_sdk} import {SyncClient}
|
||||
import instructor
|
||||
from pydantic import BaseModel
|
||||
|
||||
# Initialize the client
|
||||
client = {SyncClient}()
|
||||
|
||||
# Enable instructor patches
|
||||
client = instructor.from_{provider}(client)
|
||||
|
||||
class User(BaseModel):
|
||||
name: str
|
||||
age: int
|
||||
|
||||
# Extract structured data
|
||||
user = client.chat.completions.create(
|
||||
model="your-model-name",
|
||||
messages=[{"role": "user", "content": "Extract: Jason is 25 years old"}],
|
||||
response_model=User
|
||||
)
|
||||
|
||||
print(user.name) # Jason
|
||||
print(user.age) # 25
|
||||
```
|
||||
|
||||
## Simple User Example (Async)
|
||||
|
||||
```python
|
||||
from {provider_sdk} import {AsyncClient}
|
||||
import instructor
|
||||
from pydantic import BaseModel
|
||||
import asyncio
|
||||
|
||||
# Initialize async client
|
||||
client = {AsyncClient}()
|
||||
|
||||
# Enable instructor patches
|
||||
client = instructor.from_{provider}(client)
|
||||
|
||||
class User(BaseModel):
|
||||
name: str
|
||||
age: int
|
||||
|
||||
async def extract_user():
|
||||
user = await client.chat.completions.create(
|
||||
model="your-model-name",
|
||||
messages=[{"role": "user", "content": "Extract: Jason is 25 years old"}],
|
||||
response_model=User
|
||||
)
|
||||
return user
|
||||
|
||||
# Run async function
|
||||
user = asyncio.run(extract_user())
|
||||
print(user.name) # Jason
|
||||
print(user.age) # 25
|
||||
```
|
||||
|
||||
## Supported Models
|
||||
|
||||
- `model-1` - Description and capabilities
|
||||
- `model-2` - Description and capabilities
|
||||
|
||||
Check [{Provider} documentation](provider-docs-url) for the complete list of available models.
|
||||
|
||||
## Modes
|
||||
|
||||
The {Provider} provider supports these modes:
|
||||
|
||||
- `instructor.Mode.{PROVIDER}_TOOLS` - Uses {provider} function calling (recommended)
|
||||
- `instructor.Mode.{PROVIDER}_JSON` - Uses JSON mode responses
|
||||
|
||||
```python
|
||||
client = instructor.from_{provider}(client, mode=instructor.Mode.{PROVIDER}_TOOLS)
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Validation and Retries
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel, field_validator
|
||||
|
||||
class User(BaseModel):
|
||||
name: str
|
||||
age: int
|
||||
|
||||
@field_validator('age')
|
||||
def validate_age(cls, v):
|
||||
if v < 0:
|
||||
raise ValueError('Age must be positive')
|
||||
return v
|
||||
|
||||
# Automatic retries on validation errors
|
||||
user = client.chat.completions.create(
|
||||
model="your-model-name",
|
||||
messages=[{"role": "user", "content": "Extract: Jason is -5 years old"}],
|
||||
response_model=User,
|
||||
max_retries=3
|
||||
)
|
||||
```
|
||||
|
||||
### Complex Nested Models
|
||||
|
||||
```python
|
||||
from typing import List
|
||||
|
||||
class Address(BaseModel):
|
||||
street: str
|
||||
city: str
|
||||
country: str
|
||||
|
||||
class User(BaseModel):
|
||||
name: str
|
||||
age: int
|
||||
addresses: List[Address]
|
||||
|
||||
users = client.chat.completions.create(
|
||||
model="your-model-name",
|
||||
messages=[{"role": "user", "content": "Extract user info with multiple addresses..."}],
|
||||
response_model=User
|
||||
)
|
||||
```
|
||||
|
||||
## Migration from Other Providers
|
||||
|
||||
If you're migrating from another provider:
|
||||
|
||||
```python
|
||||
# Old way (other provider)
|
||||
# client = instructor.from_openai(openai_client)
|
||||
|
||||
# New way ({Provider})
|
||||
client = instructor.from_{provider}({provider_sdk}.{SyncClient}())
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
For detailed API documentation, see the [Instructor API reference](../api/index.md).
|
||||
```
|
||||
|
||||
## Example Provider: Groq
|
||||
|
||||
Here's a concrete example implementing a Groq provider:
|
||||
|
||||
#### File: `instructor/providers/groq/client.py`
|
||||
```python
|
||||
from __future__ import annotations
|
||||
from typing import Any, overload
|
||||
import instructor
|
||||
from ...core.client import AsyncInstructor, Instructor
|
||||
from groq import Groq, AsyncGroq
|
||||
|
||||
@overload
|
||||
def from_groq(
|
||||
client: Groq,
|
||||
mode: instructor.Mode = instructor.Mode.GROQ_TOOLS,
|
||||
**kwargs: Any,
|
||||
) -> Instructor: ...
|
||||
|
||||
@overload
|
||||
def from_groq(
|
||||
client: AsyncGroq,
|
||||
mode: instructor.Mode = instructor.Mode.GROQ_TOOLS,
|
||||
**kwargs: Any,
|
||||
) -> AsyncInstructor: ...
|
||||
|
||||
def from_groq(
|
||||
client: Groq | AsyncGroq,
|
||||
mode: instructor.Mode = instructor.Mode.GROQ_TOOLS,
|
||||
**kwargs: Any,
|
||||
) -> Instructor | AsyncInstructor:
|
||||
valid_modes = {
|
||||
instructor.Mode.GROQ_TOOLS,
|
||||
instructor.Mode.GROQ_JSON,
|
||||
}
|
||||
|
||||
if mode not in valid_modes:
|
||||
from ...core.exceptions import ModeError
|
||||
raise ModeError(
|
||||
mode=str(mode),
|
||||
provider="Groq",
|
||||
valid_modes=[str(m) for m in valid_modes],
|
||||
)
|
||||
|
||||
if not isinstance(client, (AsyncGroq, Groq)):
|
||||
from ...core.exceptions import ClientError
|
||||
raise ClientError(
|
||||
f"Client must be an instance of Groq or AsyncGroq. "
|
||||
f"Got: {type(client).__name__}"
|
||||
)
|
||||
|
||||
if isinstance(client, AsyncGroq):
|
||||
async def async_wrapper(*args: Any, **kwargs: Any):
|
||||
return await client.chat.completions.acreate(*args, **kwargs)
|
||||
|
||||
return AsyncInstructor(
|
||||
client=client,
|
||||
create=instructor.patch(create=async_wrapper, mode=mode),
|
||||
provider=instructor.Provider.GROQ,
|
||||
mode=mode,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
return Instructor(
|
||||
client=client,
|
||||
create=instructor.patch(create=client.chat.completions.create, mode=mode),
|
||||
provider=instructor.Provider.GROQ,
|
||||
mode=mode,
|
||||
**kwargs,
|
||||
)
|
||||
```
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
Before submitting your provider implementation, verify:
|
||||
|
||||
### Core Implementation
|
||||
- [ ] `from_{provider}` function implemented following the exact pattern
|
||||
- [ ] Both sync and async clients supported with proper overloads
|
||||
- [ ] Valid modes defined and enforced with proper error messages
|
||||
- [ ] Client type validation with helpful error messages
|
||||
- [ ] Proper use of `instructor.patch()` for both sync and async
|
||||
|
||||
### Testing
|
||||
- [ ] `conftest.py` skips tests if API key missing or package not installed
|
||||
- [ ] `util.py` defines supported models and modes
|
||||
- [ ] `test_simple.py` covers basic sync/async functionality with validation
|
||||
- [ ] Tests use parametrized approach with `product(models, modes)`
|
||||
- [ ] All tests pass with real API key: `pytest tests/llm/test_{provider}/`
|
||||
|
||||
### Infrastructure Updates
|
||||
- [ ] Modes added to `instructor/mode.py`
|
||||
- [ ] Provider added to `instructor/utils/providers.py` Provider enum
|
||||
- [ ] Conditional import added to `instructor/__init__.py`
|
||||
- [ ] Dependencies added to `pyproject.toml` optional-dependencies
|
||||
- [ ] Dependencies added to `pyproject.toml` dependency-groups
|
||||
|
||||
### Documentation
|
||||
- [ ] Provider documentation created in `docs/integrations/{provider}.md`
|
||||
- [ ] Follows exact pattern with frontmatter, examples, and sections
|
||||
- [ ] All code examples are tested and work
|
||||
- [ ] Covers sync/async usage, validation, nested models
|
||||
- [ ] Links to provider documentation and API reference
|
||||
|
||||
### Integration
|
||||
- [ ] Works with existing instructor patterns and conventions
|
||||
- [ ] Error messages are helpful and actionable
|
||||
- [ ] Follows the same API as other providers
|
||||
- [ ] No performance regressions
|
||||
|
||||
## Submission Process
|
||||
|
||||
1. **Test Locally**: Ensure all tests pass and examples work
|
||||
2. **Create PR**: Submit to instructor repository
|
||||
3. **Package Registry**: Publish to PyPI as `instructor-{provider}`
|
||||
4. **Documentation**: Add to instructor docs site
|
||||
5. **Announcement**: Share with community
|
||||
|
||||
## Common Issues & Solutions
|
||||
|
||||
### "Provider not found" error
|
||||
- Check entry point configuration in pyproject.toml
|
||||
- Verify provider name matches exactly
|
||||
- Ensure package is installed in same environment
|
||||
|
||||
### Validation errors not retrying
|
||||
- Verify error handling in chat() method catches ValidationError
|
||||
- Check that validation messages are added to conversation
|
||||
- Ensure max_retries parameter is respected
|
||||
|
||||
### Mode not supported
|
||||
- Implement handler in handlers.py for the mode
|
||||
- Add to _HANDLERS registry
|
||||
- Test with provider's actual API capabilities
|
||||
|
||||
### Streaming issues
|
||||
- Check if provider supports streaming at all
|
||||
- Implement incremental parsing for partial responses
|
||||
- Handle stream interruption and reconnection
|
||||
|
||||
### Type checking failures
|
||||
- Ensure all method signatures match BaseProvider protocol exactly
|
||||
- Add proper type hints for all parameters and returns
|
||||
- Use Union/Optional types where appropriate
|
||||
|
||||
---
|
||||
|
||||
**This completes the full provider implementation guide. Follow these instructions systematically and you'll have a production-ready instructor provider that integrates seamlessly with the existing ecosystem.**
|
||||
Reference in New Issue
Block a user