참고소스 수정본

This commit is contained in:
LASTA_DEV01\lasta
2026-05-12 19:40:31 +09:00
parent 0f34a451fc
commit 2e9204243d
8708 changed files with 3259488 additions and 869 deletions

View File

@@ -0,0 +1,81 @@
# Actions
## ReAsk
```python
class ReAsk(IReask)
```
Base class for ReAsk objects.
**Attributes**:
- `incorrect_value` _Any_ - The value that failed validation.
- `fail_results` _List[FailResult]_ - The results of the failed validations.
## FieldReAsk
```python
class FieldReAsk(ReAsk)
```
An implementation of ReAsk that is used to reask for a specific field.
Inherits from ReAsk.
**Attributes**:
- `path` _Optional[List[Any]]_ - a list of keys that
designated the path to the field that failed validation.
## SkeletonReAsk
```python
class SkeletonReAsk(ReAsk)
```
An implementation of ReAsk that is used to reask for structured data
when the response does not match the expected schema.
Inherits from ReAsk.
## NonParseableReAsk
```python
class NonParseableReAsk(ReAsk)
```
An implementation of ReAsk that is used to reask for structured data
when the response is not parseable as JSON.
Inherits from ReAsk.
## Filter
```python
class Filter()
```
#### apply\_filters
```python
def apply_filters(value: Any) -> Any
```
Recursively filter out any values that are instances of Filter.
## Refrain
```python
class Refrain()
```
#### apply\_refrain
```python
def apply_refrain(value: Any, output_type: OutputTypes) -> Any
```
Recursively check for any values that are instances of Refrain.
If found, return an empty value of the appropriate type.

View File

@@ -0,0 +1,15 @@
# Errors
## ValidationError
```python
class ValidationError(Exception)
```
Top level validation error.
This is thrown from the validation engine when a Validator has
on_fail=OnFailActions.EXCEPTION set and validation fails.
Inherits from Exception.

View File

@@ -0,0 +1,23 @@
# Formatters
## BaseFormatter
```python
class BaseFormatter(ABC)
```
A Formatter takes an LLM Callable and wraps the method into an abstract
callable.
Used to perform manipulations of the input or the output, like JSON
constrained- decoding.
## JsonFormatter
```python
class JsonFormatter(BaseFormatter)
```
A formatter that uses Jsonformer to ensure the shape of structured data
for Hugging Face models.

View File

@@ -0,0 +1,139 @@
# Generics And Base Classes
## ArbitraryModel
```python
class ArbitraryModel(BaseModel)
```
Empty Pydantic model with a config that allows arbitrary types.
## Stack
```python
class Stack(List[T])
```
#### empty
```python
def empty() -> bool
```
Tests if this stack is empty.
#### peek
```python
def peek() -> Optional[T]
```
Looks at the object at the top (last/most recently added) of this
stack without removing it from the stack.
#### pop
```python
def pop() -> Optional[T]
```
Removes the object at the top of this stack and returns that object
as the value of this function.
#### push
```python
def push(item: T) -> None
```
Pushes an item onto the top of this stack.
Proxy of List.append
Limits Stack Length to _max_length entries
#### search
```python
def search(x: T) -> Optional[int]
```
Returns the 0-based position of the last item whose value is equal
to x on this stack.
We deviate from the typical 1-based position used by Stack
classes (i.e. Java) because most python users (and developers in
general) are accustomed to 0-based indexing.
#### at
```python
def at(index: int, default: Optional[T] = None) -> Optional[T]
```
Returns the item located at the index.
If the index does not exist in the stack (Overflow or
Underflow), None is returned instead.
#### copy
```python
def copy() -> "Stack[T]"
```
Returns a copy of the current Stack.
#### first
```python
@property
def first() -> Optional[T]
```
Returns the first item of the stack without removing it.
Same as Stack.bottom.
#### last
```python
@property
def last() -> Optional[T]
```
Returns the last item of the stack without removing it.
Same as Stack.top.
#### bottom
```python
@property
def bottom() -> Optional[T]
```
Returns the item on the bottom of the stack without removing it.
Same as Stack.first.
#### top
```python
@property
def top() -> Optional[T]
```
Returns the item on the top of the stack without removing it.
Same as Stack.last.
#### length
```python
@property
def length() -> int
```
Returns the number of items in the Stack.

View File

@@ -0,0 +1,500 @@
# Guards
## Guard
```python
class Guard(IGuard, Generic[OT])
```
The Guard class.
This class is the main entry point for using Guardrails. It can be
initialized by one of the following patterns:
- `Guard().use(...)`
- `Guard.for_string(...)`
- `Guard.for_pydantic(...)`
- `Guard.for_rail(...)`
- `Guard.for_rail_string(...)`
The `__call__`
method functions as a wrapper around LLM APIs. It takes in an LLM
API, and optional prompt parameters, and returns a ValidationOutcome
class that contains the raw output from
the LLM, the validated output, as well as other helpful information.
#### \_\_init\_\_
```python
def __init__(*,
id: Optional[str] = None,
name: Optional[str] = None,
description: Optional[str] = None,
validators: Optional[List[ValidatorReference]] = None,
output_schema: Optional[Dict[str, Any]] = None,
base_url: Optional[str] = None,
api_key: Optional[str] = None,
history_max_length: Optional[int] = None,
use_server: Optional[bool] = None)
```
Initialize the Guard with serialized validator references and an
output schema.
Output schema must be a valid JSON Schema.
#### configure
```python
def configure(*,
num_reasks: Optional[int] = None,
allow_metrics_collection: Optional[bool] = None)
```
Configure the Guard.
**Arguments**:
- `num_reasks` _int, optional_ - The max times to re-ask the LLM
if validation fails. Defaults to None.
- `allow_metrics_collection` _bool, optional_ - Whether to allow
Guardrails to collect anonymous metrics.
Defaults to None, and falls back to waht is
set via the `guardrails configure` command.
#### for\_rail
```python
@classmethod
def for_rail(cls,
rail_file: str,
*,
name: Optional[str] = None,
description: Optional[str] = None)
```
Create a Guard using a `.rail` file to specify the output schema,
prompt, etc.
**Arguments**:
- `rail_file` - The path to the `.rail` file.
- `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id.
- `description` _str, optional_ - A description for this Guard. Defaults to None.
**Returns**:
An instance of the `Guard` class.
#### for\_rail\_string
```python
@classmethod
def for_rail_string(cls,
rail_string: str,
*,
name: Optional[str] = None,
description: Optional[str] = None)
```
Create a Guard using a `.rail` string to specify the output schema,
prompt, etc..
**Arguments**:
- `rail_string` - The `.rail` string.
- `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id.
- `description` _str, optional_ - A description for this Guard. Defaults to None.
**Returns**:
An instance of the `Guard` class.
#### for\_pydantic
```python
@classmethod
def for_pydantic(cls,
output_class: ModelOrListOfModels,
*,
reask_messages: Optional[List[Dict]] = None,
messages: Optional[List[Dict]] = None,
name: Optional[str] = None,
description: Optional[str] = None,
output_formatter: Optional[Union[str, BaseFormatter]] = None)
```
Create a Guard instance using a Pydantic model to specify the output
schema.
**Arguments**:
- `output_class` - (Union[Type[BaseModel], List[Type[BaseModel]]]): The pydantic model that describes
the desired structure of the output.
- `messages` _List[Dict], optional_ - A list of messages to give to the llm. Defaults to None.
- `reask_messages` _List[Dict], optional_ - A list of messages to use during reasks. Defaults to None.
- `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id.
- `description` _str, optional_ - A description for this Guard. Defaults to None.
- `output_formatter` _str | Formatter, optional_ - 'none' (default), 'jsonformer', or a Guardrails Formatter.
#### for\_string
```python
@classmethod
def for_string(cls,
validators: Sequence[Validator],
*,
string_description: Optional[str] = None,
reask_messages: Optional[List[Dict]] = None,
messages: Optional[List[Dict]] = None,
name: Optional[str] = None,
description: Optional[str] = None)
```
Create a Guard instance for a string response.
**Arguments**:
- `validators` - (List[Validator]): The list of validators to apply to the string output.
- `string_description` _str, optional_ - A description for the string to be generated. Defaults to None.
- `messages` _List[Dict], optional_ - A list of messages to pass to llm. Defaults to None.
- `reask_messages` _List[Dict], optional_ - A list of messages to use during reasks. Defaults to None.
- `name` _str, optional_ - A unique name for this Guard. Defaults to `gr-` + the object id.
- `description` _str, optional_ - A description for this Guard. Defaults to None.
#### \_\_call\_\_
```python
@trace(name="/guard_call", origin="Guard.__call__")
def __call__(
llm_api: Optional[Callable] = None,
*args,
prompt_params: Optional[Dict] = None,
num_reasks: Optional[int] = 1,
messages: Optional[List[Dict]] = None,
metadata: Optional[Dict] = None,
full_schema_reask: Optional[bool] = None,
**kwargs
) -> Union[ValidationOutcome[OT], Iterator[ValidationOutcome[OT]]]
```
Call the LLM and validate the output.
**Arguments**:
- `llm_api` - The LLM API to call
(e.g. openai.completions.create or openai.Completion.acreate)
- `prompt_params` - The parameters to pass to the prompt.format() method.
- `num_reasks` - The max times to re-ask the LLM for invalid output.
- `messages` - The message history to pass to the LLM.
- `metadata` - Metadata to pass to the validators.
- `full_schema_reask` - When reasking, whether to regenerate the full schema
or just the incorrect values.
Defaults to `True` if a base model is provided,
`False` otherwise.
**Returns**:
ValidationOutcome
#### parse
```python
@trace(name="/guard_call", origin="Guard.parse")
def parse(llm_output: str,
*args,
metadata: Optional[Dict] = None,
llm_api: Optional[Callable] = None,
num_reasks: Optional[int] = None,
prompt_params: Optional[Dict] = None,
full_schema_reask: Optional[bool] = None,
**kwargs) -> ValidationOutcome[OT]
```
Alternate flow to using Guard where the llm_output is known.
**Arguments**:
- `llm_output` - The output being parsed and validated.
- `metadata` - Metadata to pass to the validators.
- `llm_api` - The LLM API to call
(e.g. openai.completions.create or openai.Completion.acreate)
- `num_reasks` - The max times to re-ask the LLM for invalid output.
- `prompt_params` - The parameters to pass to the prompt.format() method.
- `full_schema_reask` - When reasking, whether to regenerate the full schema
or just the incorrect values.
**Returns**:
ValidationOutcome
#### error\_spans\_in\_output
```python
def error_spans_in_output() -> List[ErrorSpan]
```
Get the error spans in the last output.
#### use
```python
def use(*validator_spread: Validator,
validators: List[Validator] = [],
on: str = "output") -> "Guard"
```
Applies validators to the property specified in the `on` argument.
Calling `Guard.use` with the same `on` value multiple times will
overwrite previously configured validators on the specified property.
**Arguments**:
*validator_spread:
One or more validators passed as positional arguments to use.
validators:
Keyword argument that allows explicitly setting a list of
validators to use.
on:
The property to validate. Valid options include "output", "messages",
or a JSON path starting with "$.". Defaults to "output".
#### get\_validators
```python
def get_validators(on: str) -> List[Validator]
```
The read-only counterpart to `Guard.use`. Retrieves the validators
applied to the specified property.
**Arguments**:
- `on` - The property for which to return configured validators.
Valid options include "output", "messages",
or a JSON path starting with "$.".
#### validate
```python
@trace(name="/guard_call", origin="Guard.validate")
def validate(llm_output: str, *args, **kwargs) -> ValidationOutcome[OT]
```
#### to\_runnable
```python
def to_runnable() -> Runnable
```
Convert a Guard to a LangChain Runnable.
#### to\_dict
```python
def to_dict() -> Dict[str, Any]
```
#### json\_function\_calling\_tool
```python
def json_function_calling_tool(
tools: Optional[list] = None) -> List[Dict[str, Any]]
```
Appends an OpenAI tool that specifies the output structure using
JSON Schema for chat models.
#### from\_dict
```python
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional["Guard"]
```
## AsyncGuard
```python
class AsyncGuard(Guard, Generic[OT])
```
The AsyncGuard class.
This class one of the main entry point for using Guardrails. It is
initialized from one of the following class methods:
- `for_rail`
- `for_rail_string`
- `for_pydantic`
- `for_string`
The `__call__`
method functions as a wrapper around LLM APIs. It takes in an Async LLM
API, and optional prompt parameters, and returns the raw output stream from
the LLM and the validated output stream.
#### \_\_init\_\_
```python
def __init__(*args, **kwargs)
```
#### for\_pydantic
```python
@classmethod
def for_pydantic(cls,
output_class: ModelOrListOfModels,
*,
messages: Optional[List[Dict]] = None,
reask_messages: Optional[List[Dict]] = None,
name: Optional[str] = None,
description: Optional[str] = None,
output_formatter: Optional[Union[str, BaseFormatter]] = None)
```
#### for\_string
```python
@classmethod
def for_string(cls,
validators: Sequence[Validator],
*,
string_description: Optional[str] = None,
messages: Optional[List[Dict]] = None,
reask_messages: Optional[List[Dict]] = None,
name: Optional[str] = None,
description: Optional[str] = None)
```
#### from\_dict
```python
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional["AsyncGuard"]
```
#### use
```python
def use(*validator_spread: Validator,
validators: List[Validator] = [],
on: str = "output") -> "AsyncGuard"
```
#### \_\_call\_\_
```python
@async_trace(name="/guard_call", origin="AsyncGuard.__call__")
async def __call__(
llm_api: Optional[Callable[..., Awaitable[Any]]] = None,
*args,
prompt_params: Optional[Dict] = None,
num_reasks: Optional[int] = 1,
messages: Optional[List[Dict]] = None,
metadata: Optional[Dict] = None,
full_schema_reask: Optional[bool] = None,
**kwargs
) -> Union[
ValidationOutcome[OT],
Awaitable[ValidationOutcome[OT]],
AsyncIterator[ValidationOutcome[OT]],
]
```
Call the LLM and validate the output. Pass an async LLM API to
return a coroutine.
**Arguments**:
- `llm_api` - The LLM API to call
(e.g. openai.completions.create or openai.chat.completions.create)
- `prompt_params` - The parameters to pass to the prompt.format() method.
- `num_reasks` - The max times to re-ask the LLM for invalid output.
- `messages` - The message history to pass to the LLM.
- `metadata` - Metadata to pass to the validators.
- `full_schema_reask` - When reasking, whether to regenerate the full schema
or just the incorrect values.
Defaults to `True` if a base model is provided,
`False` otherwise.
**Returns**:
The raw text output from the LLM and the validated output.
#### parse
```python
@async_trace(name="/guard_call", origin="AsyncGuard.parse")
async def parse(llm_output: str,
*args,
metadata: Optional[Dict] = None,
llm_api: Optional[Callable[..., Awaitable[Any]]] = None,
num_reasks: Optional[int] = None,
prompt_params: Optional[Dict] = None,
full_schema_reask: Optional[bool] = None,
**kwargs) -> Awaitable[ValidationOutcome[OT]]
```
Alternate flow to using AsyncGuard where the llm_output is known.
**Arguments**:
- `llm_output` - The output being parsed and validated.
- `metadata` - Metadata to pass to the validators.
- `llm_api` - The LLM API to call
(e.g. openai.completions.create or openai.Completion.acreate)
- `num_reasks` - The max times to re-ask the LLM for invalid output.
- `prompt_params` - The parameters to pass to the prompt.format() method.
- `full_schema_reask` - When reasking, whether to regenerate the full schema
or just the incorrect values.
**Returns**:
The validated response. This is either a string or a dictionary,
determined by the object schema defined in the RAILspec.
#### validate
```python
@async_trace(name="/guard_call", origin="AsyncGuard.validate")
async def validate(llm_output: str, *args,
**kwargs) -> Awaitable[ValidationOutcome[OT]]
```
## ValidationOutcome
```python
class ValidationOutcome(IValidationOutcome, ArbitraryModel, Generic[OT])
```
The final output from a Guard execution.
**Attributes**:
- `call_id` - The id of the Call that produced this ValidationOutcome.
- `raw_llm_output` - The raw, unchanged output from the LLM call.
- `validated_output` - The validated, and potentially fixed, output from the LLM call
after passing through validation.
- `reask` - If validation continuously fails and all allocated reasks are used,
this field will contain the final reask that would have been sent
to the LLM if additional reasks were available.
- `validation_passed` - A boolean to indicate whether or not the LLM output
passed validation. If this is False, the validated_output may be invalid.
- `error` - If the validation failed, this field will contain the error message
#### from\_guard\_history
```python
@classmethod
def from_guard_history(cls, call: Call)
```
Create a ValidationOutcome from a history Call object.

View File

@@ -0,0 +1,507 @@
# History and Logs
## Call
```python
class Call(ICall, ArbitraryModel)
```
A Call represents a single execution of a Guard. One Call is created
each time the user invokes the `Guard.__call__`, `Guard.parse`, or
`Guard.validate` method.
**Attributes**:
- `iterations` _Stack[Iteration]_ - A stack of iterations
for the initial validation round
and one for each reask that occurs during a Call.
- `inputs` _CallInputs_ - The inputs as passed in to
`Guard.__call__`, `Guard.parse`, or `Guard.validate`
- `exception` _Optional[Exception]_ - The exception that interrupted
the Guard execution.
#### prompt\_params
```python
@property
def prompt_params() -> Optional[Dict]
```
The prompt parameters as provided by the user when initializing or
calling the Guard.
#### messages
```python
@property
def messages() -> Optional[Union[Messages, list[dict[str, str]]]]
```
The messages as provided by the user when initializing or calling
the Guard.
#### compiled\_messages
```python
@property
def compiled_messages() -> Optional[list[dict[str, str]]]
```
The initial compiled messages that were passed to the LLM on the
first call.
#### reask\_messages
```python
@property
def reask_messages() -> Stack[Messages]
```
The compiled messages used during reasks.
Does not include the initial messages.
#### logs
```python
@property
def logs() -> Stack[str]
```
Returns all logs from all iterations as a stack.
#### tokens\_consumed
```python
@property
def tokens_consumed() -> Optional[int]
```
Returns the total number of tokens consumed during all iterations
with this call.
#### prompt\_tokens\_consumed
```python
@property
def prompt_tokens_consumed() -> Optional[int]
```
Returns the total number of prompt tokens consumed during all
iterations with this call.
#### completion\_tokens\_consumed
```python
@property
def completion_tokens_consumed() -> Optional[int]
```
Returns the total number of completion tokens consumed during all
iterations with this call.
#### raw\_outputs
```python
@property
def raw_outputs() -> Stack[str]
```
The exact outputs from all LLM calls.
#### parsed\_outputs
```python
@property
def parsed_outputs() -> Stack[Union[str, List, Dict]]
```
The outputs from the LLM after undergoing parsing but before
validation.
#### validation\_response
```python
@property
def validation_response() -> Optional[Union[str, List, Dict, ReAsk]]
```
The aggregated responses from the validation process across all
iterations within the current call.
This value could contain ReAsks.
#### fixed\_output
```python
@property
def fixed_output() -> Optional[Union[str, List, Dict]]
```
The cumulative output from the validation process across all current
iterations with any automatic fixes applied.
Could still contain ReAsks if a fix was not available.
#### guarded\_output
```python
@property
def guarded_output() -> Optional[Union[str, List, Dict]]
```
The complete validated output after all stages of validation are
completed.
This property contains the aggregate validated output after all
validation stages have been completed. Some values in the
validated output may be "fixed" values that were corrected
during validation.
This will only have a value if the Guard is in a passing state
OR if the action is no-op.
#### reasks
```python
@property
def reasks() -> Stack[ReAsk]
```
Reasks generated during validation that could not be automatically
fixed.
These would be incorporated into the prompt for the next LLM
call if additional reasks were granted.
#### validator\_logs
```python
@property
def validator_logs() -> Stack[ValidatorLogs]
```
The results of each individual validation performed on the LLM
responses during all iterations.
#### error
```python
@property
def error() -> Optional[str]
```
The error message from any exception that raised and interrupted the
run.
#### failed\_validations
```python
@property
def failed_validations() -> Stack[ValidatorLogs]
```
The validator logs for any validations that failed during the
entirety of the run.
#### status
```python
@property
def status() -> str
```
Returns the cumulative status of the run based on the validity of
the final merged output.
#### tree
```python
@property
def tree() -> Tree
```
Returns the tree.
## Iteration
```python
class Iteration(IIteration, ArbitraryModel)
```
An Iteration represents a single iteration of the validation loop
including a single call to the LLM if applicable.
**Attributes**:
- `id` _str_ - The unique identifier for the iteration.
- `call_id` _str_ - The unique identifier for the Call
that this iteration is a part of.
- `index` _int_ - The index of this iteration within the Call.
- `inputs` _Inputs_ - The inputs for the validation loop.
- `outputs` _Outputs_ - The outputs from the validation loop.
#### logs
```python
@property
def logs() -> Stack[str]
```
Returns the logs from this iteration as a stack.
#### tokens\_consumed
```python
@property
def tokens_consumed() -> Optional[int]
```
Returns the total number of tokens consumed during this
iteration.
#### prompt\_tokens\_consumed
```python
@property
def prompt_tokens_consumed() -> Optional[int]
```
Returns the number of prompt/input tokens consumed during this
iteration.
#### completion\_tokens\_consumed
```python
@property
def completion_tokens_consumed() -> Optional[int]
```
Returns the number of completion/output tokens consumed during this
iteration.
#### raw\_output
```python
@property
def raw_output() -> Optional[str]
```
The exact output from the LLM.
#### parsed\_output
```python
@property
def parsed_output() -> Optional[Union[str, List, Dict]]
```
The output from the LLM after undergoing parsing but before
validation.
#### validation\_response
```python
@property
def validation_response() -> Optional[Union[ReAsk, str, List, Dict]]
```
The response from a single stage of validation.
Validation response is the output of a single stage of validation
and could be a combination of valid output and reasks.
Note that a Guard may run validation multiple times if reasks occur.
To access the final output after all steps of validation are completed,
check out `Call.guarded_output`."
#### guarded\_output
```python
@property
def guarded_output() -> Optional[Union[str, List, Dict]]
```
Any valid values after undergoing validation.
Some values in the validated output may be "fixed" values that
were corrected during validation. This property may be a partial
structure if field level reasks occur.
#### reasks
```python
@property
def reasks() -> Sequence[ReAsk]
```
Reasks generated during validation.
These would be incorporated into the prompt or the next LLM
call.
#### validator\_logs
```python
@property
def validator_logs() -> List[ValidatorLogs]
```
The results of each individual validation performed on the LLM
response during this iteration.
#### error
```python
@property
def error() -> Optional[str]
```
The error message from any exception that raised and interrupted
this iteration.
#### exception
```python
@property
def exception() -> Optional[Exception]
```
The exception that interrupted this iteration.
#### failed\_validations
```python
@property
def failed_validations() -> List[ValidatorLogs]
```
The validator logs for any validations that failed during this
iteration.
#### error\_spans\_in\_output
```python
@property
def error_spans_in_output() -> List[ErrorSpan]
```
The error spans from the LLM response.
These indices are relative to the complete LLM output.
#### status
```python
@property
def status() -> str
```
Representation of the end state of this iteration.
OneOf: pass, fail, error, not run
## Inputs
```python
class Inputs(IInputs, ArbitraryModel)
```
Inputs represent the input data that is passed into the validation loop.
**Attributes**:
- `llm_api` _Optional[PromptCallableBase]_ - The constructed class
for calling the LLM.
- `llm_output` _Optional[str]_ - The string output from an
external LLM call provided by the user via Guard.parse.
- `messages` _Optional[List[Dict]]_ - The message history
provided by the user for chat model calls.
- `prompt_params` _Optional[Dict]_ - The parameters provided
by the user that will be formatted into the final LLM prompt.
- `num_reasks` _Optional[int]_ - The total number of reasks allowed;
user provided or defaulted.
- `metadata` _Optional[Dict[str, Any]]_ - The metadata provided
by the user to be used during validation.
- `full_schema_reask` _Optional[bool]_ - Whether reasks we
performed across the entire schema or at the field level.
- `stream` _Optional[bool]_ - Whether or not streaming was used.
## Outputs
```python
class Outputs(IOutputs, ArbitraryModel)
```
Outputs represent the data that is output from the validation loop.
**Attributes**:
- `llm_response_info` _Optional[LLMResponse]_ - Information from the LLM response
- `raw_output` _Optional[str]_ - The exact output from the LLM.
- `parsed_output` _Optional[Union[str, List, Dict]]_ - The output parsed from the LLM
response as it was passed into validation.
- `validation_response` _Optional[Union[str, ReAsk, List, Dict]]_ - The response
from the validation process.
- `guarded_output` _Optional[Union[str, List, Dict]]_ - Any valid values after
undergoing validation.
Some values may be "fixed" values that were corrected during validation.
This property may be a partial structure if field level reasks occur.
- `reasks` _List[ReAsk]_ - Information from the validation process used to construct
a ReAsk to the LLM on validation failure. Default [].
- `validator_logs` _List[ValidatorLogs]_ - The results of each individual
validation. Default [].
- `error` _Optional[str]_ - The error message from any exception that raised
and interrupted the process.
- `exception` _Optional[Exception]_ - The exception that interrupted the process.
#### failed\_validations
```python
@property
def failed_validations() -> List[ValidatorLogs]
```
Returns the validator logs for any validation that failed.
#### error\_spans\_in\_output
```python
@property
def error_spans_in_output() -> List[ErrorSpan]
```
The error spans from the LLM response.
These indices are relative to the complete LLM output.
#### status
```python
@property
def status() -> str
```
Representation of the end state of the validation run.
OneOf: pass, fail, error, not run
## CallInputs
```python
class CallInputs(Inputs, ICallInputs, ArbitraryModel)
```
CallInputs represent the input data that is passed into the Guard from
the user. Inherits from Inputs with the below overrides and additional
attributes.
**Attributes**:
- `llm_api` _Optional[Callable[[Any], Awaitable[Any]]]_ - The LLM function
provided by the user during Guard.__call__ or Guard.parse.
- `messages` _Optional[dict[str, str]]_ - The messages as provided by the user.
- `args` _List[Any]_ - Additional arguments for the LLM as provided by the user.
Default [].
- `kwargs` _Dict[str, Any]_ - Additional keyword-arguments for
the LLM as provided by the user. Default {}.

View File

@@ -0,0 +1,116 @@
# Helpers for LLM Interactions
Class for representing a prompt entry.
## BasePrompt
```python
class BasePrompt()
```
Base class for representing an LLM prompt.
#### \_\_init\_\_
```python
def __init__(source: str,
output_schema: Optional[str] = None,
*,
xml_output_schema: Optional[str] = None)
```
Initialize and substitute constants in the prompt.
#### substitute\_constants
```python
def substitute_constants(text: str) -> str
```
Substitute constants in the prompt.
#### get\_prompt\_variables
```python
def get_prompt_variables() -> List[str]
```
#### format
```python
def format(**kwargs) -> "BasePrompt"
```
#### escape
```python
def escape() -> str
```
Escape single curly braces into double curly braces.
The LLM prompt.
## Prompt
```python
class Prompt(BasePrompt)
```
Prompt class.
The prompt is passed to the LLM as primary instructions.
#### format
```python
def format(**kwargs) -> "Prompt"
```
Format the prompt using the given keyword arguments.
Instructions to the LLM, to be passed in the prompt.
## Instructions
```python
class Instructions(BasePrompt)
```
Instructions class.
The instructions are passed to the LLM as secondary input. Different
model may use these differently. For example, chat models may
receive instructions in the system-prompt.
#### format
```python
def format(**kwargs) -> "Instructions"
```
Format the prompt using the given keyword arguments.
## PromptCallableBase
## LLMResponse
```python
class LLMResponse(ILLMResponse)
```
Standard information collection from LLM responses to feed the
validation loop.
**Attributes**:
- `output` _str_ - The output from the LLM.
- `stream_output` _Optional[Iterator]_ - A stream of output from the LLM.
Default None.
- `async_stream_output` _Optional[AsyncIterator]_ - An async stream of output
from the LLM. Default None.
- `prompt_token_count` _Optional[int]_ - The number of tokens in the prompt.
Default None.
- `response_token_count` _Optional[int]_ - The number of tokens in the response.
Default None.

View File

@@ -0,0 +1,115 @@
# Types
## OnFailAction
```python
class OnFailAction(str, Enum)
```
OnFailAction is an Enum that represents the different actions that can
be taken when a validation fails.
**Attributes**:
- `REASK` _Literal["reask"]_ - On failure, Reask the LLM.
- `FIX` _Literal["fix"]_ - On failure, apply a static fix.
- `FILTER` _Literal["filter"]_ - On failure, filter out the invalid values.
- `REFRAIN` _Literal["refrain"]_ - On failure, refrain from responding;
return an empty value.
- `NOOP` _Literal["noop"]_ - On failure, do nothing.
- `EXCEPTION` _Literal["exception"]_ - On failure, raise a ValidationError.
- `FIX_REASK` _Literal["fix_reask"]_ - On failure, apply a static fix,
check if the fixed value passed validation, if not then reask the LLM.
- `CUSTOM` _Literal["custom"]_ - On failure, call a custom function with the
invalid value and the FailResult's from any validators run on the value.
## RailTypes
```python
class RailTypes(str, Enum)
```
RailTypes is an Enum that represents the builtin tags for RAIL xml.
**Attributes**:
- `STRING` _Literal["string"]_ - A string value.
- `INTEGER` _Literal["integer"]_ - An integer value.
- `FLOAT` _Literal["float"]_ - A float value.
- `BOOL` _Literal["bool"]_ - A boolean value.
- `DATE` _Literal["date"]_ - A date value.
- `TIME` _Literal["time"]_ - A time value.
DATETIME (Literal["date-time: - A datetime value.
- `PERCENTAGE` _Literal["percentage"]_ - A percentage value represented as a string.
Example "20.5%".
- `ENUM` _Literal["enum"]_ - An enum value.
- `LIST` _Literal["list"]_ - A list/array value.
- `OBJECT` _Literal["object"]_ - An object/dictionary value.
- `CHOICE` _Literal["choice"]_ - The options for a discrimated union.
- `CASE` _Literal["case"]_ - A dictionary that contains a discrimated union.
## MessageHistory
```python
MessageHistory = List[Dict[str, Union[Prompt, str]]]
```
## ModelOrListOfModels
```python
ModelOrListOfModels = Union[Type[BaseModel], Type[List[Type[BaseModel]]]]
```
## ModelOrListOrDict
```python
ModelOrListOrDict = Union[Type[BaseModel], Type[List[Type[BaseModel]]],
Type[Dict[str, Type[BaseModel]]]]
```
## ModelOrModelUnion
```python
ModelOrModelUnion = Union[Type[BaseModel], Union[Type[BaseModel], Any]]
```
## PydanticValidatorTuple
```python
PydanticValidatorTuple = Tuple[Union[Validator, str, Callable], str]
```
## PydanticValidatorSpec
```python
PydanticValidatorSpec = Union[Validator, PydanticValidatorTuple]
```
## UseValidatorSpec
```python
UseValidatorSpec = Union[Validator, Type[Validator]]
```
## UseManyValidatorTuple
```python
UseManyValidatorTuple = Tuple[
Type[Validator],
Optional[Union[List[Any], Dict[str, Any]]],
Optional[Dict[str, Any]],
]
```
## UseManyValidatorSpec
```python
UseManyValidatorSpec = Union[Validator, UseManyValidatorTuple]
```
## ValidatorMap
```python
ValidatorMap = Dict[str, List[Validator]]
```

View File

@@ -0,0 +1,191 @@
# Validation
## Validator
```python
@dataclass
class Validator()
```
Base class for validators.
#### \_\_init\_\_
```python
def __init__(on_fail: Optional[Union[Callable[[Any, FailResult], Any],
OnFailAction]] = None,
**kwargs)
```
#### validate
```python
def validate(value: Any, metadata: Dict[str, Any]) -> ValidationResult
```
Do not override this function, instead implement _validate().
External facing validate function. This function acts as a
wrapper for _validate() and is intended to apply any meta-
validation requirements, logic, or pre/post processing.
#### validate\_stream
```python
def validate_stream(chunk: Any,
metadata: Dict[str, Any],
*,
property_path: Optional[str] = "$",
context_vars: Optional[ContextVar[Dict[
str, ContextVar[List[str]]]]] = None,
context: Optional[Context] = None,
**kwargs) -> Optional[ValidationResult]
```
Validates a chunk emitted by an LLM. If the LLM chunk is smaller
than the validator's chunking strategy, it will be accumulated until it
reaches the desired size. In the meantime, the validator will return
None.
If the LLM chunk is larger than the validator's chunking
strategy, it will split it into validator-sized chunks and
validate each one, returning an array of validation results.
Otherwise, the validator will validate the chunk and return the
result.
#### with\_metadata
```python
def with_metadata(metadata: Dict[str, Any])
```
Assigns metadata to this validator to use during validation.
#### to\_runnable
```python
def to_runnable() -> Runnable
```
#### register\_validator
```python
def register_validator(
name: str,
data_type: Union[str, List[str]],
has_guardrails_endpoint: bool = False
) -> Callable[[Union[Type[V], Callable]], Union[Type[V], Type[Validator]]]
```
Register a validator for a data type.
## ValidationResult
```python
class ValidationResult(IValidationResult, ArbitraryModel)
```
ValidationResult is the output type of Validator.validate and the
abstract base class for all validation results.
**Attributes**:
- `outcome` _str_ - The outcome of the validation. Must be one of "pass" or "fail".
- `metadata` _Optional[Dict[str, Any]]_ - The metadata associated with this
validation result.
- `validated_chunk` _Optional[Any]_ - The value argument passed to
validator.validate or validator.validate_stream.
## PassResult
```python
class PassResult(ValidationResult, IPassResult)
```
PassResult is the output type of Validator.validate when validation
succeeds.
**Attributes**:
- `outcome` _Literal["pass"]_ - The outcome of the validation. Must be "pass".
- `value_override` _Optional[Any]_ - The value to use as an override
if validation passes.
## FailResult
```python
class FailResult(ValidationResult, IFailResult)
```
FailResult is the output type of Validator.validate when validation
fails.
**Attributes**:
- `outcome` _Literal["fail"]_ - The outcome of the validation. Must be "fail".
- `error_message` _str_ - The error message indicating why validation failed.
- `fix_value` _Optional[Any]_ - The auto-fix value that would be applied
if the Validator's on_fail method is "fix".
- `error_spans` _Optional[List[ErrorSpan]]_ - Segments that caused
validation to fail.
## ErrorSpan
```python
class ErrorSpan(IErrorSpan, ArbitraryModel)
```
ErrorSpan provide additional context for why a validation failed. They
specify the start and end index of the segment that caused the failure,
which can be useful when validating large chunks of text or validating
while streaming with different chunking methods.
**Attributes**:
- `start` _int_ - Starting index relative to the validated chunk.
- `end` _int_ - Ending index relative to the validated chunk.
- `reason` _str_ - Reason validation failed for this chunk.
## ValidatorLogs
```python
class ValidatorLogs(IValidatorLog, ArbitraryModel)
```
Logs for a single validator execution.
**Attributes**:
- `validator_name` _str_ - The class name of the validator
- `registered_name` _str_ - The snake_cased id of the validator
- `property_path` _str_ - The JSON path to the property being validated
- `value_before_validation` _Any_ - The value before validation
- `value_after_validation` _Optional[Any]_ - The value after validation;
could be different if `value_override`s or `fix`es are applied
- `validation_result` _Optional[ValidationResult]_ - The result of the validation
- `start_time` _Optional[datetime]_ - The time the validation started
- `end_time` _Optional[datetime]_ - The time the validation ended
- `instance_id` _Optional[int]_ - The unique id of this instance of the validator
## ValidatorReference
```python
class ValidatorReference(IValidatorReference)
```
ValidatorReference is a serialized reference for constructing a
Validator.
**Attributes**:
- `id` _Optional[str]_ - The unique identifier for this Validator.
Often the hub id; e.g. guardrails/regex_match. Default None.
- `on` _Optional[str]_ - A reference to the property this validator should be
applied against. Can be a valid JSON path or a meta-property
such as `prompt` or `output`. Default None.
- `on_fail` _Optional[str]_ - The OnFailAction to apply during validation.
Default None.
- `args` _Optional[List[Any]]_ - Positional arguments. Default None.
- `kwargs` _Optional[Dict[str, Any]]_ - Keyword arguments. Default None.