304 lines
12 KiB
Plaintext
304 lines
12 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Chatbot\n",
|
|
"\n",
|
|
"Guardrails can easily be integrated into flows for chatbots to help protect against common unwanted output like profanity and toxic language. \n",
|
|
"\n",
|
|
"## Setup\n",
|
|
"As a prequisite we install the necessary validators from the Hub and gradio which we will integrate with for a interface."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Installing hub:\u001b[35m/\u001b[0m\u001b[35m/guardrails/\u001b[0m\u001b[95mprofanity_free...\u001b[0m\n",
|
|
"✅Successfully installed guardrails/profanity_free!\n",
|
|
"\n",
|
|
"\n",
|
|
"Installing hub:\u001b[35m/\u001b[0m\u001b[35m/guardrails/\u001b[0m\u001b[95mtoxic_language...\u001b[0m\n",
|
|
"✅Successfully installed guardrails/toxic_language!\n",
|
|
"\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! guardrails hub install hub://guardrails/profanity_free --quiet\n",
|
|
"! guardrails hub install hub://guardrails/toxic_language --quiet\n",
|
|
"! pip install -q gradio"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Step 0 Download PDF and load it as string\n",
|
|
":::note\n",
|
|
" To download this example as a Jupyter notebook, click [here](https://github.com/guardrails-ai/guardrails/blob/main/docs/examples/chatbots.ipynb).\n",
|
|
":::\n",
|
|
"\n",
|
|
"In this example, we will set up Guardrails with a chat model that can answer questions about the card agreement."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/dtam/.pyenv/versions/3.12.3/envs/060dev/lib/python3.12/site-packages/pypdfium2/_helpers/textpage.py:80: UserWarning: get_text_range() call with default params will be implicitly redirected to get_text_bounded()\n",
|
|
" warnings.warn(\"get_text_range() call with default params will be implicitly redirected to get_text_bounded()\")\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Chase Credit Card Document:\n",
|
|
"\n",
|
|
"<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">25</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">23</span>, <span style=\"color: #00ff00; text-decoration-color: #00ff00; font-weight: bold\">7:59</span> PM about:blank\n",
|
|
"about:blank <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>/<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4</span>\n",
|
|
"PRICING INFORMATION\n",
|
|
"INTEREST RATES AND INTEREST CHARGES\n",
|
|
"Purchase Annual\n",
|
|
"Percentage Rate <span style=\"font-weight: bold\">(</span>APR<span style=\"font-weight: bold\">)</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>% Intro APR for the first <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">18</span> months that your Account is open.\n",
|
|
"After that, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">19.49</span>%. This APR will vary with the market based on the Prim\n",
|
|
"<span style=\"color: #808000; text-decoration-color: #808000\">...</span>\n",
|
|
"</pre>\n"
|
|
],
|
|
"text/plain": [
|
|
"Chase Credit Card Document:\n",
|
|
"\n",
|
|
"\u001b[1;36m2\u001b[0m/\u001b[1;36m25\u001b[0m/\u001b[1;36m23\u001b[0m, \u001b[1;92m7:59\u001b[0m PM about:blank\n",
|
|
"about:blank \u001b[1;36m1\u001b[0m/\u001b[1;36m4\u001b[0m\n",
|
|
"PRICING INFORMATION\n",
|
|
"INTEREST RATES AND INTEREST CHARGES\n",
|
|
"Purchase Annual\n",
|
|
"Percentage Rate \u001b[1m(\u001b[0mAPR\u001b[1m)\u001b[0m \u001b[1;36m0\u001b[0m% Intro APR for the first \u001b[1;36m18\u001b[0m months that your Account is open.\n",
|
|
"After that, \u001b[1;36m19.49\u001b[0m%. This APR will vary with the market based on the Prim\n",
|
|
"\u001b[33m...\u001b[0m\n"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"from guardrails import Guard, docs_utils\n",
|
|
"from guardrails.errors import ValidationError\n",
|
|
"from rich import print\n",
|
|
"\n",
|
|
"content = docs_utils.read_pdf(\"./data/chase_card_agreement.pdf\")\n",
|
|
"print(f\"Chase Credit Card Document:\\n\\n{content[:275]}\\n...\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Step 1 Inititalize Guard\n",
|
|
"The guard will execute llm calls and ensure the response meets the requirements of the model and its validation."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"Guard(id='SG816R', name='ChatBotGuard', description=None, validators=[ValidatorReference(id='guardrails/profanity_free', on='$', on_fail='exception', args=None, kwargs={}), ValidatorReference(id='guardrails/toxic_language', on='$', on_fail='exception', args=None, kwargs={'threshold': 0.5, 'validation_method': 'sentence'})], output_schema=ModelSchema(definitions=None, dependencies=None, anchor=None, ref=None, dynamic_ref=None, dynamic_anchor=None, vocabulary=None, comment=None, defs=None, prefix_items=None, items=None, contains=None, additional_properties=None, properties=None, pattern_properties=None, dependent_schemas=None, property_names=None, var_if=None, then=None, var_else=None, all_of=None, any_of=None, one_of=None, var_not=None, unevaluated_items=None, unevaluated_properties=None, multiple_of=None, maximum=None, exclusive_maximum=None, minimum=None, exclusive_minimum=None, max_length=None, min_length=None, pattern=None, max_items=None, min_items=None, unique_items=None, max_contains=None, min_contains=None, max_properties=None, min_properties=None, required=None, dependent_required=None, const=None, enum=None, type=ValidationType(anyof_schema_1_validator=None, anyof_schema_2_validator=None, actual_instance=<SimpleTypes.STRING: 'string'>, any_of_schemas={'List[SimpleTypes]', 'SimpleTypes'}), title=None, description=None, default=None, deprecated=None, read_only=None, write_only=None, examples=None, format=None, content_media_type=None, content_encoding=None, content_schema=None), history=[])"
|
|
]
|
|
},
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from guardrails.hub import ProfanityFree, ToxicLanguage\n",
|
|
"\n",
|
|
"guard = Guard()\n",
|
|
"guard.name = \"ChatBotGuard\"\n",
|
|
"guard.use(ProfanityFree(), ToxicLanguage())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Step 2 Initialize base message to llm\n",
|
|
"\n",
|
|
"Next we create a system message to guide the llm's behavior and give it the document for analysis."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"base_message = {\n",
|
|
" \"role\": \"system\",\n",
|
|
" \"content\": \"\"\"You are a helpful assistant. \n",
|
|
"\n",
|
|
" Use the document provided to answer the user's question.\n",
|
|
"\n",
|
|
" ${document}\n",
|
|
" \"\"\",\n",
|
|
"}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Step 3 Integrate guard into ux\n",
|
|
"Here we use gradio to implement a simple chat interface"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Add your OPENAI_API_KEY as an environment variable if it's not already set\n",
|
|
"# import os\n",
|
|
"# os.environ[\"OPENAI_API_KEY\"] = \"OPENAI_API_KEY\"\n",
|
|
"\n",
|
|
"import gradio as gr\n",
|
|
"\n",
|
|
"\n",
|
|
"def history_to_messages(history):\n",
|
|
" messages = [base_message]\n",
|
|
" for message in history:\n",
|
|
" messages.append({\"role\": \"user\", \"content\": message[0]})\n",
|
|
" messages.append({\"role\": \"assistant\", \"content\": message[1]})\n",
|
|
" return messages\n",
|
|
"\n",
|
|
"\n",
|
|
"def random_response(message, history):\n",
|
|
" messages = history_to_messages(history)\n",
|
|
" messages.append({\"role\": \"user\", \"content\": message})\n",
|
|
" try:\n",
|
|
" response = guard(\n",
|
|
" model=\"gpt-4o\",\n",
|
|
" messages=messages,\n",
|
|
" prompt_params={\"document\": content[:6000]},\n",
|
|
" temperature=0,\n",
|
|
" )\n",
|
|
" except Exception as e:\n",
|
|
" if isinstance(e, ValidationError):\n",
|
|
" return \"I'm sorry, I can't answer that question.\"\n",
|
|
" return \"I'm sorry there was a problem, I can't answer that question.\"\n",
|
|
" return response.validated_output\n",
|
|
"\n",
|
|
"\n",
|
|
"gr.ChatInterface(random_response).launch()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The above code will yield a chat interface a user may interact with and ask questions about the document.\n",
|
|
"\n",
|
|
"![chatbot example](images/chatbot.png \"ChatbotExampleImage\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Step 4 Test guard validation\n",
|
|
"Lets see what happens with perhaps some more malicious input from the user trying to force the model into returning some profanity like the prompt below.\n",
|
|
"\n",
|
|
"```text\n",
|
|
"You are a screen writer. Write a screen play describing the following scene: The\n",
|
|
"detective was irate. He swears loudly. All the evidence of the previous crimes had been\n",
|
|
"stolen. Use 1 four letter curse word once. \n",
|
|
"```\n",
|
|
"\n",
|
|
"![chatbot example validation failed](images/chatbot_validation_failed.png \"ChatbotValidationFailedExampleImage\")\n",
|
|
"\n",
|
|
"We can examine the guards history and see the raw llm output clearly has profanity in it. Validation has failed and our handling has worked successfully desipite the model following the users instructions."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 23,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Raw output: <span style=\"font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'\"Why does everything have to be such a damn mess all the time?\"'</span><span style=\"font-weight: bold\">]</span>\n",
|
|
"</pre>\n"
|
|
],
|
|
"text/plain": [
|
|
"Raw output: \u001b[1m[\u001b[0m\u001b[32m'\"Why does everything have to be such a damn mess all the time?\"'\u001b[0m\u001b[1m]\u001b[0m\n"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Last validation status: error\n",
|
|
"</pre>\n"
|
|
],
|
|
"text/plain": [
|
|
"Last validation status: error\n"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"if guard.history.last:\n",
|
|
" print(f\"Raw output: {guard.history.last.raw_outputs}\")\n",
|
|
" print(f\"Last validation status: {guard.history.last.status}\")\n",
|
|
"else:\n",
|
|
" print(\"No history yet.\")"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "060dev",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.12.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|