379 lines
11 KiB
Plaintext
379 lines
11 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "e1c59071",
|
||
"metadata": {},
|
||
"source": [
|
||
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/output_parsing/GuardrailsDemo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9c48213d-6e6a-4c10-838a-2a7c710c3a05",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Guardrails Output Parsing (Deprecated)\n",
|
||
"\n",
|
||
"## DEPRECATION NOTE\n",
|
||
"This integration between LlamaIndex and Guardrails is only valid for llama-index ~0.9.x and guardrails-ai < 0.5.x. and thus has been deprecated. For an updated example of using Guardrails with LlamaIndex with their latest versions, see the [GuardrailsEngine](https://guardrailsai.com/guardrails/docs/integrations/llamaindex)\n"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "cf54a5a8",
|
||
"metadata": {},
|
||
"source": [
|
||
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "39bc790a",
|
||
"metadata": {},
|
||
"source": [
|
||
"#### Download Data"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "649bea0c",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"--2023-12-01 10:30:22-- https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt\n",
|
||
"Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'\n",
|
||
"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.108.133, 185.199.110.133, ...\n",
|
||
"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.\n",
|
||
"HTTP request sent, awaiting response... 200 OK\n",
|
||
"Length: 75042 (73K) [text/plain]\n",
|
||
"Saving to: ‘data/paul_graham/paul_graham_essay.txt’\n",
|
||
"\n",
|
||
"data/paul_graham/pa 100%[===================>] 73.28K --.-KB/s in 0.02s \n",
|
||
"\n",
|
||
"2023-12-01 10:30:23 (3.82 MB/s) - ‘data/paul_graham/paul_graham_essay.txt’ saved [75042/75042]\n",
|
||
"\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"!mkdir -p 'data/paul_graham/'\n",
|
||
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "50d3b817-b70e-4667-be4f-d3a0fe4bd119",
|
||
"metadata": {},
|
||
"source": [
|
||
"#### Load documents, build the VectorStoreIndex\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "690a6918-7c75-4f95-9ccc-d2c4a1fe00d7",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import logging\n",
|
||
"import sys\n",
|
||
"from llama_index import VectorStoreIndex, SimpleDirectoryReader\n",
|
||
"\n",
|
||
"logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
|
||
"logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "03d1691e-544b-454f-825b-5ee12f7faa8a",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# load documents\n",
|
||
"documents = SimpleDirectoryReader(\"./data/paul_graham/\").load_data()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "ad144ee7-96da-4dd6-be00-fd6cf0c78e58",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
|
||
"HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
|
||
"INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
|
||
"HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
|
||
"INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
|
||
"HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"index = VectorStoreIndex.from_documents(documents, chunk_size=512)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8b7d7c61-b5d7-4b8f-b90b-3ebee1103f27",
|
||
"metadata": {},
|
||
"source": [
|
||
"#### Define Query + Guardrails Spec\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "6fb88295-0840-4e2d-b79b-def0b0a63a7f",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from llama_index.output_parsers import GuardrailsOutputParser\n",
|
||
"from llama_index.llm_predictor import StructuredLLMPredictor"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "057139d2-09e8-4b8d-83a1-a2356a1475a8",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"llm_predictor = StructuredLLMPredictor()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "bc25edf7-9343-4e82-a3f1-eec4281a9371",
|
||
"metadata": {},
|
||
"source": [
|
||
"**Define custom QA and Refine Prompts**\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "2833d086-d240-4798-b3c5-a83ac4593b0e",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from llama_index.prompts import PromptTemplate\n",
|
||
"from llama_index.prompts.default_prompts import (\n",
|
||
" DEFAULT_TEXT_QA_PROMPT_TMPL,\n",
|
||
" DEFAULT_REFINE_PROMPT_TMPL,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dba8513e",
|
||
"metadata": {},
|
||
"source": [
|
||
"**Define Guardrails Spec**\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "a4b9201d-fe16-4cc0-8135-a08d9928625d",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# You can either define a RailSpec and initialise a Guard object for_rail_string()\n",
|
||
"# OR define Pydantic classes and initialise a Guard object for_pydantic()\n",
|
||
"# For more info: https://guardrailsai.com/guardrails/docs/getting_started/quickstart#usage\n",
|
||
"# Guardrails recommends Pydantic\n",
|
||
"\n",
|
||
"from pydantic import BaseModel, Field\n",
|
||
"import guardrails as gd\n",
|
||
"\n",
|
||
"\n",
|
||
"class Point(BaseModel):\n",
|
||
" # In all the fields below, you can define validators as well\n",
|
||
" # Left out for brevity\n",
|
||
" explanation: str = Field(description=\"The first thing the author worked on\")\n",
|
||
" explanation2: str = Field(description=\"The second thing the author worked on\")\n",
|
||
" explanation3: str = Field(description=\"The third thing the author worked on\")\n",
|
||
"\n",
|
||
"\n",
|
||
"class BulletPoints(BaseModel):\n",
|
||
" points: Point = Field(\n",
|
||
" description=\"Bullet points regarding events in the author's life.\"\n",
|
||
" )\n",
|
||
"\n",
|
||
"\n",
|
||
"# Define the prompt\n",
|
||
"prompt = \"\"\"\n",
|
||
"Query string here.\n",
|
||
"\n",
|
||
"${output_schema}\n",
|
||
"\n",
|
||
"${gr.json_suffix_prompt_v2_wo_none}\n",
|
||
"\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "f7af4ebf-1dff-48ec-9fb7-8926af45b6a0",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Create a guard object\n",
|
||
"guard = gd.Guard.for_pydantic(output_class=BulletPoints, prompt=prompt)\n",
|
||
"\n",
|
||
"# Create output parse object\n",
|
||
"output_parser = GuardrailsOutputParser(guard, llm=llm_predictor.llm.complete)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "a9b440d4-6fb4-46e6-973f-44207b432d3f",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# NOTE: we use the same output parser for both prompts, though you can choose to use different parsers\n",
|
||
"# NOTE: here we add formatting instructions to the prompts.\n",
|
||
"\n",
|
||
"fmt_qa_tmpl = output_parser.format(DEFAULT_TEXT_QA_PROMPT_TMPL)\n",
|
||
"fmt_refine_tmpl = output_parser.format(DEFAULT_REFINE_PROMPT_TMPL)\n",
|
||
"\n",
|
||
"qa_prompt = PromptTemplate(fmt_qa_tmpl, output_parser=output_parser)\n",
|
||
"refine_prompt = PromptTemplate(fmt_refine_tmpl, output_parser=output_parser)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "1ba18a80-35f4-4fd4-9b13-9f13f84db4fe",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Context information is below.\n",
|
||
"---------------------\n",
|
||
"{context_str}\n",
|
||
"---------------------\n",
|
||
"Given the context information and not prior knowledge, answer the query.\n",
|
||
"Query: {query_str}\n",
|
||
"Answer: \n",
|
||
"\n",
|
||
"ct name=\"points\" description=\"Bullet points regarding events in the author's life.\">\n",
|
||
" <string name=\"explanation\" description=\"The first thing the author worked on\"/>\n",
|
||
" <string name=\"explanation2\" description=\"The second thing the author worked on\"/>\n",
|
||
" <string name=\"explanation3\" description=\"The third thing the author worked on\"/>\n",
|
||
" </object>\n",
|
||
"</output>\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"ONLY return a valid JSON object (no other text is necessary). The JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and specific types. Be correct and concise.\n",
|
||
"\n",
|
||
"\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# take a look at the new QA template!\n",
|
||
"print(fmt_qa_tmpl)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b6caf93b-6345-4c65-a346-a95b0f1746c4",
|
||
"metadata": {},
|
||
"source": [
|
||
"#### Query Index\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "fb9cdf43-0f31-4c36-869b-df9fa50aebdb",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
|
||
"HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
|
||
"INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
|
||
"HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"query_engine = index.as_query_engine(\n",
|
||
" text_qa_template=qa_prompt,\n",
|
||
" refine_template=refine_prompt,\n",
|
||
" llm_predictor=llm_predictor,\n",
|
||
")\n",
|
||
"response = query_engine.query(\n",
|
||
" \"What are the three items the author did growing up?\",\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "bc7760b6-5be3-4303-b97e-3f5edacf674b",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"{\n",
|
||
" \"points\": {\n",
|
||
" \"explanation\": \"writing\",\n",
|
||
" \"explanation2\": \"programming\",\n",
|
||
" \"explanation3\": \"philosophy\"\n",
|
||
" }\n",
|
||
"}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(response)"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "litellm",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.12.3"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|