1582 lines
49 KiB
ReStructuredText
1582 lines
49 KiB
ReStructuredText
.. _user-guide-rag:
|
|
|
|
User Guide: RAG
|
|
###############
|
|
|
|
This guide provides a starting point for using the Neo4j GraphRAG package
|
|
and configuring it according to specific requirements.
|
|
|
|
|
|
**********
|
|
Quickstart
|
|
**********
|
|
|
|
To perform a GraphRAG query using the `neo4j-graphrag` package, a few components are needed:
|
|
|
|
1. A Neo4j driver: used to query your Neo4j database.
|
|
2. A Retriever: the `neo4j-graphrag` package provides some implementations (see the :ref:`dedicated section <retriever-configuration>`) and lets you write your own if none of the provided implementations matches your needs (see :ref:`how to write a custom retriever <custom-retriever>`).
|
|
3. An LLM: to generate the answer, we need to call an LLM model. The neo4j-graphrag package's LLM interface is compatible with LangChain. Developers can also write their own interface if needed.
|
|
|
|
In practice, it's done with only a few lines of code:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j import GraphDatabase
|
|
from neo4j_graphrag.retrievers import VectorRetriever
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
from neo4j_graphrag.generation import GraphRAG
|
|
from neo4j_graphrag.embeddings import OpenAIEmbeddings
|
|
|
|
# 1. Neo4j driver
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
INDEX_NAME = "index-name"
|
|
|
|
# Connect to Neo4j database
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
|
|
# 2. Retriever
|
|
# Create Embedder object, needed to convert the user question (text) to a vector
|
|
embedder = OpenAIEmbeddings(model="text-embedding-3-large")
|
|
|
|
# Initialize the retriever
|
|
retriever = VectorRetriever(driver, INDEX_NAME, embedder)
|
|
|
|
# 3. LLM
|
|
# Note: the OPENAI_API_KEY must be in the env vars
|
|
llm = OpenAILLM(model_name="gpt-5", model_params={"temperature": 0})
|
|
|
|
# Initialize the RAG pipeline
|
|
rag = GraphRAG(retriever=retriever, llm=llm)
|
|
|
|
# Query the graph
|
|
query_text = "How do I do similarity search in Neo4j?"
|
|
response = rag.search(query_text=query_text, retriever_config={"top_k": 5})
|
|
print(response.answer)
|
|
|
|
|
|
.. warning::
|
|
|
|
Using `OpenAILLM` requires the `openai` Python client. You can install it with `pip install "neo4j_graphrag[openai]"`.
|
|
|
|
|
|
The following sections provide more details about how to customize this code.
|
|
|
|
**********************
|
|
GraphRAG Configuration
|
|
**********************
|
|
|
|
Each component can be configured individually: the LLM and the prompt.
|
|
|
|
Using Another LLM Model
|
|
========================
|
|
|
|
If OpenAI cannot be used directly, there are a few available alternatives:
|
|
|
|
- Use Azure OpenAI (GPT...).
|
|
- Use Google VertexAI (Gemini...).
|
|
- Use Anthropic LLM (Claude...).
|
|
- Use Mistral LLM
|
|
- Use Cohere.
|
|
- Use a local Ollama model.
|
|
- Implement a custom interface.
|
|
- Utilize any LangChain chat model.
|
|
|
|
All options are illustrated below.
|
|
|
|
Using Azure Open AI LLM
|
|
-----------------------
|
|
|
|
It is possible to use Azure OpenAI switching to the `AzureOpenAILLM` class:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import AzureOpenAILLM
|
|
llm = AzureOpenAILLM(
|
|
model_name="gpt-5",
|
|
azure_endpoint="https://example-endpoint.openai.azure.com/", # update with your endpoint
|
|
api_version="2024-06-01", # update appropriate version
|
|
api_key="...", # api_key is optional and can also be set with OPENAI_API_KEY env var
|
|
)
|
|
llm.invoke("say something")
|
|
|
|
Check the OpenAI Python client `documentation <https://github.com/openai/openai-python?tab=readme-ov-file#microsoft-azure-openai>`_.
|
|
to learn more about the configuration.
|
|
|
|
.. note::
|
|
|
|
In order to run this code, the `openai` Python package needs to be installed:
|
|
`pip install "neo4j_graphrag[openai]"`
|
|
|
|
|
|
See :ref:`azureopenaillm`.
|
|
|
|
|
|
Using VertexAI LLM
|
|
------------------
|
|
|
|
To use VertexAI, instantiate the `VertexAILLM` class:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import VertexAILLM
|
|
from vertexai.generative_models import GenerationConfig
|
|
|
|
generation_config = GenerationConfig(temperature=0.0)
|
|
llm = VertexAILLM(
|
|
model_name="gemini-2.5-flash", generation_config=generation_config
|
|
)
|
|
llm.invoke("say something")
|
|
|
|
|
|
.. note::
|
|
|
|
In order to run this code, the `google-cloud-aiplatform` Python package needs to be installed:
|
|
`pip install "neo4j_graphrag[google]"`
|
|
|
|
|
|
See :ref:`vertexaillm`.
|
|
|
|
|
|
Using Anthropic LLM
|
|
-------------------
|
|
|
|
To use Anthropic, instantiate the `AnthropicLLM` class:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import AnthropicLLM
|
|
|
|
llm = AnthropicLLM(
|
|
model_name="claude-3-opus-20240229",
|
|
model_params={"max_tokens": 1000}, # max_tokens must be specified
|
|
api_key=api_key, # can also set `ANTHROPIC_API_KEY` in env vars
|
|
)
|
|
llm.invoke("say something")
|
|
|
|
|
|
.. note::
|
|
|
|
In order to run this code, the `anthropic` Python package needs to be installed:
|
|
`pip install "neo4j_graphrag[anthropic]"`
|
|
|
|
See :ref:`anthropicllm`.
|
|
|
|
|
|
Using MistralAI LLM
|
|
-------------------
|
|
|
|
To use MistralAI, instantiate the `MistralAILLM` class:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import MistralAILLM
|
|
|
|
llm = MistralAILLM(
|
|
model_name="mistral-small-latest",
|
|
api_key=api_key, # can also set `MISTRAL_API_KEY` in env vars
|
|
)
|
|
llm.invoke("say something")
|
|
|
|
|
|
.. note::
|
|
|
|
In order to run this code, the `mistralai` Python package needs to be installed:
|
|
`pip install "neo4j_graphrag[mistralai]"`
|
|
|
|
See :ref:`mistralaillm`.
|
|
|
|
|
|
|
|
Using Cohere LLM
|
|
----------------
|
|
|
|
To use Cohere, instantiate the `CohereLLM` class:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import CohereLLM
|
|
|
|
llm = CohereLLM(
|
|
model_name="command-r",
|
|
api_key=api_key, # can also set `CO_API_KEY` in env vars
|
|
)
|
|
llm.invoke("say something")
|
|
|
|
|
|
.. note::
|
|
|
|
In order to run this code, the `cohere` Python package needs to be installed:
|
|
`pip install "neo4j_graphrag[cohere]"`
|
|
|
|
|
|
See :ref:`coherellm`.
|
|
|
|
|
|
Using a Local Model via Ollama
|
|
-------------------------------
|
|
|
|
Assuming Ollama is running on the default address `127.0.0.1:11434`,
|
|
it can be queried using the following:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import OllamaLLM
|
|
llm = OllamaLLM(
|
|
model_name="orca-mini",
|
|
# model_params={"options": {"temperature": 0}, "format": "json"},
|
|
# host="...", # when using a remote server
|
|
)
|
|
llm.invoke("say something")
|
|
|
|
|
|
Using a Model from LangChain
|
|
-----------------------------
|
|
|
|
The LangChain Python package contains implementations for various LLMs and providers.
|
|
Its interface is compatible with our `GraphRAG` interface, facilitating integration:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.generation import GraphRAG
|
|
from langchain_community.chat_models import ChatOllama
|
|
|
|
# retriever = ...
|
|
|
|
llm = ChatOllama(model="llama3:8b")
|
|
rag = GraphRAG(retriever=retriever, llm=llm)
|
|
query_text = "How do I do similarity search in Neo4j?"
|
|
response = rag.search(query_text=query_text, retriever_config={"top_k": 5})
|
|
print(response.answer)
|
|
|
|
|
|
It is however not mandatory to use LangChain.
|
|
|
|
Using a Custom Model
|
|
--------------------
|
|
|
|
If the provided implementations do not match their needs, developers can create a
|
|
custom LLM class by subclassing :class:`neo4j_graphrag.llm.LLMBase`.
|
|
``LLMBase`` combines ``LLMInterface`` (str input, v1 API) and ``LLMInterfaceV2``
|
|
(``List[LLMMessage]`` input, structured output) into a single abstract base class.
|
|
Subclasses implement one ``invoke`` and one ``ainvoke`` dispatcher that branches on
|
|
the type of *input*.
|
|
|
|
Here's an example using the Python Ollama client:
|
|
|
|
.. code:: python
|
|
|
|
from typing import Any, List, Optional, Type, Union
|
|
import ollama
|
|
from pydantic import BaseModel
|
|
from neo4j_graphrag.llm import LLMBase, LLMResponse
|
|
from neo4j_graphrag.message_history import MessageHistory
|
|
from neo4j_graphrag.types import LLMMessage
|
|
|
|
class MyOllamaLLM(LLMBase):
|
|
|
|
def invoke(
|
|
self,
|
|
input: Union[str, List[LLMMessage]],
|
|
message_history=None,
|
|
system_instruction=None,
|
|
response_format=None,
|
|
**kwargs: Any,
|
|
) -> LLMResponse:
|
|
if isinstance(input, str):
|
|
messages = [{"role": "user", "content": input}]
|
|
else:
|
|
messages = list(input)
|
|
response = ollama.chat(model=self.model_name, messages=messages)
|
|
return LLMResponse(content=response["message"]["content"])
|
|
|
|
async def ainvoke(
|
|
self,
|
|
input: Union[str, List[LLMMessage]],
|
|
message_history=None,
|
|
system_instruction=None,
|
|
response_format=None,
|
|
**kwargs: Any,
|
|
) -> LLMResponse:
|
|
return self.invoke(input) # TODO: implement with ollama.AsyncClient
|
|
|
|
|
|
# retriever = ...
|
|
|
|
llm = MyOllamaLLM("llama3:8b")
|
|
|
|
rag = GraphRAG(retriever=retriever, llm=llm)
|
|
query_text = "How do I do similarity search in Neo4j?"
|
|
response = rag.search(query_text=query_text, retriever_config={"top_k": 5})
|
|
print(response.answer)
|
|
|
|
See :ref:`llminterface`.
|
|
|
|
|
|
Structured Output with LLMs
|
|
============================
|
|
|
|
Structured output enables LLMs to return responses conforming to a predefined schema (Pydantic model or JSON schema), ensuring type-safe and consistent data structures. This is useful for extracting entities, relationships, or any structured data with automatic validation.
|
|
|
|
**V2 Interface (Recommended)**: For :ref:`OpenAILLM <openaillm>` and :ref:`VertexAILLM <vertexaillm>`, pass `response_format` as a parameter to the `invoke()` method when using the V2 interface (list of `LLMMessage`). The `response_format` accepts either a Pydantic model class or a JSON schema dictionary. Other LLM providers will raise `NotImplementedError` if `response_format` is used.
|
|
|
|
**V1 Interface (Legacy)**: With the V1 interface (string input), standard JSON mode is supported for both OpenAI and VertexAI via constructor parameters only (`model_params` for OpenAI, `generation_config` for VertexAI). The `response_format` parameter in `invoke()` is not permitted with V1.
|
|
|
|
.. code:: python
|
|
|
|
from pydantic import BaseModel, ConfigDict
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
from neo4j_graphrag.types import LLMMessage
|
|
|
|
class Person(BaseModel):
|
|
model_config = ConfigDict(extra="forbid") # Required for OpenAI structured output
|
|
name: str
|
|
age: int
|
|
occupation: str
|
|
|
|
llm = OpenAILLM(model_name="gpt-5-mini")
|
|
|
|
# V2: Pass response_format to invoke()
|
|
messages = [LLMMessage(role="user", content="Extract: John is a 30 year old engineer.")]
|
|
response = llm.invoke(messages, response_format=Person, temperature=0)
|
|
person = Person.model_validate_json(response.content) # {"name": "John", "age": 30, ...}
|
|
|
|
# V1: Use constructor parameters for standard JSON mode
|
|
llm_v1 = OpenAILLM(
|
|
model_name="gpt-5-mini",
|
|
model_params={"response_format": {"type": "json_object"}, "temperature": 0}
|
|
)
|
|
response_v1 = llm_v1.invoke("Extract person in JSON format: John is 30 years old.")
|
|
|
|
OpenAI Structured Output
|
|
-------------------------
|
|
|
|
OpenAI supports Pydantic models, JSON schemas, and JSON object mode. **Important**: Pydantic models must include `ConfigDict(extra="forbid")` to generate schemas with `additionalProperties: false`, which is required by OpenAI's strict mode.
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
from neo4j_graphrag.types import LLMMessage
|
|
|
|
llm = OpenAILLM(model_name="gpt-5-mini")
|
|
|
|
# JSON schema
|
|
json_schema = {
|
|
"type": "object",
|
|
"properties": {"name": {"type": "string"}, "age": {"type": "integer"}},
|
|
"required": ["name", "age"]
|
|
}
|
|
messages = [LLMMessage(role="user", content="Extract: John is 30.")]
|
|
response = llm.invoke(messages, response_format=json_schema, temperature=0)
|
|
|
|
# JSON object mode (no schema enforcement)
|
|
response = llm.invoke(messages, response_format={"type": "json_object"}, temperature=0.5)
|
|
|
|
VertexAI Structured Output
|
|
---------------------------
|
|
|
|
VertexAI uses `GenerationConfig` with `response_mime_type` and `response_schema` internally when `response_format` is passed to `invoke()`. Both Pydantic models and JSON schemas are supported. **Important**: Additional `GenerationConfig` parameters (e.g., `temperature`, `max_output_tokens`) can be passed as kwargs to `invoke()`.
|
|
|
|
.. code:: python
|
|
|
|
from pydantic import BaseModel, ConfigDict
|
|
from neo4j_graphrag.llm import VertexAILLM
|
|
from neo4j_graphrag.types import LLMMessage
|
|
|
|
class Person(BaseModel):
|
|
model_config = ConfigDict(extra="forbid")
|
|
name: str
|
|
age: int
|
|
|
|
llm = VertexAILLM(model_name="gemini-1.5-pro")
|
|
messages = [LLMMessage(role="user", content="Extract: John is 30.")]
|
|
response = llm.invoke(messages, response_format=Person, temperature=0)
|
|
person = Person.model_validate_json(response.content)
|
|
|
|
|
|
Rate Limit Handling
|
|
===================
|
|
|
|
All LLM implementations include automatic rate limiting that uses retry logic with exponential backoff by default. This feature helps handle API rate limits from LLM providers gracefully by automatically retrying failed requests with increasing wait times between attempts.
|
|
|
|
Default Rate Limit Handler
|
|
--------------------------
|
|
|
|
Rate limiting is enabled by default for all LLM instances with the following configuration:
|
|
|
|
- **Max attempts**: 3
|
|
- **Min wait**: 1.0 seconds
|
|
- **Max wait**: 60.0 seconds
|
|
- **Multiplier**: 2.0 (exponential backoff)
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
|
|
# Rate limiting is automatically enabled
|
|
llm = OpenAILLM(model_name="gpt-5")
|
|
|
|
# The LLM will automatically retry on rate limit errors
|
|
response = llm.invoke("Hello, world!")
|
|
|
|
.. note::
|
|
|
|
To change the default configuration of `RetryRateLimitHandler`:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
from neo4j_graphrag.utils.rate_limit import RetryRateLimitHandler
|
|
|
|
# Customize rate limiting parameters
|
|
llm = OpenAILLM(
|
|
model_name="gpt-5",
|
|
rate_limit_handler=RetryRateLimitHandler(
|
|
max_attempts=10, # Increase max retry attempts
|
|
min_wait=2.0, # Increase minimum wait time
|
|
max_wait=120.0, # Increase maximum wait time
|
|
multiplier=3.0 # More aggressive backoff
|
|
)
|
|
)
|
|
|
|
Custom Rate Limiting
|
|
--------------------
|
|
|
|
You can customize the rate limiting behavior by creating your own rate limit handler:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import AnthropicLLM
|
|
from neo4j_graphrag.utils.rate_limit import RateLimitHandler
|
|
|
|
class CustomRateLimitHandler(RateLimitHandler):
|
|
"""Implement your custom rate limiting strategy."""
|
|
# Implement required methods: handle_sync, handle_async
|
|
# and optionally override is_retryable_exception and to_retryable_error
|
|
# to classify additional exception types as retryable or convert them to retryable errors
|
|
pass
|
|
|
|
# Create custom rate limit handler and pass it to the LLM interface
|
|
custom_handler = CustomRateLimitHandler()
|
|
|
|
llm = AnthropicLLM(
|
|
model_name="claude-3-sonnet-20240229",
|
|
rate_limit_handler=custom_handler,
|
|
)
|
|
|
|
Disabling Rate Limiting
|
|
-----------------------
|
|
|
|
For high-throughput applications or when you handle rate limiting externally, you can disable it:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.llm import CohereLLM, NoOpRateLimitHandler
|
|
|
|
# Disable rate limiting completely
|
|
llm = CohereLLM(
|
|
model_name="command-r-plus",
|
|
rate_limit_handler=NoOpRateLimitHandler(),
|
|
)
|
|
llm.invoke("Hello, world!")
|
|
|
|
|
|
Configuring the Prompt
|
|
========================
|
|
|
|
Prompts are managed through `PromptTemplate` classes. Specifically, the `GraphRAG` pipeline
|
|
utilizes a `RagTemplate` with a default prompt that can be accessed through
|
|
`rag.prompt_template.template`. To use a different prompt, subclass the `RagTemplate`
|
|
class and pass it to the `GraphRAG` pipeline object during initialization:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.generation import RagTemplate, GraphRAG
|
|
|
|
# retriever = ...
|
|
# llm = ...
|
|
|
|
prompt_template = RagTemplate(
|
|
prompt="Answer the question {question} using context {context} and examples {examples}",
|
|
expected_inputs=["context", "question", "examples"]
|
|
)
|
|
|
|
rag = GraphRAG(retriever=retriever, llm=llm, prompt_template=prompt_template)
|
|
|
|
# ...
|
|
|
|
|
|
See :ref:`prompttemplate`.
|
|
|
|
|
|
The final configurable component in the `GraphRAG` pipeline is the retriever.
|
|
Below are descriptions of the various options available.
|
|
|
|
.. _retriever-configuration:
|
|
|
|
************************
|
|
Retriever Configuration
|
|
************************
|
|
|
|
We provide implementations for the following retrievers:
|
|
|
|
.. list-table:: List of retrievers
|
|
:widths: 30 100
|
|
:header-rows: 1
|
|
|
|
* - Retriever
|
|
- Description
|
|
* - :ref:`VectorRetriever <vector-retriever-user-guide>`
|
|
- Performs a similarity search based on a Neo4j vector index and a query text or vector. Returns the matched `node` and similarity `score`.
|
|
* - :ref:`VectorCypherRetriever <vector-cypher-retriever-user-guide>`
|
|
- Performs a similarity search based on a Neo4j vector index and a query text or vector. The returned results can be configured through a retrieval query parameter that will be executed after the index search. It can be used to fetch more context around the matched node.
|
|
* - :ref:`HybridRetriever <hybrid-retriever-user-guide>`
|
|
- Uses both a vector and a full-text index in Neo4j.
|
|
* - :ref:`HybridCypherRetriever <hybrid-cypher-retriever-user-guide>`
|
|
- Same as HybridRetriever with a retrieval query similar to VectorCypherRetriever.
|
|
* - :ref:`ToolsRetriever <tools-retriever-user-guide>`
|
|
- Uses an LLM to intelligently select and execute appropriate tools based on user queries. Combines results from multiple tools with proper attribution.
|
|
* - :ref:`Text2Cypher <text2cypher-retriever-user-guide>`
|
|
- Translates the user question into a Cypher query to be run against a Neo4j database (or Knowledge Graph). The results of the query are then passed to the LLM to generate the final answer.
|
|
* - :ref:`WeaviateNeo4jRetriever <weaviate-neo4j-retriever-user-guide>`
|
|
- Use this retriever when vectors are saved in a Weaviate vector database
|
|
* - :ref:`PineconeNeo4jRetriever <pinecone-neo4j-retriever-user-guide>`
|
|
- Use this retriever when vectors are saved in a Pinecone vector database
|
|
* - :ref:`QdrantNeo4jRetriever <qdrant-neo4j-retriever-user-guide>`
|
|
- Use this retriever when vectors are saved in a Qdrant vector database
|
|
|
|
Retrievers all expose a `search` method that we will discuss in the next sections.
|
|
|
|
|
|
.. _vector-retriever-user-guide:
|
|
|
|
Vector Retriever
|
|
===================
|
|
|
|
The simplest method to instantiate a vector retriever is:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.retrievers import VectorRetriever
|
|
|
|
retriever = VectorRetriever(
|
|
driver,
|
|
index_name=POSTER_INDEX_NAME,
|
|
)
|
|
|
|
The `index_name` is the name of the Neo4j vector index that will be used for similarity search.
|
|
|
|
|
|
.. warning::
|
|
|
|
Vector index use an **approximate nearest neighbor** algorithm.
|
|
Refer to the `Neo4j Documentation <https://neo4j.com/docs/cypher-manual/current/indexes/semantic-indexes/vector-indexes/#limitations-and-issues>`_ to learn about its limitations.
|
|
|
|
|
|
Search Similar Vector
|
|
-----------------------------
|
|
|
|
To identify the top 3 most similar nodes, perform a search by vector:
|
|
|
|
.. code:: python
|
|
|
|
vector = [] # a list of floats, same size as the vectors in the Neo4j vector index
|
|
retriever_result = retriever.search(query_vector=vector, top_k=3)
|
|
|
|
However, in most cases, a text (from the user) will be provided instead of a vector.
|
|
In this scenario, an `Embedder` is required.
|
|
|
|
Search Similar Text
|
|
--------------------
|
|
|
|
When searching for a text, specifying how the retriever transforms (embeds) the text
|
|
into a vector is required. Therefore, the retriever requires knowledge of an embedder:
|
|
|
|
.. code:: python
|
|
|
|
embedder = OpenAIEmbeddings(model="text-embedding-3-large")
|
|
|
|
# Initialize the retriever
|
|
retriever = VectorRetriever(
|
|
driver,
|
|
index_name=POSTER_INDEX_NAME,
|
|
embedder=embedder,
|
|
)
|
|
|
|
query_text = "How do I do similarity search in Neo4j?"
|
|
retriever_result = retriever.search(query_text=query_text, top_k=3)
|
|
|
|
|
|
Embedders
|
|
---------
|
|
|
|
Currently, this package supports the following embedders:
|
|
|
|
- :ref:`openaiembeddings`
|
|
- :ref:`sentencetransformerembeddings`
|
|
- :ref:`vertexaiembeddings`
|
|
- :ref:`mistralaiembeddings`
|
|
- :ref:`cohereembeddings`
|
|
- :ref:`azureopenaiembeddings`
|
|
- :ref:`ollamaembeddings`
|
|
|
|
The `OpenAIEmbeddings` was illustrated previously. Here is how to use the `SentenceTransformerEmbeddings`:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.embeddings import SentenceTransformerEmbeddings
|
|
|
|
embedder = SentenceTransformerEmbeddings(model="all-MiniLM-L6-v2") # Note: this is the default model
|
|
|
|
|
|
If another embedder is desired, a custom embedder can be created, using the `Embedder` interface.
|
|
|
|
Embedder Rate Limiting
|
|
----------------------
|
|
|
|
All embedder implementations include automatic rate limiting that uses retry logic with exponential backoff by default, similar to LLM implementations. This feature helps handle API rate limits from embedding providers gracefully.
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.embeddings import OpenAIEmbeddings
|
|
from neo4j_graphrag.utils.rate_limit import RetryRateLimitHandler, NoOpRateLimitHandler
|
|
|
|
# Default rate limiting (automatically enabled)
|
|
embedder = OpenAIEmbeddings(model="text-embedding-3-large")
|
|
|
|
# Custom rate limiting configuration
|
|
embedder = OpenAIEmbeddings(
|
|
model="text-embedding-3-large",
|
|
rate_limit_handler=RetryRateLimitHandler(
|
|
max_attempts=5,
|
|
min_wait=2.0,
|
|
max_wait=120.0
|
|
)
|
|
)
|
|
|
|
# Disable rate limiting
|
|
embedder = OpenAIEmbeddings(
|
|
model="text-embedding-3-large",
|
|
rate_limit_handler=NoOpRateLimitHandler()
|
|
)
|
|
|
|
The rate limiting configuration works the same way as for LLMs. See the :ref:`Rate Limit Handling <Rate Limit Handling>` section above for more details on customization options.
|
|
|
|
|
|
Other Vector Retriever Configuration
|
|
----------------------------------------
|
|
|
|
Often, not all node properties are pertinent for the RAG context; only a selected few are relevant
|
|
for inclusion in the LLM prompt context. You can specify which properties to return
|
|
using the `return_properties` parameter:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.retrievers import VectorRetriever
|
|
|
|
retriever = VectorRetriever(
|
|
driver,
|
|
index_name=POSTER_INDEX_NAME,
|
|
embedder=embedder,
|
|
return_properties=["title"],
|
|
)
|
|
|
|
|
|
Pre-Filters
|
|
-----------
|
|
|
|
When performing a similarity search, one may have constraints to apply.
|
|
For instance, filtering out movies released before 2000. This can be achieved
|
|
using `filters`.
|
|
|
|
.. note::
|
|
|
|
Filters are implemented for all retrievers except the Hybrid retrievers.
|
|
The documentation below is not valid for external retrievers, which use
|
|
their own filter syntax (see :ref:`vector-databases-section`).
|
|
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.retrievers import VectorRetriever
|
|
|
|
retriever = VectorRetriever(
|
|
driver,
|
|
index_name=POSTER_INDEX_NAME,
|
|
)
|
|
|
|
filters = {
|
|
"year": {
|
|
"$gte": 2000,
|
|
}
|
|
}
|
|
|
|
query_text = "How do I do similarity search in Neo4j?"
|
|
retriever_result = retriever.search(query_text=query_text, filters=filters)
|
|
|
|
.. warning::
|
|
|
|
On Neo4j versions prior to 2026.01, filters cause the similarity search to bypass the
|
|
vector index and use a brute-force exact match algorithm. Ensure that the pre-filtering
|
|
is stringent enough to prevent query overload.
|
|
|
|
On Neo4j 2026.01+, simple filters (see below) are automatically routed to use the
|
|
SEARCH clause with in-index filtering, which is significantly faster. See
|
|
:ref:`search-clause-filtering` for details.
|
|
|
|
The currently supported operators are:
|
|
|
|
- `$eq`: equal.
|
|
- `$ne`: not equal.
|
|
- `$lt`: less than.
|
|
- `$lte`: less than or equal to.
|
|
- `$gt`: greater than.
|
|
- `$gte`: greater than or equal to.
|
|
- `$between`: between.
|
|
- `$in`: value is in a given list.
|
|
- `$nin`: not in.
|
|
- `$like`: LIKE operator case-sensitive.
|
|
- `$ilike`: LIKE operator case-insensitive.
|
|
|
|
.. _search-clause-filtering:
|
|
|
|
In-Index Filtering with the SEARCH Clause (Neo4j 2026.01+)
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
On Neo4j 2026.01 and later, the library automatically detects the server version and
|
|
uses the Cypher ``SEARCH`` clause for vector queries. This enables in-index filtering,
|
|
where compatible filter predicates are evaluated inside the vector index itself rather
|
|
than via post-hoc brute-force scanning.
|
|
|
|
**Requirements:**
|
|
|
|
1. Neo4j server version 2026.01 or later.
|
|
2. To filter, the vector index must declare every filtered property in
|
|
``filterable_properties`` (see :ref:`filterable-index-creation`). If any filtered
|
|
property is missing, the library falls back to the procedure path and logs
|
|
a warning. Unfiltered queries do not require ``filterable_properties``.
|
|
3. The filter must use only SEARCH-compatible operators.
|
|
|
|
**SEARCH-compatible operators** (use in-index filtering):
|
|
|
|
- ``$eq``, ``$ne``
|
|
- ``$lt``, ``$lte``, ``$gt``, ``$gte``
|
|
- ``$between``
|
|
- Multiple filters combined with implicit ``$and``
|
|
|
|
**Operators that fall back to brute-force filtering:**
|
|
|
|
- ``$or``
|
|
- ``$in``, ``$nin``
|
|
- ``$like``, ``$ilike``
|
|
|
|
When a filter uses incompatible operators on a SEARCH-capable server, the library
|
|
automatically falls back to the procedure-based search path and logs a warning.
|
|
|
|
.. note::
|
|
|
|
The SEARCH clause routing is fully automatic. No code changes are needed beyond
|
|
creating the index with ``filterable_properties``. The same ``filters`` parameter
|
|
works on all Neo4j versions.
|
|
|
|
|
|
Here are examples of valid filter syntaxes and their meaning:
|
|
|
|
.. list-table:: Filters syntax
|
|
:widths: 80 80
|
|
:header-rows: 1
|
|
|
|
* - Filter
|
|
- Meaning
|
|
* - {"year": 1999}
|
|
- year = 1999
|
|
* - {"year": {"$eq": 1999}}
|
|
- year = 1999
|
|
* - {"year": 2000, "title": "The Matrix"}
|
|
- year = 1999 AND title = "The Matrix"
|
|
* - {"$and": [{"year": 2000}, {"title": "The Matrix"}]}
|
|
- year = 1999 AND title = "The Matrix"
|
|
* - {"$or": [{"title": "The Matrix Revolution"}, {"title": "The Matrix"}]}
|
|
- title = "The Matrix" OR title = "The Matrix Revolution"
|
|
* - {"title": {"$like": "The Matrix"}}
|
|
- title CONTAINS "The Matrix"
|
|
* - {"title": {"$ilike": "the matrix"}}
|
|
- toLower(title) CONTAINS "The Matrix"
|
|
|
|
|
|
See also :ref:`vectorretriever`.
|
|
|
|
.. _vector-cypher-retriever-user-guide:
|
|
|
|
Vector Cypher Retriever
|
|
=======================
|
|
|
|
The `VectorCypherRetriever` fully leverages Neo4j's graph capabilities by combining vector-based similarity searches with graph traversal techniques. It processes a query embedding to perform a similarity search against a specified vector index, retrieves relevant node variables, and then executes a Cypher query to traverse the graph based on these nodes. This integration ensures that retrievals are both semantically meaningful and contextually enriched by the underlying graph structure.
|
|
|
|
|
|
Retrieval Query
|
|
---------------
|
|
|
|
When crafting the retrieval query, it's important to note two available variables
|
|
are in the query scope:
|
|
|
|
- `node`: represents the node retrieved from the vector index search.
|
|
- `score`: denotes the similarity score.
|
|
|
|
For instance, in a movie graph with actors where the vector index pertains to
|
|
certain movie properties, the retrieval query can be structured as follows:
|
|
|
|
.. code:: python
|
|
|
|
retrieval_query = """
|
|
RETURN node.title as movieTitle,
|
|
node.plot as moviePlot,
|
|
collect { MATCH (actor:Actor)-[:ACTED_IN]->(node) RETURN actor.name } AS actors
|
|
"""
|
|
retriever = VectorCypherRetriever(
|
|
driver,
|
|
index_name=INDEX_NAME,
|
|
retrieval_query=retrieval_query,
|
|
)
|
|
|
|
|
|
It is recommended that the retrieval query returns node properties, as opposed to nodes.
|
|
|
|
|
|
Format the Results
|
|
------------------
|
|
|
|
.. warning::
|
|
|
|
This API is in beta mode and will be subject to change in the future.
|
|
|
|
The result_formatter function customizes the output of Cypher retrievers for improved prompt engineering and readability. It converts each Neo4j record into a RetrieverResultItem with two fields: `content` and `metadata`.
|
|
|
|
The `content` field is a formatted string containing the key information intended for the language model, such as movie titles or descriptions. The `metadata` field holds additional details, useful for debugging or providing extra context, like scores or node properties.
|
|
|
|
|
|
.. code:: python
|
|
|
|
def result_formatter(record: neo4j.Record) -> RetrieverResultItem:
|
|
content=f"Movie title: {record.get('movieTitle')}, description: {record.get('movieDescription')}, actors: {record.get('actors')}",
|
|
return RetrieverResultItem(
|
|
metadata={
|
|
"title": record.get('movieTitle'),
|
|
"score": record.get("score"),
|
|
}
|
|
)
|
|
|
|
retriever = VectorCypherRetriever(
|
|
driver,
|
|
index_name=INDEX_NAME,
|
|
retrieval_query="OPTIONAL MATCH (node)<-[:ACTED_IN]-(p:Person) RETURN node.title as movieTitle, node.plot as movieDescription, collect(p.name) as actors, score",
|
|
result_formatter=result_formatter,
|
|
)
|
|
|
|
Also see :ref:`vectorcypherretriever`.
|
|
|
|
|
|
.. _vector-databases-section:
|
|
|
|
Vector Databases
|
|
====================
|
|
|
|
.. note::
|
|
|
|
For external retrievers, the filter syntax depends on the provider. Please refer to
|
|
the documentation of the Python client for each provider for details.
|
|
|
|
.. _weaviate-neo4j-retriever-user-guide:
|
|
|
|
Weaviate Retrievers
|
|
-------------------
|
|
|
|
.. note::
|
|
|
|
In order to import this retriever, the Weaviate Python client must be installed:
|
|
`pip install "neo4j_graphrag[weaviate]"`
|
|
|
|
|
|
.. code:: python
|
|
|
|
from weaviate.connect.helpers import connect_to_local
|
|
from neo4j_graphrag.retrievers import WeaviateNeo4jRetriever
|
|
|
|
client = connect_to_local()
|
|
retriever = WeaviateNeo4jRetriever(
|
|
driver=driver,
|
|
client=client,
|
|
embedder=embedder,
|
|
collection="Movies",
|
|
id_property_external="neo4j_id",
|
|
id_property_neo4j="id",
|
|
node_label_neo4j="Document", # optional
|
|
)
|
|
|
|
Internally, this retriever performs the vector search in Weaviate, finds the corresponding node by matching
|
|
the Weaviate metadata `id_property_external` with a Neo4j `node.id_property_neo4j`, and returns the matched node.
|
|
|
|
The `return_properties` and `retrieval_query` parameters operate similarly to those in other retrievers.
|
|
|
|
See :ref:`weaviateneo4jretriever`.
|
|
|
|
.. _pinecone-neo4j-retriever-user-guide:
|
|
|
|
Pinecone Retrievers
|
|
-------------------
|
|
|
|
.. note::
|
|
|
|
In order to import this retriever, the Pinecone Python client must be installed:
|
|
`pip install "neo4j_graphrag[pinecone]"`
|
|
|
|
|
|
.. code:: python
|
|
|
|
from pinecone import Pinecone
|
|
from neo4j_graphrag.retrievers import PineconeNeo4jRetriever
|
|
|
|
client = Pinecone() # ... create your Pinecone client
|
|
|
|
retriever = PineconeNeo4jRetriever(
|
|
driver=driver,
|
|
client=client,
|
|
index_name="Movies",
|
|
id_property_neo4j="id",
|
|
embedder=embedder,
|
|
node_label_neo4j="Document", # optional
|
|
)
|
|
|
|
Also see :ref:`pineconeneo4jretriever`.
|
|
|
|
.. _qdrant-neo4j-retriever-user-guide:
|
|
|
|
Qdrant Retrievers
|
|
-----------------
|
|
|
|
.. note::
|
|
|
|
In order to import this retriever, the Qdrant Python client must be installed:
|
|
`pip install "neo4j_graphrag[qdrant]"`
|
|
|
|
|
|
.. code:: python
|
|
|
|
from qdrant_client import QdrantClient
|
|
from neo4j_graphrag.retrievers import QdrantNeo4jRetriever
|
|
|
|
client = QdrantClient(...) # construct the Qdrant client instance
|
|
|
|
retriever = QdrantNeo4jRetriever(
|
|
driver=driver,
|
|
client=client,
|
|
collection_name="my-collection",
|
|
using="my-vector",
|
|
id_property_external="neo4j_id", # The payload field that contains identifier to a corresponding Neo4j node id property
|
|
id_property_neo4j="id",
|
|
embedder=embedder,
|
|
node_label_neo4j="Document", # optional
|
|
)
|
|
|
|
See :ref:`qdrantneo4jretriever`.
|
|
|
|
|
|
Other Retrievers
|
|
===================
|
|
|
|
.. _hybrid-retriever-user-guide:
|
|
|
|
Hybrid Retrievers
|
|
------------------------------------
|
|
|
|
In an hybrid retriever, results are searched for in both a vector and a full-text index.
|
|
For this reason, a full-text index must also exist in the database, and its name must
|
|
be provided when instantiating the retriever:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.retrievers import HybridRetriever
|
|
|
|
INDEX_NAME = "embedding-name"
|
|
FULLTEXT_INDEX_NAME = "fulltext-index-name"
|
|
|
|
retriever = HybridRetriever(
|
|
driver,
|
|
INDEX_NAME,
|
|
FULLTEXT_INDEX_NAME,
|
|
embedder,
|
|
)
|
|
|
|
|
|
See :ref:`hybridretriever`.
|
|
|
|
Also note that there is an helper function to create a full-text index (see :ref:`the API documentation<create-fulltext-index>`).
|
|
|
|
.. _hybrid-cypher-retriever-user-guide:
|
|
|
|
Hybrid Cypher Retrievers
|
|
------------------------
|
|
|
|
In an hybrid cypher retriever, results are searched for in both a vector and a
|
|
full-text index. Once the similar nodes are identified, a retrieval query can traverse
|
|
the graph and return more context:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.retrievers import HybridCypherRetriever
|
|
|
|
INDEX_NAME = "embedding-name"
|
|
FULLTEXT_INDEX_NAME = "fulltext-index-name"
|
|
|
|
retriever = HybridCypherRetriever(
|
|
driver,
|
|
INDEX_NAME,
|
|
FULLTEXT_INDEX_NAME,
|
|
retrieval_query="MATCH (node)-[:AUTHORED_BY]->(author:Author) RETURN author.name"
|
|
embedder=embedder,
|
|
)
|
|
|
|
|
|
See :ref:`hybridcypherretriever`.
|
|
|
|
|
|
.. _text2cypher-retriever-user-guide:
|
|
|
|
Text2Cypher Retriever
|
|
---------------------
|
|
|
|
This retriever first asks an LLM to generate a Cypher query to fetch the exact
|
|
information required to answer the question from the database. Then this query is
|
|
executed and the resulting records are added to the context for the LLM to write
|
|
the answer to the initial user question. The cypher-generation and answer-generation
|
|
LLMs can be different.
|
|
|
|
.. code:: python
|
|
|
|
from neo4j import GraphDatabase
|
|
from neo4j_graphrag.retrievers import Text2CypherRetriever
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
# Connect to Neo4j database
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
|
|
# Create LLM object
|
|
llm = OpenAILLM(model_name="gpt-5")
|
|
|
|
# (Optional) Specify your own Neo4j schema
|
|
neo4j_schema = """
|
|
Node properties:
|
|
Person {name: STRING, born: INTEGER}
|
|
Movie {tagline: STRING, title: STRING, released: INTEGER}
|
|
Relationship properties:
|
|
ACTED_IN {roles: LIST}
|
|
REVIEWED {summary: STRING, rating: INTEGER}
|
|
The relationships:
|
|
(:Person)-[:ACTED_IN]->(:Movie)
|
|
(:Person)-[:DIRECTED]->(:Movie)
|
|
(:Person)-[:PRODUCED]->(:Movie)
|
|
(:Person)-[:WROTE]->(:Movie)
|
|
(:Person)-[:FOLLOWS]->(:Person)
|
|
(:Person)-[:REVIEWED]->(:Movie)
|
|
"""
|
|
|
|
# (Optional) Provide user input/query pairs for the LLM to use as examples
|
|
examples = [
|
|
"USER INPUT: 'Which actors starred in the Matrix?' QUERY: MATCH (p:Person)-[:ACTED_IN]->(m:Movie) WHERE m.title = 'The Matrix' RETURN p.name"
|
|
]
|
|
|
|
# Initialize the retriever
|
|
retriever = Text2CypherRetriever(
|
|
driver=driver,
|
|
llm=llm, # type: ignore
|
|
neo4j_schema=neo4j_schema,
|
|
examples=examples,
|
|
)
|
|
|
|
# Generate a Cypher query using the LLM, send it to the Neo4j database, and return the results
|
|
query_text = "Which movies did Hugo Weaving star in?"
|
|
print(retriever.search(query_text=query_text))
|
|
|
|
|
|
.. warning::
|
|
|
|
Using `OpenAILLM` requires the `openai` Python client. You can install it with `pip install "neo4j_graphrag[openai]"`.
|
|
|
|
.. note::
|
|
|
|
Since we are not performing any similarity search (vector index), the Text2Cypher
|
|
retriever does not require any embedder.
|
|
|
|
.. warning::
|
|
|
|
The LLM-generated query is not guaranteed to be syntactically correct. In case it can't be
|
|
executed, a `Text2CypherRetrievalError` is raised.
|
|
|
|
|
|
See :ref:`text2cypherretriever`.
|
|
|
|
.. _tools-retriever-user-guide:
|
|
|
|
Tools Retriever
|
|
---------------
|
|
|
|
The ToolsRetriever uses an LLM to intelligently select and execute appropriate tools based on user queries. This retriever analyzes the user's question using an LLM to determine which tools from a provided set would be most helpful for retrieving relevant information. It can select multiple tools if necessary or none if no tools are appropriate for the query, then combines results from the executed tools with proper attribution. This is particularly useful when different types of information retrieval might be needed for complex queries.
|
|
|
|
.. code-block:: python
|
|
|
|
from neo4j import GraphDatabase
|
|
from neo4j_graphrag.retrievers import ToolsRetriever, VectorRetriever, Text2CypherRetriever
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
from neo4j_graphrag.embeddings import OpenAIEmbeddings
|
|
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
# Connect to Neo4j database
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
|
|
# Create LLM object
|
|
llm = OpenAILLM(model_name="gpt-5")
|
|
|
|
# Create embedder
|
|
embedder = OpenAIEmbeddings(model="text-embedding-3-large")
|
|
|
|
# Create individual retrievers to use as tools
|
|
vector_retriever = VectorRetriever(
|
|
driver=driver,
|
|
index_name="embedding-index",
|
|
embedder=embedder,
|
|
)
|
|
|
|
text2cypher_retriever = Text2CypherRetriever(
|
|
driver=driver,
|
|
llm=llm,
|
|
)
|
|
|
|
# Convert retrievers to tools
|
|
vector_tool = vector_retriever.convert_to_tool(
|
|
name="vector_search",
|
|
description="Search for similar documents using vector similarity",
|
|
)
|
|
|
|
cypher_tool = text2cypher_retriever.convert_to_tool(
|
|
name="cypher_search",
|
|
description="Generate and execute Cypher queries for structured data retrieval",
|
|
)
|
|
|
|
# Initialize ToolsRetriever with the tools
|
|
tools_retriever = ToolsRetriever(
|
|
driver=driver,
|
|
llm=llm,
|
|
tools=[vector_tool, cypher_tool],
|
|
)
|
|
|
|
# Use the retriever - the LLM will automatically select appropriate tools
|
|
result = tools_retriever.search("What movies did Tom Hanks act in and what are their plots?")
|
|
|
|
Tool Creation Patterns
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The ToolsRetriever supports two primary approaches for creating tools:
|
|
|
|
**1. Converting Retrievers to Tools**
|
|
|
|
Any retriever can be converted to a tool using the ``convert_to_tool()`` method:
|
|
|
|
.. code-block:: python
|
|
|
|
# Convert retriever with custom parameter descriptions
|
|
tool = retriever.convert_to_tool(
|
|
name="search_tool",
|
|
description="Custom tool description",
|
|
parameter_descriptions={
|
|
"query_text": "Search query for finding relevant information",
|
|
"top_k": "Maximum number of results to return",
|
|
}
|
|
)
|
|
|
|
The ``convert_to_tool()`` method automatically infers parameters from the retriever's ``get_search_results()`` method signature, mapping Python types to tool parameter types (str → StringParameter, int → IntegerParameter, etc.).
|
|
|
|
**2. Creating Custom Tools**
|
|
|
|
Create custom tools by inheriting from the Tool class:
|
|
|
|
.. code-block:: python
|
|
|
|
from neo4j_graphrag.tool import Tool, ObjectParameter, StringParameter
|
|
|
|
class CustomTool(Tool):
|
|
def __init__(self):
|
|
parameters = ObjectParameter(
|
|
description="Parameters for custom tool",
|
|
properties={
|
|
"input": StringParameter(
|
|
description="Input parameter description",
|
|
),
|
|
},
|
|
required_properties=["input"],
|
|
)
|
|
|
|
super().__init__(
|
|
name="custom_tool",
|
|
description="Tool description for LLM selection",
|
|
parameters=parameters,
|
|
execute_func=self.execute_custom_logic,
|
|
)
|
|
|
|
def execute_custom_logic(self, **kwargs):
|
|
# Custom implementation
|
|
return f"Processed: {kwargs.get('input')}"
|
|
|
|
Tool Configuration
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
**Parameter Types and Validation**
|
|
|
|
Tools support various parameter types with automatic validation:
|
|
|
|
.. code-block:: python
|
|
|
|
from neo4j_graphrag.tool import (
|
|
ObjectParameter, StringParameter, IntegerParameter,
|
|
NumberParameter, BooleanParameter, ArrayParameter
|
|
)
|
|
|
|
parameters = ObjectParameter(
|
|
properties={
|
|
"text": StringParameter(description="Text input"),
|
|
"count": IntegerParameter(description="Count", minimum=1, maximum=100),
|
|
"threshold": NumberParameter(description="Threshold", minimum=0.0, maximum=1.0),
|
|
"enabled": BooleanParameter(description="Enable feature"),
|
|
"items": ArrayParameter(
|
|
description="List of items",
|
|
items=StringParameter(description="String item")
|
|
),
|
|
},
|
|
required_properties=["text", "count"]
|
|
)
|
|
|
|
**System Instruction Customization**
|
|
|
|
Customize how the LLM selects tools by providing a custom system instruction:
|
|
|
|
.. code-block:: python
|
|
|
|
custom_instruction = """
|
|
You are a specialized assistant for movie database queries.
|
|
Select tools based on query type: use vector_search for plot similarity,
|
|
cypher_search for specific actor/director queries, and both for complex requests.
|
|
"""
|
|
|
|
tools_retriever = ToolsRetriever(
|
|
driver=driver,
|
|
llm=llm,
|
|
tools=[vector_tool, cypher_tool],
|
|
system_instruction=custom_instruction,
|
|
)
|
|
|
|
The system instruction guides the LLM's tool selection logic and can be tailored to your specific domain or use case.
|
|
|
|
**Error Handling Patterns**
|
|
|
|
Tools should implement proper error handling for common failure scenarios:
|
|
|
|
.. code-block:: python
|
|
|
|
def execute_custom_logic(self, **kwargs):
|
|
try:
|
|
# Tool parameter validation
|
|
required_param = kwargs.get('required_param')
|
|
if not required_param:
|
|
raise ValueError("Required parameter 'required_param' is missing")
|
|
|
|
# Tool execution logic
|
|
result = perform_operation(required_param)
|
|
return result
|
|
|
|
except ValueError as e:
|
|
# Parameter validation errors
|
|
return f"Parameter error: {str(e)}"
|
|
except Exception as e:
|
|
# Tool execution errors
|
|
return f"Execution error: {str(e)}"
|
|
|
|
The ToolsRetriever handles LLM selection errors internally and will retry tool selection or return appropriate error messages when tools cannot be executed successfully.
|
|
|
|
**Integration with GraphRAG**
|
|
|
|
The ToolsRetriever integrates seamlessly with the GraphRAG pipeline for automated question answering:
|
|
|
|
.. code-block:: python
|
|
|
|
from neo4j import GraphDatabase
|
|
from neo4j_graphrag.retrievers import ToolsRetriever, VectorRetriever, Text2CypherRetriever
|
|
from neo4j_graphrag.llm import OpenAILLM
|
|
from neo4j_graphrag.embeddings import OpenAIEmbeddings
|
|
from neo4j_graphrag.generation import GraphRAG
|
|
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
# Connect to Neo4j database
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
|
|
# Create LLM and embedder
|
|
llm = OpenAILLM(model_name="gpt-5", model_params={"temperature": 0})
|
|
embedder = OpenAIEmbeddings(model="text-embedding-3-large")
|
|
|
|
# Create retrievers to use as tools
|
|
vector_retriever = VectorRetriever(
|
|
driver=driver,
|
|
index_name="vector-index",
|
|
embedder=embedder,
|
|
)
|
|
|
|
text2cypher_retriever = Text2CypherRetriever(
|
|
driver=driver,
|
|
llm=llm,
|
|
)
|
|
|
|
# Convert to tools
|
|
vector_tool = vector_retriever.convert_to_tool(
|
|
name="vector_search",
|
|
description="Search for similar documents using vector similarity",
|
|
)
|
|
|
|
cypher_tool = text2cypher_retriever.convert_to_tool(
|
|
name="cypher_search",
|
|
description="Generate and execute Cypher queries for structured data retrieval",
|
|
)
|
|
|
|
# Initialize ToolsRetriever
|
|
tools_retriever = ToolsRetriever(
|
|
driver=driver,
|
|
llm=llm,
|
|
tools=[vector_tool, cypher_tool],
|
|
)
|
|
|
|
# Initialize GraphRAG pipeline with ToolsRetriever
|
|
rag = GraphRAG(retriever=tools_retriever, llm=llm)
|
|
|
|
# Query the pipeline - the LLM will automatically select appropriate tools
|
|
query_text = "What movies did Tom Hanks act in and what are their plots?"
|
|
response = rag.search(query_text=query_text, retriever_config={"top_k": 5})
|
|
print(response.answer)
|
|
|
|
.. warning::
|
|
|
|
ToolsRetriever requires an LLM instance to function. Using LLM providers like OpenAI requires their respective Python packages to be installed: `pip install "neo4j_graphrag[openai]"`.
|
|
|
|
See :ref:`toolsretriever`.
|
|
|
|
.. _custom-retriever:
|
|
|
|
Custom Retriever
|
|
================
|
|
|
|
If the application requires very specific retrieval strategy, it is possible to implement
|
|
a custom retriever using the `Retriever` interface:
|
|
|
|
.. code:: python
|
|
|
|
from neo4j_graphrag.retrievers.base import Retriever
|
|
|
|
class MyCustomRetriever(Retriever):
|
|
def __init__(
|
|
self,
|
|
driver: neo4j.Driver,
|
|
# any other required parameters
|
|
) -> None:
|
|
super().__init__(driver)
|
|
|
|
def get_search_results(
|
|
self,
|
|
query_vector: Optional[list[float]] = None,
|
|
query_text: Optional[str] = None,
|
|
top_k: int = 5,
|
|
filters: Optional[dict[str, Any]] = None,
|
|
) -> RawSearchResult:
|
|
pass
|
|
|
|
|
|
See :ref:`rawsearchresult` for a description of the returned type.
|
|
|
|
|
|
***********************
|
|
GraphRAG search options
|
|
***********************
|
|
|
|
Return context
|
|
==============
|
|
|
|
By default, the search method only returns the final answer. It is possible to see
|
|
what was retrieved as part of the context by setting `return_context=True`:
|
|
|
|
.. code:: python
|
|
|
|
rag.search("my question", return_context=True)
|
|
|
|
|
|
Return a custom message if context is empty
|
|
===========================================
|
|
|
|
If the retriever is not able to find any context, the LLM will return an answer anyway.
|
|
It is possible to skip the LLM call if the context is empty and return a user-defined message
|
|
instead:
|
|
|
|
.. code:: python
|
|
|
|
rag.search(
|
|
"my question",
|
|
response_fallback="I can not answer this question because I have no relevant context."
|
|
)
|
|
|
|
|
|
Pass configuration to the retriever search method
|
|
=================================================
|
|
|
|
The retrievers search method have a bunch of configuration options (see above),
|
|
which can also be configured through the GraphRAG search method using the `retriever_config`
|
|
argument. For instance, the following code snippet illustrates how to define the `top_k`
|
|
parameter for the retriever:
|
|
|
|
.. code:: python
|
|
|
|
rag.search(
|
|
"my question",
|
|
retriever_config={"top_k": 2}
|
|
)
|
|
|
|
|
|
**************
|
|
DB Operations
|
|
**************
|
|
|
|
See :ref:`database-interaction-section`.
|
|
|
|
Create a Vector Index
|
|
=====================
|
|
|
|
.. code:: python
|
|
|
|
from neo4j import GraphDatabase
|
|
from neo4j_graphrag.indexes import create_vector_index
|
|
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
INDEX_NAME = "chunk-index"
|
|
DIMENSION=1536
|
|
|
|
# Connect to Neo4j database
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
|
|
# Creating the index
|
|
create_vector_index(
|
|
driver,
|
|
INDEX_NAME,
|
|
label="Document",
|
|
embedding_property="vectorProperty",
|
|
dimensions=DIMENSION,
|
|
similarity_fn="euclidean",
|
|
)
|
|
|
|
.. _filterable-index-creation:
|
|
|
|
Create a Vector Index with Filterable Properties
|
|
=================================================
|
|
|
|
On Neo4j 2026.01+, you can create a vector index with filterable properties to enable
|
|
in-index filtering via the ``SEARCH`` clause. This avoids brute-force scanning and
|
|
provides significantly better performance for filtered vector searches.
|
|
|
|
.. code:: python
|
|
|
|
from neo4j import GraphDatabase
|
|
from neo4j_graphrag.indexes import create_vector_index
|
|
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
INDEX_NAME = "chunk-index"
|
|
DIMENSION = 1536
|
|
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
|
|
# Create index with filterable properties for in-index filtering
|
|
create_vector_index(
|
|
driver,
|
|
INDEX_NAME,
|
|
label="Document",
|
|
embedding_property="vectorProperty",
|
|
dimensions=DIMENSION,
|
|
similarity_fn="cosine",
|
|
filterable_properties=["year", "category"],
|
|
)
|
|
|
|
The ``filterable_properties`` parameter accepts a list of node property names. These
|
|
properties will be indexed alongside the vector embeddings, enabling the ``SEARCH`` clause
|
|
to filter results within the index itself.
|
|
|
|
.. note::
|
|
|
|
The ``filterable_properties`` parameter requires Neo4j 2026.01+. On older versions,
|
|
the parameter is accepted but the generated ``WITH`` clause may cause an error if the
|
|
server does not support it.
|
|
|
|
|
|
Populate a Vector Index
|
|
=======================
|
|
|
|
.. code:: python
|
|
|
|
from random import random
|
|
|
|
from neo4j import GraphDatabase
|
|
from neo4j_graphrag.indexes import upsert_vectors
|
|
from neo4j_graphrag.types import EntityType
|
|
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
# Connect to Neo4j database
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
|
|
# Upsert the vector
|
|
DIMENSION = 1536
|
|
vector = [random() for _ in range(DIMENSION)]
|
|
upsert_vectors(
|
|
driver,
|
|
ids=["1234"],
|
|
embedding_property="vectorProperty",
|
|
embeddings=[vector],
|
|
entity_type=EntityType.NODE,
|
|
)
|
|
|
|
This will update the node with `id(node)=1234` to add (or update) a `node.vectorProperty` property.
|
|
This property will also be added to the vector index.
|
|
|
|
|
|
Drop a Vector Index
|
|
===================
|
|
|
|
.. warning::
|
|
|
|
This operation is irreversible and should be used with caution.
|
|
|
|
|
|
.. code:: python
|
|
|
|
from neo4j import GraphDatabase
|
|
|
|
URI = "neo4j://localhost:7687"
|
|
AUTH = ("neo4j", "password")
|
|
|
|
# Connect to Neo4j database
|
|
driver = GraphDatabase.driver(URI, auth=AUTH)
|
|
drop_index_if_exists(driver, INDEX_NAME)
|