.. _user-guide-rag: User Guide: RAG ############### This guide provides a starting point for using the Neo4j GraphRAG package and configuring it according to specific requirements. ********** Quickstart ********** To perform a GraphRAG query using the `neo4j-graphrag` package, a few components are needed: 1. A Neo4j driver: used to query your Neo4j database. 2. A Retriever: the `neo4j-graphrag` package provides some implementations (see the :ref:`dedicated section `) and lets you write your own if none of the provided implementations matches your needs (see :ref:`how to write a custom retriever `). 3. An LLM: to generate the answer, we need to call an LLM model. The neo4j-graphrag package's LLM interface is compatible with LangChain. Developers can also write their own interface if needed. In practice, it's done with only a few lines of code: .. code:: python from neo4j import GraphDatabase from neo4j_graphrag.retrievers import VectorRetriever from neo4j_graphrag.llm import OpenAILLM from neo4j_graphrag.generation import GraphRAG from neo4j_graphrag.embeddings import OpenAIEmbeddings # 1. Neo4j driver URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") INDEX_NAME = "index-name" # Connect to Neo4j database driver = GraphDatabase.driver(URI, auth=AUTH) # 2. Retriever # Create Embedder object, needed to convert the user question (text) to a vector embedder = OpenAIEmbeddings(model="text-embedding-3-large") # Initialize the retriever retriever = VectorRetriever(driver, INDEX_NAME, embedder) # 3. LLM # Note: the OPENAI_API_KEY must be in the env vars llm = OpenAILLM(model_name="gpt-5", model_params={"temperature": 0}) # Initialize the RAG pipeline rag = GraphRAG(retriever=retriever, llm=llm) # Query the graph query_text = "How do I do similarity search in Neo4j?" response = rag.search(query_text=query_text, retriever_config={"top_k": 5}) print(response.answer) .. warning:: Using `OpenAILLM` requires the `openai` Python client. You can install it with `pip install "neo4j_graphrag[openai]"`. The following sections provide more details about how to customize this code. ********************** GraphRAG Configuration ********************** Each component can be configured individually: the LLM and the prompt. Using Another LLM Model ======================== If OpenAI cannot be used directly, there are a few available alternatives: - Use Azure OpenAI (GPT...). - Use Google VertexAI (Gemini...). - Use Anthropic LLM (Claude...). - Use Mistral LLM - Use Cohere. - Use a local Ollama model. - Implement a custom interface. - Utilize any LangChain chat model. All options are illustrated below. Using Azure Open AI LLM ----------------------- It is possible to use Azure OpenAI switching to the `AzureOpenAILLM` class: .. code:: python from neo4j_graphrag.llm import AzureOpenAILLM llm = AzureOpenAILLM( model_name="gpt-5", azure_endpoint="https://example-endpoint.openai.azure.com/", # update with your endpoint api_version="2024-06-01", # update appropriate version api_key="...", # api_key is optional and can also be set with OPENAI_API_KEY env var ) llm.invoke("say something") Check the OpenAI Python client `documentation `_. to learn more about the configuration. .. note:: In order to run this code, the `openai` Python package needs to be installed: `pip install "neo4j_graphrag[openai]"` See :ref:`azureopenaillm`. Using VertexAI LLM ------------------ To use VertexAI, instantiate the `VertexAILLM` class: .. code:: python from neo4j_graphrag.llm import VertexAILLM from vertexai.generative_models import GenerationConfig generation_config = GenerationConfig(temperature=0.0) llm = VertexAILLM( model_name="gemini-2.5-flash", generation_config=generation_config ) llm.invoke("say something") .. note:: In order to run this code, the `google-cloud-aiplatform` Python package needs to be installed: `pip install "neo4j_graphrag[google]"` See :ref:`vertexaillm`. Using Anthropic LLM ------------------- To use Anthropic, instantiate the `AnthropicLLM` class: .. code:: python from neo4j_graphrag.llm import AnthropicLLM llm = AnthropicLLM( model_name="claude-3-opus-20240229", model_params={"max_tokens": 1000}, # max_tokens must be specified api_key=api_key, # can also set `ANTHROPIC_API_KEY` in env vars ) llm.invoke("say something") .. note:: In order to run this code, the `anthropic` Python package needs to be installed: `pip install "neo4j_graphrag[anthropic]"` See :ref:`anthropicllm`. Using MistralAI LLM ------------------- To use MistralAI, instantiate the `MistralAILLM` class: .. code:: python from neo4j_graphrag.llm import MistralAILLM llm = MistralAILLM( model_name="mistral-small-latest", api_key=api_key, # can also set `MISTRAL_API_KEY` in env vars ) llm.invoke("say something") .. note:: In order to run this code, the `mistralai` Python package needs to be installed: `pip install "neo4j_graphrag[mistralai]"` See :ref:`mistralaillm`. Using Cohere LLM ---------------- To use Cohere, instantiate the `CohereLLM` class: .. code:: python from neo4j_graphrag.llm import CohereLLM llm = CohereLLM( model_name="command-r", api_key=api_key, # can also set `CO_API_KEY` in env vars ) llm.invoke("say something") .. note:: In order to run this code, the `cohere` Python package needs to be installed: `pip install "neo4j_graphrag[cohere]"` See :ref:`coherellm`. Using a Local Model via Ollama ------------------------------- Assuming Ollama is running on the default address `127.0.0.1:11434`, it can be queried using the following: .. code:: python from neo4j_graphrag.llm import OllamaLLM llm = OllamaLLM( model_name="orca-mini", # model_params={"options": {"temperature": 0}, "format": "json"}, # host="...", # when using a remote server ) llm.invoke("say something") Using a Model from LangChain ----------------------------- The LangChain Python package contains implementations for various LLMs and providers. Its interface is compatible with our `GraphRAG` interface, facilitating integration: .. code:: python from neo4j_graphrag.generation import GraphRAG from langchain_community.chat_models import ChatOllama # retriever = ... llm = ChatOllama(model="llama3:8b") rag = GraphRAG(retriever=retriever, llm=llm) query_text = "How do I do similarity search in Neo4j?" response = rag.search(query_text=query_text, retriever_config={"top_k": 5}) print(response.answer) It is however not mandatory to use LangChain. Using a Custom Model -------------------- If the provided implementations do not match their needs, developers can create a custom LLM class by subclassing :class:`neo4j_graphrag.llm.LLMBase`. ``LLMBase`` combines ``LLMInterface`` (str input, v1 API) and ``LLMInterfaceV2`` (``List[LLMMessage]`` input, structured output) into a single abstract base class. Subclasses implement one ``invoke`` and one ``ainvoke`` dispatcher that branches on the type of *input*. Here's an example using the Python Ollama client: .. code:: python from typing import Any, List, Optional, Type, Union import ollama from pydantic import BaseModel from neo4j_graphrag.llm import LLMBase, LLMResponse from neo4j_graphrag.message_history import MessageHistory from neo4j_graphrag.types import LLMMessage class MyOllamaLLM(LLMBase): def invoke( self, input: Union[str, List[LLMMessage]], message_history=None, system_instruction=None, response_format=None, **kwargs: Any, ) -> LLMResponse: if isinstance(input, str): messages = [{"role": "user", "content": input}] else: messages = list(input) response = ollama.chat(model=self.model_name, messages=messages) return LLMResponse(content=response["message"]["content"]) async def ainvoke( self, input: Union[str, List[LLMMessage]], message_history=None, system_instruction=None, response_format=None, **kwargs: Any, ) -> LLMResponse: return self.invoke(input) # TODO: implement with ollama.AsyncClient # retriever = ... llm = MyOllamaLLM("llama3:8b") rag = GraphRAG(retriever=retriever, llm=llm) query_text = "How do I do similarity search in Neo4j?" response = rag.search(query_text=query_text, retriever_config={"top_k": 5}) print(response.answer) See :ref:`llminterface`. Structured Output with LLMs ============================ Structured output enables LLMs to return responses conforming to a predefined schema (Pydantic model or JSON schema), ensuring type-safe and consistent data structures. This is useful for extracting entities, relationships, or any structured data with automatic validation. **V2 Interface (Recommended)**: For :ref:`OpenAILLM ` and :ref:`VertexAILLM `, pass `response_format` as a parameter to the `invoke()` method when using the V2 interface (list of `LLMMessage`). The `response_format` accepts either a Pydantic model class or a JSON schema dictionary. Other LLM providers will raise `NotImplementedError` if `response_format` is used. **V1 Interface (Legacy)**: With the V1 interface (string input), standard JSON mode is supported for both OpenAI and VertexAI via constructor parameters only (`model_params` for OpenAI, `generation_config` for VertexAI). The `response_format` parameter in `invoke()` is not permitted with V1. .. code:: python from pydantic import BaseModel, ConfigDict from neo4j_graphrag.llm import OpenAILLM from neo4j_graphrag.types import LLMMessage class Person(BaseModel): model_config = ConfigDict(extra="forbid") # Required for OpenAI structured output name: str age: int occupation: str llm = OpenAILLM(model_name="gpt-5-mini") # V2: Pass response_format to invoke() messages = [LLMMessage(role="user", content="Extract: John is a 30 year old engineer.")] response = llm.invoke(messages, response_format=Person, temperature=0) person = Person.model_validate_json(response.content) # {"name": "John", "age": 30, ...} # V1: Use constructor parameters for standard JSON mode llm_v1 = OpenAILLM( model_name="gpt-5-mini", model_params={"response_format": {"type": "json_object"}, "temperature": 0} ) response_v1 = llm_v1.invoke("Extract person in JSON format: John is 30 years old.") OpenAI Structured Output ------------------------- OpenAI supports Pydantic models, JSON schemas, and JSON object mode. **Important**: Pydantic models must include `ConfigDict(extra="forbid")` to generate schemas with `additionalProperties: false`, which is required by OpenAI's strict mode. .. code:: python from neo4j_graphrag.llm import OpenAILLM from neo4j_graphrag.types import LLMMessage llm = OpenAILLM(model_name="gpt-5-mini") # JSON schema json_schema = { "type": "object", "properties": {"name": {"type": "string"}, "age": {"type": "integer"}}, "required": ["name", "age"] } messages = [LLMMessage(role="user", content="Extract: John is 30.")] response = llm.invoke(messages, response_format=json_schema, temperature=0) # JSON object mode (no schema enforcement) response = llm.invoke(messages, response_format={"type": "json_object"}, temperature=0.5) VertexAI Structured Output --------------------------- VertexAI uses `GenerationConfig` with `response_mime_type` and `response_schema` internally when `response_format` is passed to `invoke()`. Both Pydantic models and JSON schemas are supported. **Important**: Additional `GenerationConfig` parameters (e.g., `temperature`, `max_output_tokens`) can be passed as kwargs to `invoke()`. .. code:: python from pydantic import BaseModel, ConfigDict from neo4j_graphrag.llm import VertexAILLM from neo4j_graphrag.types import LLMMessage class Person(BaseModel): model_config = ConfigDict(extra="forbid") name: str age: int llm = VertexAILLM(model_name="gemini-1.5-pro") messages = [LLMMessage(role="user", content="Extract: John is 30.")] response = llm.invoke(messages, response_format=Person, temperature=0) person = Person.model_validate_json(response.content) Rate Limit Handling =================== All LLM implementations include automatic rate limiting that uses retry logic with exponential backoff by default. This feature helps handle API rate limits from LLM providers gracefully by automatically retrying failed requests with increasing wait times between attempts. Default Rate Limit Handler -------------------------- Rate limiting is enabled by default for all LLM instances with the following configuration: - **Max attempts**: 3 - **Min wait**: 1.0 seconds - **Max wait**: 60.0 seconds - **Multiplier**: 2.0 (exponential backoff) .. code:: python from neo4j_graphrag.llm import OpenAILLM # Rate limiting is automatically enabled llm = OpenAILLM(model_name="gpt-5") # The LLM will automatically retry on rate limit errors response = llm.invoke("Hello, world!") .. note:: To change the default configuration of `RetryRateLimitHandler`: .. code:: python from neo4j_graphrag.llm import OpenAILLM from neo4j_graphrag.utils.rate_limit import RetryRateLimitHandler # Customize rate limiting parameters llm = OpenAILLM( model_name="gpt-5", rate_limit_handler=RetryRateLimitHandler( max_attempts=10, # Increase max retry attempts min_wait=2.0, # Increase minimum wait time max_wait=120.0, # Increase maximum wait time multiplier=3.0 # More aggressive backoff ) ) Custom Rate Limiting -------------------- You can customize the rate limiting behavior by creating your own rate limit handler: .. code:: python from neo4j_graphrag.llm import AnthropicLLM from neo4j_graphrag.utils.rate_limit import RateLimitHandler class CustomRateLimitHandler(RateLimitHandler): """Implement your custom rate limiting strategy.""" # Implement required methods: handle_sync, handle_async # and optionally override is_retryable_exception and to_retryable_error # to classify additional exception types as retryable or convert them to retryable errors pass # Create custom rate limit handler and pass it to the LLM interface custom_handler = CustomRateLimitHandler() llm = AnthropicLLM( model_name="claude-3-sonnet-20240229", rate_limit_handler=custom_handler, ) Disabling Rate Limiting ----------------------- For high-throughput applications or when you handle rate limiting externally, you can disable it: .. code:: python from neo4j_graphrag.llm import CohereLLM, NoOpRateLimitHandler # Disable rate limiting completely llm = CohereLLM( model_name="command-r-plus", rate_limit_handler=NoOpRateLimitHandler(), ) llm.invoke("Hello, world!") Configuring the Prompt ======================== Prompts are managed through `PromptTemplate` classes. Specifically, the `GraphRAG` pipeline utilizes a `RagTemplate` with a default prompt that can be accessed through `rag.prompt_template.template`. To use a different prompt, subclass the `RagTemplate` class and pass it to the `GraphRAG` pipeline object during initialization: .. code:: python from neo4j_graphrag.generation import RagTemplate, GraphRAG # retriever = ... # llm = ... prompt_template = RagTemplate( prompt="Answer the question {question} using context {context} and examples {examples}", expected_inputs=["context", "question", "examples"] ) rag = GraphRAG(retriever=retriever, llm=llm, prompt_template=prompt_template) # ... See :ref:`prompttemplate`. The final configurable component in the `GraphRAG` pipeline is the retriever. Below are descriptions of the various options available. .. _retriever-configuration: ************************ Retriever Configuration ************************ We provide implementations for the following retrievers: .. list-table:: List of retrievers :widths: 30 100 :header-rows: 1 * - Retriever - Description * - :ref:`VectorRetriever ` - Performs a similarity search based on a Neo4j vector index and a query text or vector. Returns the matched `node` and similarity `score`. * - :ref:`VectorCypherRetriever ` - Performs a similarity search based on a Neo4j vector index and a query text or vector. The returned results can be configured through a retrieval query parameter that will be executed after the index search. It can be used to fetch more context around the matched node. * - :ref:`HybridRetriever ` - Uses both a vector and a full-text index in Neo4j. * - :ref:`HybridCypherRetriever ` - Same as HybridRetriever with a retrieval query similar to VectorCypherRetriever. * - :ref:`ToolsRetriever ` - Uses an LLM to intelligently select and execute appropriate tools based on user queries. Combines results from multiple tools with proper attribution. * - :ref:`Text2Cypher ` - Translates the user question into a Cypher query to be run against a Neo4j database (or Knowledge Graph). The results of the query are then passed to the LLM to generate the final answer. * - :ref:`WeaviateNeo4jRetriever ` - Use this retriever when vectors are saved in a Weaviate vector database * - :ref:`PineconeNeo4jRetriever ` - Use this retriever when vectors are saved in a Pinecone vector database * - :ref:`QdrantNeo4jRetriever ` - Use this retriever when vectors are saved in a Qdrant vector database Retrievers all expose a `search` method that we will discuss in the next sections. .. _vector-retriever-user-guide: Vector Retriever =================== The simplest method to instantiate a vector retriever is: .. code:: python from neo4j_graphrag.retrievers import VectorRetriever retriever = VectorRetriever( driver, index_name=POSTER_INDEX_NAME, ) The `index_name` is the name of the Neo4j vector index that will be used for similarity search. .. warning:: Vector index use an **approximate nearest neighbor** algorithm. Refer to the `Neo4j Documentation `_ to learn about its limitations. Search Similar Vector ----------------------------- To identify the top 3 most similar nodes, perform a search by vector: .. code:: python vector = [] # a list of floats, same size as the vectors in the Neo4j vector index retriever_result = retriever.search(query_vector=vector, top_k=3) However, in most cases, a text (from the user) will be provided instead of a vector. In this scenario, an `Embedder` is required. Search Similar Text -------------------- When searching for a text, specifying how the retriever transforms (embeds) the text into a vector is required. Therefore, the retriever requires knowledge of an embedder: .. code:: python embedder = OpenAIEmbeddings(model="text-embedding-3-large") # Initialize the retriever retriever = VectorRetriever( driver, index_name=POSTER_INDEX_NAME, embedder=embedder, ) query_text = "How do I do similarity search in Neo4j?" retriever_result = retriever.search(query_text=query_text, top_k=3) Embedders --------- Currently, this package supports the following embedders: - :ref:`openaiembeddings` - :ref:`sentencetransformerembeddings` - :ref:`vertexaiembeddings` - :ref:`mistralaiembeddings` - :ref:`cohereembeddings` - :ref:`azureopenaiembeddings` - :ref:`ollamaembeddings` The `OpenAIEmbeddings` was illustrated previously. Here is how to use the `SentenceTransformerEmbeddings`: .. code:: python from neo4j_graphrag.embeddings import SentenceTransformerEmbeddings embedder = SentenceTransformerEmbeddings(model="all-MiniLM-L6-v2") # Note: this is the default model If another embedder is desired, a custom embedder can be created, using the `Embedder` interface. Embedder Rate Limiting ---------------------- All embedder implementations include automatic rate limiting that uses retry logic with exponential backoff by default, similar to LLM implementations. This feature helps handle API rate limits from embedding providers gracefully. .. code:: python from neo4j_graphrag.embeddings import OpenAIEmbeddings from neo4j_graphrag.utils.rate_limit import RetryRateLimitHandler, NoOpRateLimitHandler # Default rate limiting (automatically enabled) embedder = OpenAIEmbeddings(model="text-embedding-3-large") # Custom rate limiting configuration embedder = OpenAIEmbeddings( model="text-embedding-3-large", rate_limit_handler=RetryRateLimitHandler( max_attempts=5, min_wait=2.0, max_wait=120.0 ) ) # Disable rate limiting embedder = OpenAIEmbeddings( model="text-embedding-3-large", rate_limit_handler=NoOpRateLimitHandler() ) The rate limiting configuration works the same way as for LLMs. See the :ref:`Rate Limit Handling ` section above for more details on customization options. Other Vector Retriever Configuration ---------------------------------------- Often, not all node properties are pertinent for the RAG context; only a selected few are relevant for inclusion in the LLM prompt context. You can specify which properties to return using the `return_properties` parameter: .. code:: python from neo4j_graphrag.retrievers import VectorRetriever retriever = VectorRetriever( driver, index_name=POSTER_INDEX_NAME, embedder=embedder, return_properties=["title"], ) Pre-Filters ----------- When performing a similarity search, one may have constraints to apply. For instance, filtering out movies released before 2000. This can be achieved using `filters`. .. note:: Filters are implemented for all retrievers except the Hybrid retrievers. The documentation below is not valid for external retrievers, which use their own filter syntax (see :ref:`vector-databases-section`). .. code:: python from neo4j_graphrag.retrievers import VectorRetriever retriever = VectorRetriever( driver, index_name=POSTER_INDEX_NAME, ) filters = { "year": { "$gte": 2000, } } query_text = "How do I do similarity search in Neo4j?" retriever_result = retriever.search(query_text=query_text, filters=filters) .. warning:: On Neo4j versions prior to 2026.01, filters cause the similarity search to bypass the vector index and use a brute-force exact match algorithm. Ensure that the pre-filtering is stringent enough to prevent query overload. On Neo4j 2026.01+, simple filters (see below) are automatically routed to use the SEARCH clause with in-index filtering, which is significantly faster. See :ref:`search-clause-filtering` for details. The currently supported operators are: - `$eq`: equal. - `$ne`: not equal. - `$lt`: less than. - `$lte`: less than or equal to. - `$gt`: greater than. - `$gte`: greater than or equal to. - `$between`: between. - `$in`: value is in a given list. - `$nin`: not in. - `$like`: LIKE operator case-sensitive. - `$ilike`: LIKE operator case-insensitive. .. _search-clause-filtering: In-Index Filtering with the SEARCH Clause (Neo4j 2026.01+) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On Neo4j 2026.01 and later, the library automatically detects the server version and uses the Cypher ``SEARCH`` clause for vector queries. This enables in-index filtering, where compatible filter predicates are evaluated inside the vector index itself rather than via post-hoc brute-force scanning. **Requirements:** 1. Neo4j server version 2026.01 or later. 2. To filter, the vector index must declare every filtered property in ``filterable_properties`` (see :ref:`filterable-index-creation`). If any filtered property is missing, the library falls back to the procedure path and logs a warning. Unfiltered queries do not require ``filterable_properties``. 3. The filter must use only SEARCH-compatible operators. **SEARCH-compatible operators** (use in-index filtering): - ``$eq``, ``$ne`` - ``$lt``, ``$lte``, ``$gt``, ``$gte`` - ``$between`` - Multiple filters combined with implicit ``$and`` **Operators that fall back to brute-force filtering:** - ``$or`` - ``$in``, ``$nin`` - ``$like``, ``$ilike`` When a filter uses incompatible operators on a SEARCH-capable server, the library automatically falls back to the procedure-based search path and logs a warning. .. note:: The SEARCH clause routing is fully automatic. No code changes are needed beyond creating the index with ``filterable_properties``. The same ``filters`` parameter works on all Neo4j versions. Here are examples of valid filter syntaxes and their meaning: .. list-table:: Filters syntax :widths: 80 80 :header-rows: 1 * - Filter - Meaning * - {"year": 1999} - year = 1999 * - {"year": {"$eq": 1999}} - year = 1999 * - {"year": 2000, "title": "The Matrix"} - year = 1999 AND title = "The Matrix" * - {"$and": [{"year": 2000}, {"title": "The Matrix"}]} - year = 1999 AND title = "The Matrix" * - {"$or": [{"title": "The Matrix Revolution"}, {"title": "The Matrix"}]} - title = "The Matrix" OR title = "The Matrix Revolution" * - {"title": {"$like": "The Matrix"}} - title CONTAINS "The Matrix" * - {"title": {"$ilike": "the matrix"}} - toLower(title) CONTAINS "The Matrix" See also :ref:`vectorretriever`. .. _vector-cypher-retriever-user-guide: Vector Cypher Retriever ======================= The `VectorCypherRetriever` fully leverages Neo4j's graph capabilities by combining vector-based similarity searches with graph traversal techniques. It processes a query embedding to perform a similarity search against a specified vector index, retrieves relevant node variables, and then executes a Cypher query to traverse the graph based on these nodes. This integration ensures that retrievals are both semantically meaningful and contextually enriched by the underlying graph structure. Retrieval Query --------------- When crafting the retrieval query, it's important to note two available variables are in the query scope: - `node`: represents the node retrieved from the vector index search. - `score`: denotes the similarity score. For instance, in a movie graph with actors where the vector index pertains to certain movie properties, the retrieval query can be structured as follows: .. code:: python retrieval_query = """ RETURN node.title as movieTitle, node.plot as moviePlot, collect { MATCH (actor:Actor)-[:ACTED_IN]->(node) RETURN actor.name } AS actors """ retriever = VectorCypherRetriever( driver, index_name=INDEX_NAME, retrieval_query=retrieval_query, ) It is recommended that the retrieval query returns node properties, as opposed to nodes. Format the Results ------------------ .. warning:: This API is in beta mode and will be subject to change in the future. The result_formatter function customizes the output of Cypher retrievers for improved prompt engineering and readability. It converts each Neo4j record into a RetrieverResultItem with two fields: `content` and `metadata`. The `content` field is a formatted string containing the key information intended for the language model, such as movie titles or descriptions. The `metadata` field holds additional details, useful for debugging or providing extra context, like scores or node properties. .. code:: python def result_formatter(record: neo4j.Record) -> RetrieverResultItem: content=f"Movie title: {record.get('movieTitle')}, description: {record.get('movieDescription')}, actors: {record.get('actors')}", return RetrieverResultItem( metadata={ "title": record.get('movieTitle'), "score": record.get("score"), } ) retriever = VectorCypherRetriever( driver, index_name=INDEX_NAME, retrieval_query="OPTIONAL MATCH (node)<-[:ACTED_IN]-(p:Person) RETURN node.title as movieTitle, node.plot as movieDescription, collect(p.name) as actors, score", result_formatter=result_formatter, ) Also see :ref:`vectorcypherretriever`. .. _vector-databases-section: Vector Databases ==================== .. note:: For external retrievers, the filter syntax depends on the provider. Please refer to the documentation of the Python client for each provider for details. .. _weaviate-neo4j-retriever-user-guide: Weaviate Retrievers ------------------- .. note:: In order to import this retriever, the Weaviate Python client must be installed: `pip install "neo4j_graphrag[weaviate]"` .. code:: python from weaviate.connect.helpers import connect_to_local from neo4j_graphrag.retrievers import WeaviateNeo4jRetriever client = connect_to_local() retriever = WeaviateNeo4jRetriever( driver=driver, client=client, embedder=embedder, collection="Movies", id_property_external="neo4j_id", id_property_neo4j="id", node_label_neo4j="Document", # optional ) Internally, this retriever performs the vector search in Weaviate, finds the corresponding node by matching the Weaviate metadata `id_property_external` with a Neo4j `node.id_property_neo4j`, and returns the matched node. The `return_properties` and `retrieval_query` parameters operate similarly to those in other retrievers. See :ref:`weaviateneo4jretriever`. .. _pinecone-neo4j-retriever-user-guide: Pinecone Retrievers ------------------- .. note:: In order to import this retriever, the Pinecone Python client must be installed: `pip install "neo4j_graphrag[pinecone]"` .. code:: python from pinecone import Pinecone from neo4j_graphrag.retrievers import PineconeNeo4jRetriever client = Pinecone() # ... create your Pinecone client retriever = PineconeNeo4jRetriever( driver=driver, client=client, index_name="Movies", id_property_neo4j="id", embedder=embedder, node_label_neo4j="Document", # optional ) Also see :ref:`pineconeneo4jretriever`. .. _qdrant-neo4j-retriever-user-guide: Qdrant Retrievers ----------------- .. note:: In order to import this retriever, the Qdrant Python client must be installed: `pip install "neo4j_graphrag[qdrant]"` .. code:: python from qdrant_client import QdrantClient from neo4j_graphrag.retrievers import QdrantNeo4jRetriever client = QdrantClient(...) # construct the Qdrant client instance retriever = QdrantNeo4jRetriever( driver=driver, client=client, collection_name="my-collection", using="my-vector", id_property_external="neo4j_id", # The payload field that contains identifier to a corresponding Neo4j node id property id_property_neo4j="id", embedder=embedder, node_label_neo4j="Document", # optional ) See :ref:`qdrantneo4jretriever`. Other Retrievers =================== .. _hybrid-retriever-user-guide: Hybrid Retrievers ------------------------------------ In an hybrid retriever, results are searched for in both a vector and a full-text index. For this reason, a full-text index must also exist in the database, and its name must be provided when instantiating the retriever: .. code:: python from neo4j_graphrag.retrievers import HybridRetriever INDEX_NAME = "embedding-name" FULLTEXT_INDEX_NAME = "fulltext-index-name" retriever = HybridRetriever( driver, INDEX_NAME, FULLTEXT_INDEX_NAME, embedder, ) See :ref:`hybridretriever`. Also note that there is an helper function to create a full-text index (see :ref:`the API documentation`). .. _hybrid-cypher-retriever-user-guide: Hybrid Cypher Retrievers ------------------------ In an hybrid cypher retriever, results are searched for in both a vector and a full-text index. Once the similar nodes are identified, a retrieval query can traverse the graph and return more context: .. code:: python from neo4j_graphrag.retrievers import HybridCypherRetriever INDEX_NAME = "embedding-name" FULLTEXT_INDEX_NAME = "fulltext-index-name" retriever = HybridCypherRetriever( driver, INDEX_NAME, FULLTEXT_INDEX_NAME, retrieval_query="MATCH (node)-[:AUTHORED_BY]->(author:Author) RETURN author.name" embedder=embedder, ) See :ref:`hybridcypherretriever`. .. _text2cypher-retriever-user-guide: Text2Cypher Retriever --------------------- This retriever first asks an LLM to generate a Cypher query to fetch the exact information required to answer the question from the database. Then this query is executed and the resulting records are added to the context for the LLM to write the answer to the initial user question. The cypher-generation and answer-generation LLMs can be different. .. code:: python from neo4j import GraphDatabase from neo4j_graphrag.retrievers import Text2CypherRetriever from neo4j_graphrag.llm import OpenAILLM URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") # Connect to Neo4j database driver = GraphDatabase.driver(URI, auth=AUTH) # Create LLM object llm = OpenAILLM(model_name="gpt-5") # (Optional) Specify your own Neo4j schema neo4j_schema = """ Node properties: Person {name: STRING, born: INTEGER} Movie {tagline: STRING, title: STRING, released: INTEGER} Relationship properties: ACTED_IN {roles: LIST} REVIEWED {summary: STRING, rating: INTEGER} The relationships: (:Person)-[:ACTED_IN]->(:Movie) (:Person)-[:DIRECTED]->(:Movie) (:Person)-[:PRODUCED]->(:Movie) (:Person)-[:WROTE]->(:Movie) (:Person)-[:FOLLOWS]->(:Person) (:Person)-[:REVIEWED]->(:Movie) """ # (Optional) Provide user input/query pairs for the LLM to use as examples examples = [ "USER INPUT: 'Which actors starred in the Matrix?' QUERY: MATCH (p:Person)-[:ACTED_IN]->(m:Movie) WHERE m.title = 'The Matrix' RETURN p.name" ] # Initialize the retriever retriever = Text2CypherRetriever( driver=driver, llm=llm, # type: ignore neo4j_schema=neo4j_schema, examples=examples, ) # Generate a Cypher query using the LLM, send it to the Neo4j database, and return the results query_text = "Which movies did Hugo Weaving star in?" print(retriever.search(query_text=query_text)) .. warning:: Using `OpenAILLM` requires the `openai` Python client. You can install it with `pip install "neo4j_graphrag[openai]"`. .. note:: Since we are not performing any similarity search (vector index), the Text2Cypher retriever does not require any embedder. .. warning:: The LLM-generated query is not guaranteed to be syntactically correct. In case it can't be executed, a `Text2CypherRetrievalError` is raised. See :ref:`text2cypherretriever`. .. _tools-retriever-user-guide: Tools Retriever --------------- The ToolsRetriever uses an LLM to intelligently select and execute appropriate tools based on user queries. This retriever analyzes the user's question using an LLM to determine which tools from a provided set would be most helpful for retrieving relevant information. It can select multiple tools if necessary or none if no tools are appropriate for the query, then combines results from the executed tools with proper attribution. This is particularly useful when different types of information retrieval might be needed for complex queries. .. code-block:: python from neo4j import GraphDatabase from neo4j_graphrag.retrievers import ToolsRetriever, VectorRetriever, Text2CypherRetriever from neo4j_graphrag.llm import OpenAILLM from neo4j_graphrag.embeddings import OpenAIEmbeddings URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") # Connect to Neo4j database driver = GraphDatabase.driver(URI, auth=AUTH) # Create LLM object llm = OpenAILLM(model_name="gpt-5") # Create embedder embedder = OpenAIEmbeddings(model="text-embedding-3-large") # Create individual retrievers to use as tools vector_retriever = VectorRetriever( driver=driver, index_name="embedding-index", embedder=embedder, ) text2cypher_retriever = Text2CypherRetriever( driver=driver, llm=llm, ) # Convert retrievers to tools vector_tool = vector_retriever.convert_to_tool( name="vector_search", description="Search for similar documents using vector similarity", ) cypher_tool = text2cypher_retriever.convert_to_tool( name="cypher_search", description="Generate and execute Cypher queries for structured data retrieval", ) # Initialize ToolsRetriever with the tools tools_retriever = ToolsRetriever( driver=driver, llm=llm, tools=[vector_tool, cypher_tool], ) # Use the retriever - the LLM will automatically select appropriate tools result = tools_retriever.search("What movies did Tom Hanks act in and what are their plots?") Tool Creation Patterns ~~~~~~~~~~~~~~~~~~~~~~ The ToolsRetriever supports two primary approaches for creating tools: **1. Converting Retrievers to Tools** Any retriever can be converted to a tool using the ``convert_to_tool()`` method: .. code-block:: python # Convert retriever with custom parameter descriptions tool = retriever.convert_to_tool( name="search_tool", description="Custom tool description", parameter_descriptions={ "query_text": "Search query for finding relevant information", "top_k": "Maximum number of results to return", } ) The ``convert_to_tool()`` method automatically infers parameters from the retriever's ``get_search_results()`` method signature, mapping Python types to tool parameter types (str → StringParameter, int → IntegerParameter, etc.). **2. Creating Custom Tools** Create custom tools by inheriting from the Tool class: .. code-block:: python from neo4j_graphrag.tool import Tool, ObjectParameter, StringParameter class CustomTool(Tool): def __init__(self): parameters = ObjectParameter( description="Parameters for custom tool", properties={ "input": StringParameter( description="Input parameter description", ), }, required_properties=["input"], ) super().__init__( name="custom_tool", description="Tool description for LLM selection", parameters=parameters, execute_func=self.execute_custom_logic, ) def execute_custom_logic(self, **kwargs): # Custom implementation return f"Processed: {kwargs.get('input')}" Tool Configuration ~~~~~~~~~~~~~~~~~~ **Parameter Types and Validation** Tools support various parameter types with automatic validation: .. code-block:: python from neo4j_graphrag.tool import ( ObjectParameter, StringParameter, IntegerParameter, NumberParameter, BooleanParameter, ArrayParameter ) parameters = ObjectParameter( properties={ "text": StringParameter(description="Text input"), "count": IntegerParameter(description="Count", minimum=1, maximum=100), "threshold": NumberParameter(description="Threshold", minimum=0.0, maximum=1.0), "enabled": BooleanParameter(description="Enable feature"), "items": ArrayParameter( description="List of items", items=StringParameter(description="String item") ), }, required_properties=["text", "count"] ) **System Instruction Customization** Customize how the LLM selects tools by providing a custom system instruction: .. code-block:: python custom_instruction = """ You are a specialized assistant for movie database queries. Select tools based on query type: use vector_search for plot similarity, cypher_search for specific actor/director queries, and both for complex requests. """ tools_retriever = ToolsRetriever( driver=driver, llm=llm, tools=[vector_tool, cypher_tool], system_instruction=custom_instruction, ) The system instruction guides the LLM's tool selection logic and can be tailored to your specific domain or use case. **Error Handling Patterns** Tools should implement proper error handling for common failure scenarios: .. code-block:: python def execute_custom_logic(self, **kwargs): try: # Tool parameter validation required_param = kwargs.get('required_param') if not required_param: raise ValueError("Required parameter 'required_param' is missing") # Tool execution logic result = perform_operation(required_param) return result except ValueError as e: # Parameter validation errors return f"Parameter error: {str(e)}" except Exception as e: # Tool execution errors return f"Execution error: {str(e)}" The ToolsRetriever handles LLM selection errors internally and will retry tool selection or return appropriate error messages when tools cannot be executed successfully. **Integration with GraphRAG** The ToolsRetriever integrates seamlessly with the GraphRAG pipeline for automated question answering: .. code-block:: python from neo4j import GraphDatabase from neo4j_graphrag.retrievers import ToolsRetriever, VectorRetriever, Text2CypherRetriever from neo4j_graphrag.llm import OpenAILLM from neo4j_graphrag.embeddings import OpenAIEmbeddings from neo4j_graphrag.generation import GraphRAG URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") # Connect to Neo4j database driver = GraphDatabase.driver(URI, auth=AUTH) # Create LLM and embedder llm = OpenAILLM(model_name="gpt-5", model_params={"temperature": 0}) embedder = OpenAIEmbeddings(model="text-embedding-3-large") # Create retrievers to use as tools vector_retriever = VectorRetriever( driver=driver, index_name="vector-index", embedder=embedder, ) text2cypher_retriever = Text2CypherRetriever( driver=driver, llm=llm, ) # Convert to tools vector_tool = vector_retriever.convert_to_tool( name="vector_search", description="Search for similar documents using vector similarity", ) cypher_tool = text2cypher_retriever.convert_to_tool( name="cypher_search", description="Generate and execute Cypher queries for structured data retrieval", ) # Initialize ToolsRetriever tools_retriever = ToolsRetriever( driver=driver, llm=llm, tools=[vector_tool, cypher_tool], ) # Initialize GraphRAG pipeline with ToolsRetriever rag = GraphRAG(retriever=tools_retriever, llm=llm) # Query the pipeline - the LLM will automatically select appropriate tools query_text = "What movies did Tom Hanks act in and what are their plots?" response = rag.search(query_text=query_text, retriever_config={"top_k": 5}) print(response.answer) .. warning:: ToolsRetriever requires an LLM instance to function. Using LLM providers like OpenAI requires their respective Python packages to be installed: `pip install "neo4j_graphrag[openai]"`. See :ref:`toolsretriever`. .. _custom-retriever: Custom Retriever ================ If the application requires very specific retrieval strategy, it is possible to implement a custom retriever using the `Retriever` interface: .. code:: python from neo4j_graphrag.retrievers.base import Retriever class MyCustomRetriever(Retriever): def __init__( self, driver: neo4j.Driver, # any other required parameters ) -> None: super().__init__(driver) def get_search_results( self, query_vector: Optional[list[float]] = None, query_text: Optional[str] = None, top_k: int = 5, filters: Optional[dict[str, Any]] = None, ) -> RawSearchResult: pass See :ref:`rawsearchresult` for a description of the returned type. *********************** GraphRAG search options *********************** Return context ============== By default, the search method only returns the final answer. It is possible to see what was retrieved as part of the context by setting `return_context=True`: .. code:: python rag.search("my question", return_context=True) Return a custom message if context is empty =========================================== If the retriever is not able to find any context, the LLM will return an answer anyway. It is possible to skip the LLM call if the context is empty and return a user-defined message instead: .. code:: python rag.search( "my question", response_fallback="I can not answer this question because I have no relevant context." ) Pass configuration to the retriever search method ================================================= The retrievers search method have a bunch of configuration options (see above), which can also be configured through the GraphRAG search method using the `retriever_config` argument. For instance, the following code snippet illustrates how to define the `top_k` parameter for the retriever: .. code:: python rag.search( "my question", retriever_config={"top_k": 2} ) ************** DB Operations ************** See :ref:`database-interaction-section`. Create a Vector Index ===================== .. code:: python from neo4j import GraphDatabase from neo4j_graphrag.indexes import create_vector_index URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") INDEX_NAME = "chunk-index" DIMENSION=1536 # Connect to Neo4j database driver = GraphDatabase.driver(URI, auth=AUTH) # Creating the index create_vector_index( driver, INDEX_NAME, label="Document", embedding_property="vectorProperty", dimensions=DIMENSION, similarity_fn="euclidean", ) .. _filterable-index-creation: Create a Vector Index with Filterable Properties ================================================= On Neo4j 2026.01+, you can create a vector index with filterable properties to enable in-index filtering via the ``SEARCH`` clause. This avoids brute-force scanning and provides significantly better performance for filtered vector searches. .. code:: python from neo4j import GraphDatabase from neo4j_graphrag.indexes import create_vector_index URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") INDEX_NAME = "chunk-index" DIMENSION = 1536 driver = GraphDatabase.driver(URI, auth=AUTH) # Create index with filterable properties for in-index filtering create_vector_index( driver, INDEX_NAME, label="Document", embedding_property="vectorProperty", dimensions=DIMENSION, similarity_fn="cosine", filterable_properties=["year", "category"], ) The ``filterable_properties`` parameter accepts a list of node property names. These properties will be indexed alongside the vector embeddings, enabling the ``SEARCH`` clause to filter results within the index itself. .. note:: The ``filterable_properties`` parameter requires Neo4j 2026.01+. On older versions, the parameter is accepted but the generated ``WITH`` clause may cause an error if the server does not support it. Populate a Vector Index ======================= .. code:: python from random import random from neo4j import GraphDatabase from neo4j_graphrag.indexes import upsert_vectors from neo4j_graphrag.types import EntityType URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") # Connect to Neo4j database driver = GraphDatabase.driver(URI, auth=AUTH) # Upsert the vector DIMENSION = 1536 vector = [random() for _ in range(DIMENSION)] upsert_vectors( driver, ids=["1234"], embedding_property="vectorProperty", embeddings=[vector], entity_type=EntityType.NODE, ) This will update the node with `id(node)=1234` to add (or update) a `node.vectorProperty` property. This property will also be added to the vector index. Drop a Vector Index =================== .. warning:: This operation is irreversible and should be used with caution. .. code:: python from neo4j import GraphDatabase URI = "neo4j://localhost:7687" AUTH = ("neo4j", "password") # Connect to Neo4j database driver = GraphDatabase.driver(URI, auth=AUTH) drop_index_if_exists(driver, INDEX_NAME)