Knowledge Graphs and GraphRAG

Vector RAG retrieves similar chunks. Knowledge graphs retrieve related entities and traverse relationships. For domains where relationships are the answer (org charts, drug interactions, supply chains, fraud rings), graphs beat vectors.

When graphs beat vectors

Vector RAG works when answers live in passages of similar text. It fails when:

• The answer requires combining facts from disconnected sources (“Who reports to Alice’s manager?”).
• The query is structural (“All companies in our supply chain affected by the Taiwan earthquake”).
• Relationships matter more than text similarity (“Doctors who prescribed drug X to patients also taking drug Y”).
• Multi-hop reasoning (“What’s the parent company of the vendor of the subcomponent that failed?”).

For these, you need explicit relationships. Knowledge graphs store them.

Graph basics

A knowledge graph is nodes (entities) connected by edges (relationships):

(Alice:Person) -[:WORKS_AT]-> (Acme:Company)
(Acme:Company) -[:LOCATED_IN]-> (Berlin:City)
(Berlin:City)  -[:IN_COUNTRY]-> (Germany:Country)

Both nodes and edges have labels (types) and properties (key-value attributes).

(Alice:Person {name: "Alice", age: 30})
  -[:WORKS_AT {since: 2022, title: "Engineer"}]->
(Acme:Company {founded: 2010})

Storage options

For most production GraphRAG: Neo4j or Memgraph. For prototyping: NetworkX.

Cypher — the query language

Neo4j’s query language (also supported by Memgraph and others). Reads like ASCII art:

// Find all people who work at companies in Germany
MATCH (p:Person)-[:WORKS_AT]->(c:Company)-[:LOCATED_IN]->(:City)-[:IN_COUNTRY]->(:Country {name: "Germany"})
RETURN p.name, c.name

Multi-hop relationships are first-class. Variable-length paths:

// Find Alice's manager chain (up to 5 levels)
MATCH (alice:Person {name: "Alice"})-[:REPORTS_TO*1..5]->(boss)
RETURN boss.name

Aggregations, filtering, ordering — all SQL-like:

MATCH (p:Person)-[:WORKS_AT]->(c:Company)
WHERE c.industry = "tech"
RETURN c.name, COUNT(p) AS employee_count
ORDER BY employee_count DESC
LIMIT 10

GraphRAG — the retrieval pattern

Microsoft’s GraphRAG paper (2024) popularized the pattern:

1. Build a knowledge graph from your corpus.
2. Extract entities and relationships using an LLM.
3. Generate community summaries (clusters of related entities).
4. At query time, retrieve relevant subgraphs (not chunks).
5. Pass the subgraph + community summaries as context to the LLM.

Stage 1: extraction

prompt = """Extract entities and relationships from the following text.
Return JSON with the structure:
{
  "entities": [{"name": ..., "type": ...}],
  "relationships": [{"source": ..., "target": ..., "type": ..., "description": ...}]
}

Text: {chunk}"""

result = llm.generate(prompt.format(chunk=chunk))
data = json.loads(result)

for ent in data["entities"]:
    graph.merge_node(name=ent["name"], type=ent["type"])
for rel in data["relationships"]:
    graph.add_edge(rel["source"], rel["target"], type=rel["type"], desc=rel["description"])

LLM-based entity/relationship extraction. Use Pydantic + structured output to enforce schema:

class Entity(BaseModel):
    name: str
    type: Literal["Person", "Company", "Location", "Event"]

class Relationship(BaseModel):
    source: str
    target: str
    type: str
    description: str

class Extraction(BaseModel):
    entities: list[Entity]
    relationships: list[Relationship]

Iterate over corpus chunks → graph grows incrementally.

Stage 2: community detection

Run a clustering algorithm (Leiden, Louvain) on the graph to find communities — groups of densely-connected nodes that probably represent topics or themes.

For each community, generate a summary via LLM:

for community in graph.communities():
    members = [node.name for node in community.nodes]
    edges = [(e.source, e.target, e.type) for e in community.edges]
    summary = llm.summarize(f"Summarize this knowledge graph community: nodes={members}, edges={edges}")
    community.summary = summary

Community summaries are pre-computed and indexed (often vector-embedded for retrieval).

Stage 3: query-time retrieval

def graphrag_query(question: str) -> str:
    # 1. Extract entities mentioned in the question
    entities = llm_extract_entities(question)

    # 2. Find relevant communities
    community_summaries = retrieve_communities(question, top_k=5)

    # 3. Pull subgraph around the entities
    subgraph = graph.subgraph_around(entities, hops=2)

    # 4. Build context: community summaries + subgraph
    context = format_for_llm(community_summaries, subgraph)

    return llm.generate(f"Answer using:\n{context}\n\nQuestion: {question}")

The LLM gets a structured view of the data — entities and relationships — instead of (or alongside) text chunks.

Hybrid: vector + graph

Production GraphRAG usually pairs both:

def hybrid_retrieve(question):
    vector_hits = vector_store.search(embed(question), top_k=5)
    entities = extract_entities(question)
    graph_subgraph = graph.expand(entities, hops=2)
    return vector_hits, graph_subgraph

Vector retrieval finds relevant chunks; graph traversal finds connected facts. The LLM consumes both. Each fills the other’s gaps:

• Vector misses cross-document relationships → graph catches them.
• Graph misses unstructured nuance → vector catches it.

Tools

LangChain’s GraphCypherQAChain lets the LLM generate Cypher queries directly:

from langchain.chains import GraphCypherQAChain
from langchain_neo4j import Neo4jGraph

graph = Neo4jGraph(url="bolt://...", username="...", password="...")
chain = GraphCypherQAChain.from_llm(llm=ChatOpenAI(), graph=graph, verbose=True)

result = chain.invoke({"query": "Who is Alice's manager's manager?"})

The chain prompts the LLM with the schema and your question, the LLM emits Cypher, the chain executes it, then the LLM formulates a natural-language answer from the results.

Building the graph — challenges

Entity resolution

LLMs extract “Alice”, “Alice Johnson”, “A. Johnson” as separate entities. Resolve them:

def canonicalize(entity):
    return embedding_match(entity, existing_entities, threshold=0.9)

Or LLM-assisted resolution as a separate pass.

Schema drift

LLM extracts “REPORTS_TO” sometimes, “MANAGED_BY” other times. Same relationship, different label.

Mitigation: provide a fixed schema in the extraction prompt:

ALLOWED_REL_TYPES = ["REPORTS_TO", "WORKS_AT", "LOCATED_IN"]
prompt = f"Use only these relationship types: {ALLOWED_REL_TYPES}..."

Or Pydantic enum types in the extraction schema.

Cost

LLM-based extraction is expensive at scale. A 1M-document corpus × $0.01 per extraction = $10k. Optimize:

• Cheaper extraction model (Haiku, GPT-4o-mini).
• Pre-filter with NER (spaCy) for entity candidates.
• Hybrid: rule-based for common patterns, LLM for ambiguous.

Update strategy

Graphs are mutable. New documents → new entities, possibly conflicting. Snapshots, lineage tracking, conflict resolution all become DB-engineering problems.

When NOT to use a knowledge graph

• Small / simple corpora. Vector RAG is much cheaper to build.
• Relationships don’t matter much. “Summarize this article” doesn’t need a graph.
• Latency-sensitive. Graph traversal + LLM hop is slow vs vector lookup.
• Schema unstable. Domains where relationships keep changing — graph maintenance is expensive.

Interview angle

• “When would you choose a knowledge graph over vector RAG?” — when relationships drive the answer (org charts, supply chains, drug interactions, fraud rings) or multi-hop reasoning is required. Vector RAG works on text similarity; graphs work on structure.
• “What’s GraphRAG?” — Microsoft’s pattern: extract entities + relationships from documents via LLM, build a knowledge graph, detect communities, summarize each, query by retrieving subgraphs + community summaries. Combines graph traversal with LLM synthesis.
• “How do you extract entities and relationships from unstructured text?” — LLM with a structured-output schema (Pydantic), bounded entity/relationship types, iterate over corpus chunks. Pre-filter with NER for cost. Entity resolution (dedup similar names) is a separate pass.
• “Hybrid graph + vector retrieval — why?” — vector finds passage-level relevant text; graph finds connected facts. They cover different failure modes. Production GraphRAG usually combines both.
• “How do you query a knowledge graph from an LLM?” — GraphCypherQAChain pattern: provide the schema to the LLM, prompt it to generate Cypher, execute the Cypher, feed results back to the LLM for natural-language answer. Or precompute subgraphs and pass them as context.
• “What goes wrong in production with GraphRAG?” — entity resolution (same person, different names), schema drift (same relationship, different labels), extraction cost at scale, graph maintenance as data changes. The graph is a maintained artifact, not a one-shot.