LlamaIndex, and choosing between frameworks

LlamaIndex sits beside LangChain in most job descriptions. The distinction is real but narrower than the marketing suggests, and being able to state it precisely is the point.

What LlamaIndex is for

LangChain/LangGraph started from the agent loop. LlamaIndex started from retrieval. Both have since grown into the other’s territory, but their centre of gravity still shows in the API.

LlamaIndex’s strength is the ingestion-to-query pipeline: loaders for a very wide range of sources, node parsers, index structures, and query engines that compose retrieval strategies.

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.node_parser import SentenceSplitter

docs = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(
    docs, transformations=[SentenceSplitter(chunk_size=512, chunk_overlap=50)]
)
response = index.as_query_engine(similarity_top_k=5).query("What is the refund window?")
print(response.source_nodes)        # citations come back by default

That’s a working RAG pipeline in five lines, with sources attached. The equivalent in LangChain is more assembly.

The concepts

Node relationships are the genuinely distinctive part. A node knows its previous and next siblings and its parent document, which is what makes auto-merging and sentence-window retrieval work — retrieve a small precise chunk, then expand to its surrounding context before generating. That’s harder to express in a flat chunk store.

KnowledgeGraphIndex and PropertyGraphIndex are also first-class, which matters if the job mentions knowledge graphs. See ../09_rag_embeddings/06_knowledge_graphs_graphrag.md.

Where each one wins

The honest summary: if the hard part is retrieval quality over a messy corpus, LlamaIndex gets you further faster. If the hard part is a multi-step agent that must survive restarts and pause for approval, LangGraph is built for that. See 03_langchain_langgraph.md and 05_durable_execution_hitl.md.

Using both is common and not a smell: LlamaIndex as the retrieval layer, exposed as a tool to a LangGraph agent.

@tool
def search_docs(query: str) -> str:
    """Search internal documentation."""
    return str(query_engine.query(query))

That’s the integration most teams end up with, and it’s a good answer to “which framework would you use”.

The framework-free position

Worth holding, because it’s often correct.

The core of RAG is: embed chunks, store vectors, embed the query, retrieve top-k, put them in a prompt. That’s maybe 60 lines against pgvector and an SDK, and you own every step.

What frameworks buy you: connectors for formats you’d otherwise parse yourself, retrieval strategies already implemented and tested, and a community answer when something breaks. What they cost: abstraction over a pipeline you need to debug, version churn, and the temptation to accept defaults you haven’t measured.

The senior answer: prototype with a framework to find out what the retrieval problem actually is, then decide whether the abstraction is still earning its place. Plenty of production RAG runs on direct SDK calls plus pgvector because the team wanted control over chunking and ranking.

Interview angle

• “LangChain or LlamaIndex?” — different centres of gravity. LlamaIndex is retrieval-first with broad connectors, rich chunking and node relationships; LangGraph is orchestration-first with durable checkpointed state and human-in-the-loop. Pick by where the hard part is, and using LlamaIndex as a retrieval tool inside a LangGraph agent is a normal combination.
• “What’s distinctive about LlamaIndex’s data model?” — nodes carry relationships to their neighbours and parent document, which enables auto-merging and sentence-window retrieval: match a small precise chunk, then expand to surrounding context before generating.
• “Would you use a framework at all?” — basic RAG is about sixty lines against pgvector and an SDK. Frameworks buy connectors, implemented retrieval strategies and community support; they cost abstraction over the pipeline you most need to debug. Prototype with one, then decide.
• “How would you add knowledge-graph retrieval?” — LlamaIndex has PropertyGraphIndex as a first-class index type, which is the least-effort path. Otherwise build the graph separately and expose traversal as a retrieval tool.
• “Where does framework choice actually matter?” — durability and human-in-the-loop. Retrieval you can rebuild; a correctly checkpointed agent that resumes after a restart and pauses days for approval is real engineering, and that’s LangGraph’s argument.