LlamaIndex vs LangChain: what to pick for RAG and agents in 2026

LlamaIndex and LangChain are often compared as if you must pick one library forever. In practice they overlap: both do RAG, both do agents, both connect to any model with an OpenAI-compatible API. The "llamaindex vs langchain" question is better put this way: which side suits you for approaching the task, the data side or the action-chain side. Below we cover the ideas behind both libraries, show working code with the same model through a gateway and count the cost of a request.

LlamaIndex vs LangChain: what they are built on

LlamaIndex grows from data. The central scheme: load documents, split them into chunks, build an index, ask questions through a query engine. Its documentation has separate sections on loading data, indexing, querying, storing and agents. In the current version an agent is built with the FunctionAgent class and a set of tools, and search over documents is wrapped as such a tool.

LangChain grows from components and the agent loop. In the current documentation the main entry point is create_agent: a minimal configurable harness around a model, where you add tools, a system prompt and middleware. For complex orchestration with state, LangGraph stands next to it, and LangChain's agent abstractions are built on top of it.

The learning curve is lower for the one whose defaults match your task. If the task sounds like "answers over my documents", starting with LlamaIndex is easier. If it sounds like "an agent with a set of tools and its own logic", starting with LangChain is easier.

One model through the gateway in both

Install: pip install langchain-openai llama-index-core llama-index-llms-openai-like. You need a fetch_... key from the dashboard↗ and the address https://api.zukko.pro/v1; the general request format is described in the Python reference↗ and chat/completions↗.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="claude-sonnet-5-5",
    base_url="https://api.zukko.pro/v1",
    api_key="fetch_your_key",
)
print(llm.invoke("Name three risks of a database migration").content)

from llama_index.core.llms import ChatMessage
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="claude-sonnet-5-5",
    api_base="https://api.zukko.pro/v1",
    api_key="fetch_your_key",
    is_chat_model=True,
    context_window=1000000,
)
reply = llm.chat([ChatMessage(role="user", content="Name three risks of a database migration")])
print(reply.message.content)

Two traps specific to LlamaIndex. First: the regular OpenAI class checks the model name against its own list and fails on claude-sonnet-5-5 with an Unknown model error. For a gateway use OpenAILike, as the LlamaIndex documentation also advises for any OpenAI-compatible API. Second: if you do not set an embedding model, VectorStoreIndex defaults to OpenAI embeddings. The Zukko documentation has no embeddings endpoint, so set them explicitly, for example with a local model.

RAG: minimal examples

LlamaIndex. Install: pip install llama-index-embeddings-huggingface.

from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.llm = llm  # the OpenAILike from the example above
Settings.embed_model = HuggingFaceEmbedding(model_name="intfloat/multilingual-e5-small")

docs = SimpleDirectoryReader("docs").load_data()
index = VectorStoreIndex.from_documents(docs)
print(index.as_query_engine().query("What is the payment term in the contract?"))

For Russian-language documents take a multilingual embedding model and check search quality on your own questions.

LangChain. Install: pip install langchain-huggingface. Here retrieval and generation are assembled explicitly:

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_huggingface import HuggingFaceEmbeddings

emb = HuggingFaceEmbeddings(model_name="intfloat/multilingual-e5-small")
store = InMemoryVectorStore.from_texts(["The contract runs for one year.", "Payment is due within 10 days."], emb)

question = "What is the payment term in the contract?"
context = "\n\n".join(d.page_content for d in store.similarity_search(question, k=2))
prompt = ChatPromptTemplate.from_messages(
    [("system", "Answer using only the context:\n{context}"), ("user", "{question}")]
)
chain = prompt | llm | StrOutputParser()  # the ChatOpenAI from the example above
print(chain.invoke({"context": context, "question": question}))

It is more lines, but each step is visible and replaceable: a different retriever, splitter or prompt.

Agents

In LangChain: create_agent(model=llm, tools=[...], system_prompt="..."), then agent.invoke({"messages": [...]}). In LlamaIndex: FunctionAgent(tools=[...], llm=llm, system_prompt="..."), then await agent.run("..."). For LlamaIndex through OpenAILike, set is_function_calling_model=True, otherwise the agent will not call functions through the API. Whichever library you use, check that the chosen model supports tool calling and set a step limit so the agent cannot loop.

What a request costs

Prices as of October 18, 2026, per 1M tokens. A typical RAG request: 6,000 input tokens (question, instruction and retrieved chunks) and 500 output tokens. Frameworks add their own prompt templates, so look at the real input under "Usage" in the dashboard.

• claude-sonnet-5-5 (0.4 / 2): 6,000 × 0.4 + 500 × 2 = 2,400 + 1,000 = 3,400 micro-dollars, $0.0034 per request, $3.40 per 1,000 requests.
• gpt-6.1-sol (0.27 / 1.32): 6,000 × 0.27 + 500 × 1.32 = 1,620 + 660 = 2,280 micro-dollars, $2.28 per 1,000 requests.
• glm-5.3-flash (0.04 / 0.04): 6,000 × 0.04 + 500 × 0.04 = 240 + 20 = 260 micro-dollars, $0.26 per 1,000 requests.

Compare model strength and context in the model table and test on your own questions: the list and prices are at GET https://api.zukko.pro/v1/models. Payment: SBP (the payment page adds an 8.5% fee on top), cryptocurrency or MasterCard. More on each library separately: LlamaIndex: indexing data↗ and LangChain: your own base_url↗.

What to choose

• The main task is answers over documents with many data sources: LlamaIndex.
• The main task is an agent with tools and its own logic, with integrations and tracing: LangChain.
• You do not want to depend on a framework: the official SDK and a couple of hundred lines of your own code are enough.
• You can mix them: loading and indexing in LlamaIndex, the agent loop in LangChain. But each dependency costs maintenance.

FAQ

Can I use both libraries in one project?

Yes. They do not conflict, and it is convenient to move the shared model into a separate module with the address https://api.zukko.pro/v1 and the key from an environment variable.

Why does LlamaIndex try to reach OpenAI although I set my own address?

Most likely no embedding model is set: OpenAI embeddings are the default. Set Settings.embed_model.

Does Claude need the Anthropic format?

No, Claude also works in the OpenAI format. The only difference is the cache: the Claude cache-read discount comes with the Anthropic format.

How do I avoid overspending while debugging?

Give the key a daily dollar limit and start with a cheap model such as glm-5.3-flash, moving to a strong one once the logic works.