A database for RAG: where to store knowledge — vector DB, search or context

The first question after hearing about RAG is usually "which database should I use for RAG". Most guides answer with a vector database. But a knowledge store for a model can be built four ways, and for small and medium volumes the most expensive one to maintain is often unnecessary. Below we compare the options, calculate the cost of one query and give code that runs without a separate server.

What a RAG store actually holds

The RAG architecture in a nutshell: the system has two stages. First you split documents into fragments and put them in a store. Then, for each question, you find a few suitable fragments and insert them into the request to the model together with the question. The store exists for the second step: quickly answering "which 3 to 5 pieces of text relate to this question".

What you keep depends on the search method: the text itself, the text plus its numeric representation (a vector, an embedding), or simply the whole corpus.

Four storage options

Vector database

Each fragment is turned into a vector by an embedding model, the question is turned into a vector by the same model, and the database returns the fragments closest in meaning. There are many ready options: the pgvector extension for PostgreSQL, Qdrant, Chroma, FAISS as a library. The main advantage is that it finds by meaning even when the words in the question and in the text differ.

What to keep in mind. You need a separate embedding model, run locally or taken from another service: the Zukko model table we work from as of October 18, 2026 contains no embedding models (check the current list with GET https://api.zukko.pro/v1/models). If you change the embedding model, all vectors must be recomputed. And it is one more component to store, back up and update.

Full-text search

Classic word search with BM25 ranking: PostgreSQL has it (tsvector), SQLite has it (the FTS5 module), as do Elasticsearch and OpenSearch. No embeddings needed. It finds what matches by words, so it works well for documentation, policies, code, product codes and names, where users use the same terms. The weak spot is synonyms and rephrasing, and for Russian also word forms, because the standard tokenizer does not reduce words to their base form.

Hybrid search

Results from full-text and vector search are merged into one list. This usually finds more of the relevant fragments, but it is also the most complex option: two systems, two indexes and a rule for merging rankings. Justified when the base is large and questions vary in nature.

Long context instead of search

All documents go straight into the request. All models in the Zukko table have a 1M-token window, except Grok (256K) and GLM (128K), so a base of hundreds of thousands of tokens physically fits. There is no store at all, just files.

Comparison

What it costs: one query

Assume a knowledge base of 300,000 tokens, a model answer of 1,000 tokens. Zukko prices as of October 18, 2026 per 1M tokens: gpt-6.1-sol input 0.27, output 1.32; deepseek-v4.1-flash input 0.013, output 0.038. This is an illustrative estimate; the real length of your base is shown by count_tokens (see the docs↗).

Long context, gpt-6.1-sol, no cache. Input: 0.3 × 0.27 = $0.081. Output: 0.001 × 1.32 = $0.00132. Total $0.08232 per question. At 10,000 questions a month: 0.08232 × 10,000 = $823.2.

Same setup, deepseek-v4.1-flash. Input: 0.3 × 0.013 = $0.0039. Output: 0.001 × 0.038 = $0.000038. Total about $0.00394, or $39.4 per 10,000 questions.

RAG with five fragments of 500 tokens and a 500-token question. Input is 3,000 tokens. gpt-6.1-sol: 0.003 × 0.27 = $0.00081, plus output $0.00132, total $0.00213; $21.3 per 10,000 questions.

The takeaway: at high query volume search saves noticeably on tokens, while at a few dozen queries a day long context is simpler and the price is tolerable. Cache changes the picture: for GPT the cache read of gpt-6.1-sol costs $0.013 per 1M, so 0.3 × 0.013 = $0.0039 for a repeating prefix, but the first request is billed at the full input price and the cache only works while the start of the request stays unchanged. How it is calculated is covered in the article on cache reads and writes↗.

A working example: full-text RAG on SQLite

No separate server, no embeddings. SQLite with the FTS5 module ships with most Python builds; if FTS5 is unavailable on your machine, you will need a different SQLite build.

import re
import sqlite3
from openai import OpenAI

db = sqlite3.connect("kb.db")
db.execute("CREATE VIRTUAL TABLE IF NOT EXISTS chunks USING fts5(title, body)")

client = OpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")


def add(title: str, body: str) -> None:
    db.execute("INSERT INTO chunks(title, body) VALUES (?, ?)", (title, body))
    db.commit()

Words in the query are quoted so that special characters from the question do not break the search syntax. For Russian the built-in tokenizer does not know word forms, so the query "договоры" will not find "договор": fix it either by reducing words to their base form before indexing and searching (for example with the pymorphy3 morphological analyzer) or by prefix search. Check the SQLite documentation for the exact tokenizer options.

How to choose

1. A base of up to a few hundred thousand tokens and few questions: start with long context, no store needed.
2. Questions use the same words as the documents: full-text search, SQLite or the PostgreSQL you probably already have.
3. You need search by meaning or a base of millions of tokens: a vector database or a hybrid, and then you must decide where embeddings come from.
4. Not sure: collect 30 to 50 real questions with known answers and check which option finds the right fragments.

More on the topic: RAG on a cheap model without a vector DB↗ and a minimal RAG pipeline↗.

FAQ

Do I need a vector database for RAG?

No. A vector database is needed when search must work by meaning rather than by words, or when there are very many documents. For a small base with uniform terminology full-text search is enough, and for a very small one you can put everything into the context.

Where do I store knowledge for an AI agent built on my data?

Keep texts where they already are: files, SQLite or PostgreSQL. The agent needs a search function that returns fragments, not a particular storage type. Fragments are usually cut to a few hundred tokens with the document title attached.

How does RAG differ from long context in price?

Long context pays for the whole base on every question, RAG only for the fragments it found. The calculation above shows the order of magnitude: $0.082 versus $0.002 per question on gpt-6.1-sol with a 300K-token base. Cache for a repeating request prefix narrows the gap but does not remove it.

Can I use Zukko models for embeddings?

The model list we work from as of October 18, 2026 has no embedding models. Check the current list with GET /v1/models; if there are none, use a local embedding model or full-text search.

To build your own version, create a key in the Zukko dashboard↗ and plug it into the example above with the address https://api.zukko.pro/v1.