Fine-tuning or RAG: what to choose to make a model work with your data

Fine-tuning or RAG: the question sounds like a choice between two options, but in fact these are different tools for different jobs. Fine-tuning changes the behavior of the model itself: style, answer format, narrow specialization. RAG (retrieval-augmented generation) and long context do not change the model but put the needed facts into the request. Below we compare three approaches and give a simple rule for choosing.

A note on scope first. Zukko is a gateway for requests to ready-made models, and below we calculate the cost of requests. We do not describe or know how fine-tuning is run and priced at specific suppliers: their terms are their own and change, so check them on the supplier's site.

How the three approaches differ in essence

Fine-tuning takes an already trained model and trains it further on your "question → correct answer" examples. After that the model reacts to requests differently. Facts are remembered unreliably and with no source: fine-tuning suits manner better than knowledge.

RAG stores your documents separately, finds suitable fragments for each question and adds them to the request. The model does not change, the input text does.

Long context puts whole documents into the request. It is the simplest option, limited by window size (most models in the Zukko catalog have 1M tokens, Grok 256K, GLM 5.3 128K) and by the price of each request.

Fine-tuning or RAG: a comparison table

When fine-tuning fits

Consider fine-tuning if all conditions hold:

1. The task repeats thousands of times in one format (for example, labeling tickets into your categories).
2. You have hundreds or thousands of verified "input → correct output" examples.
3. You need a particular style, a strict format or a short prompt, not new knowledge.
4. You are ready to evaluate quality on a held-out set and repeat training when the rules change.

Before fine-tuning try cheap substitutes: a clear instruction and a few examples right in the prompt. The techniques are described in few-shot examples in a prompt↗. Often this is enough and nothing needs retraining.

When RAG for an AI assistant fits

RAG for AI is needed when answers depend on facts that change often or do not fit in the window: a support knowledge base, internal regulations, a product catalog. It gives what fine-tuning does not: fresh data without retraining and a source reference. If a price or a rule changes in the base, updating the index is enough.

What to keep in mind: you will have to build retrieval and test it separately, and the answer will be no better than the found fragments. A minimal code example is in the article RAG by hand↗.

When long context is enough

If there are few documents and few questions, the cheapest path in effort is to put everything in the request. Especially when you need to work with a document as a whole: compare two contract versions, find contradictions, write a summary. Fragment search breaks such tasks, while long context does not.

What it costs in requests

Take an example: a handbook of 30 thousand tokens and 5000 requests per month. Output tokens are left out of the comparison: they are the same in both options. For RAG we take 3000 input tokens per request. Prices as of 18 October 2026, $ per 1M tokens (input), formula: tokens × price.

With cache the picture for long context changes a lot if the prefix is identical and requests are frequent enough. The ideal case (cache hits on every request except the first):

• gpt-6.1-sol: 0.03 × 0.27 + 4999 × 0.03 × 0.013 = 0.0081 + 1.95 = about $1.96.
• claude-sonnet-5-5 in the Anthropic format (write 0.5, read 0.05): 0.03 × 0.5 + 4999 × 0.03 × 0.05 = 0.015 + 7.50 = about $7.51.

So with a small base and frequent requests, cached long context can be comparable to RAG in price and needs no index. But the calculation is ideal: if there are long pauses between requests, the cache expires, and the next request is again billed at the input price, and for Claude with a cache write on top. How cache is accounted is described in the limits and cache documentation↗, and the general logic is in the article prompt caching↗.

Fine-tuning deliberately has no row in this table: the price depends on the supplier and the size of the training set, while the price of a request you already know from the catalog price table. Current prices are GET https://api.zukko.pro/v1/models or the Models page in the dashboard.

A five-minute rule for choosing

Ask yourself the questions in order:

1. Is it about facts or manner? If the model does not know your data, you need context or RAG. If it knows but answers in the wrong format or style, try a prompt with examples, then fine-tuning.
2. Does the data fit in the window, and how many questions are there? Little data and few questions: long context. A lot of data or many questions: RAG.
3. Does the data change? If yes, fine-tuning for facts is out: you would have to retrain constantly.
4. Do you need source references? Then RAG or context.
5. What does a mistake cost? For legal and medical answers you need a source and human review regardless of the approach.

The approaches are combined: fine-tuning handles format, RAG handles facts. But start with the simplest, long context, and complicate things only when it stops working.

Test any approach on your own questions: a model's intelligence index and price do not replace a test.

FAQ

Will fine-tuning teach the model my documents?

Partly and unreliably. The model may remember some facts but get details wrong and not show a source. For knowledge, RAG or context is better.

What is cheaper?

It depends on volume. For a small base and rare requests, long context is cheapest because there are no index or training costs. For a large base and frequent requests, RAG is cheaper. Calculate with the formula above on your own numbers.

Can fine-tuning and RAG be combined?

Yes. Fine-tuning sets format and tone, RAG brings current facts. But it is the most labor-intensive option and is not the place to start.

How do I check that the chosen approach works?

Collect 20 to 50 real questions with known answers and run them after every change. You can try a cheap model with a key from the dashboard↗.