Neural network hallucinations: why a model makes things up and how to reduce it

Neural network hallucinations are confident, smoothly written statements that do not match reality: a library function that does not exist, an invented quotation, a wrong date. The main trouble is the tone: a mistake sounds as convincing as the truth. You cannot get rid of this entirely, but you can lower the frequency and make the remaining cases visible for checking. Below are the causes and five practical measures with code. Prices are as of October 18, 2026.

Why a model makes things up

A language model continues text so that it looks plausible. It does not "look things up in a database", and it has no built-in sense of "I don't know this". Hence the typical causes:

• The data is not in the request. You asked about your internal policy or a contract the model has not seen. Instead of refusing, it builds an answer from general patterns.
• Rare and precise facts. Numbers, dates, names, quotations, links, method names in a narrow library. The narrower the topic, the higher the risk.
• A false premise in the question. "Why does function X in library Y return a list?" when no such function exists. A model tends to accept the question as given.
• Outdated knowledge. A model is trained up to a certain date and does not know about later changes.
• A very long or messy context. The needed fact got lost among the rest, and the model may not have used it. You can test this by placing a control fact in different spots of a long text.

The risk is lower where the model works with text you gave it: paraphrasing, shortening, converting a format. But numbers can be distorted there too, so checks are always needed.

Measure 1. Put the source in the request

The most effective protection is not to ask "from memory". Insert document fragments into the request and require answers only from them. This is how the RAG approach works. If you were searching for what RAG is, in plain words: before the question, a program finds several suitable fragments in your knowledge base and inserts them into the request, and the model formulates the answer from those fragments. A minimal scheme is described in the article A RAG pipeline with your own hands↗.

Fragment selection matters as much as the prompt itself: if the needed piece is not among those found, the model either admits it or invents an answer again. That is why the second measure is mandatory.

Measure 2. Allow "I don't know"

If the instruction has no explicit way out, the model will pick an answer, even a weakly grounded one. State it directly: "If the sources contain no answer, return null". Do it with structure, not with the phrase "try not to be wrong". A good habit in prompt engineering for LLMs (for the system part of a request see the article How to write system prompts↗): describe not only what to do but also what to do when there is nothing to do.

Measure 3. Require a quote

Ask the model to return not only the answer but also a verbatim quotation from the source it relies on. A quotation can be checked in code: it is either in the source or not. This turns a vague "I trust it" into a verifiable condition.

Measure 4. Check in code, not by eye

The code below combines all three measures: sources in the request, permission to refuse, and a quotation with an automatic check. You need pip install openai:

import json
from openai import OpenAI

client = OpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")

SOURCES = {
    "s1": "Refunds are issued within 5 business days after non-delivery is confirmed.",
    "s2": "Delivery within the city takes from one to three business days.",
}

SYSTEM = """Answer only from the sources. Add nothing from your own knowledge.
Return only JSON: {"answer": "...", "source": "s1", "quote": "verbatim quote from the source"}.
If the sources contain no answer, return {"answer": null, "source": null, "quote": null}."""

The second question is a trap: the sources say nothing about 30 days, and the correct behavior is "the sources contain no answer". The quote check works as a safety fuse, but it has a limit: it proves that the quotation really exists, not that the conclusion drawn from it is correct. For critical data (amounts, deadlines, legal terms) show a person both the answer and the quotation side by side.

Measure 5. A second pass and control questions

If the cost of a mistake is high, add a second call: give another model the sources and the answer and ask it to list statements that are not in the sources. It is pricier but catches what the quote check missed. Let us calculate on an example: the first call is 1600 input tokens (three fragments of 500 tokens plus the question) and 200 output; the check is 1500 input tokens and 100 output. These are assumptions. gpt-6.1-sol prices as of October 18, 2026: 0.27 input, 1.32 output per 1M.

• Main call: 1600 × 0.27 / 1M + 200 × 1.32 / 1M = 0.000432 + 0.000264 = $0.000696.
• Check: 1500 × 0.27 / 1M + 100 × 1.32 / 1M = 0.000405 + 0.000132 = $0.000537.
• Together: $0.001233, about 0.12 cents per question, or $1.23 per 1000 questions.

The code check from the previous measure costs zero tokens. Current prices: GET https://api.zukko.pro/v1/models.

To measure progress, build a set of 20–30 questions, some of which intentionally have no answer in the sources ("traps"), and count two shares: how many answers are correct and how many times the model honestly refused where refusal was needed. Change the prompt and the model by looking at these numbers, not at impressions. How to build such a run is shown in the article How to test prompts↗.

All request fields from the example, including messages and roles, are covered in the Chat Completions reference↗. A key for experiments is issued in the dashboard↗, and from Russia the balance is topped up via SBP (plus the payment page's 8.5% fee), a MasterCard or cryptocurrency.

FAQ

Can hallucinations be removed completely?

No. You can lower the frequency and make mistakes visible: sources in the request, the right to refuse, quotations and automatic checks. For critical decisions a person is needed.

Does the choice of model affect hallucinations?

Probably yes, but to what degree depends on the task, and we give no comparisons without measurements. Test several models on your own question set, including the "traps", and compare on price as well.

What is RAG and will it help?

RAG is searching your knowledge base for suitable fragments and inserting them into the request. It lowers the risk of inventions on questions about your data, but only works when the search finds the right fragments and the model follows them.

How do I tell that a model invented a link or a quotation?

Check in code: compare the quotation with the source text, and open the link or look it up in your knowledge base. Mark everything that cannot be confirmed as unverified.