Comparing documents with AI: finding the meaningful differences

An AI for comparing documents solves a problem that an ordinary editor does not: it shows all changes in a row, while you need to understand which of them change the meaning. A moved paragraph, a corrected comma and a new payment deadline look the same in a diff. A working scheme has two steps: a script finds exactly what changed, and the model explains what it means. Below are Python code, a report format and a cost estimate. The approach also works for document analysis in general: first extract the facts reliably, then ask for interpretation.

Why not just "compare these two files"

You can send the model both versions and ask it to find the differences. For small texts this works, but there are risks:

• the model may miss a small change, such as a digit in an amount or a single word "not";
• it may "find" a change that is not there;
• the result cannot be reproduced and checked line by line.

So we take the facts about changes from an algorithm (it is deterministic and invents nothing), and give the model only the interpretation of the fragments found. It is cheaper and more reliable.

Step 1. A text diff

Python's standard library has difflib. We split the documents into paragraphs and find the changed blocks:

import difflib

def load_paragraphs(path: str) -> list[str]:
    text = open(path, encoding="utf-8").read()
    return [p.strip() for p in text.split("\n\n") if p.strip()]

def changed_blocks(old: list[str], new: list[str]):
    matcher = difflib.SequenceMatcher(a=old, b=new, autojunk=False)
    blocks = []
    for tag, i1, i2, j1, j2 in matcher.get_opcodes():
        if tag == "equal":
            continue
        blocks.append({
            "type": {"replace": "changed", "delete": "removed", "insert": "added"}[tag],

If your documents are in Word or PDF, extract the text first (methods are covered in extracting structured data from documents↗). Scans need text recognition first.

Important: paragraph-level comparison suits texts with the same structure. If the new version merged or split paragraphs, the diff will show them as "changed" and the model will sort it out by meaning.

Step 2. Estimate the request size

Before sending it is useful to count tokens. For the Anthropic format there is a separate token-counting method, see the count_tokens description↗:

import json
import anthropic

claude = anthropic.Anthropic(base_url="https://api.zukko.pro", api_key="fetch_your_key")

payload = json.dumps(blocks, ensure_ascii=False)
count = claude.messages.count_tokens(
    model="claude-sonnet-5-5",
    messages=[{"role": "user", "content": payload}],
)
print("tokens in the change blocks:", count.input_tokens)

Note that for the Anthropic SDK the address is given without /v1; the SDK adds the path itself.

Step 3. The model's semantic review

We pass the change blocks and ask the model to assess the impact of each. We use the OpenAI SDK for Python:

import json
from openai import OpenAI

client = OpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")

SYSTEM = """You are given blocks of changes between two versions of a document: before / after.
For each block return a JSON object:
{"number": N, "summary": "what changed in one phrase",
"category": "deadline|amount|liability|parties|procedure|terminology|editorial|other",
"impact": "high|medium|low|none",
"comment": "why it matters, based only on the text of the block"}
Do not add conclusions that are not in the block text. Do not give legal assessments of lawfulness.
Reply: a JSON array, no explanations."""

Sorting by impact puts the edits you should read first at the top. If the reply JSON turns out invalid, wrap json.loads in a try, as in the article on structured JSON output↗.

Report format

A table is convenient for people. Example rows of the final report:

Add the block number and a "before / after" quotation to the report so a person finds the place in seconds.

What a human checks

• All blocks with high and medium impact, or better the whole report.
• Blocks where the model writes "no impact": make sure it did not miss meaning behind cosmetics.
• Moves: if a paragraph moved to another section, its meaning may have changed because of context.
• The legal force of the changes: the model describes differences but does not assess their lawfulness. For contracts with real consequences, show the report to a lawyer. A similar approach to analysis is in contract analysis with AI↗.

What the comparison costs

Prices as of October 18, 2026, $ per 1M tokens (input / output): claude-sonnet-5-5 0.4 / 2, claude-opus-5-5 0.8 / 4, gpt-6.1-sol 0.27 / 1.32.

Example: two contracts of 15,000 tokens each, the changed blocks add up to 4,000 input tokens together with the instruction, and the report is 1,500 tokens. A request through the OpenAI format, no cache. Formula: (input × input price + output × output price) / 1,000,000.

Option A, changed blocks only (recommended):

• claude-sonnet-5-5: 4,000 × 0.4 + 1,500 × 2 = 1,600 + 3,000 = 4,600, i.e. $0.0046
• gpt-6.1-sol: 4,000 × 0.27 + 1,500 × 1.32 = 1,080 + 1,980 = 3,060, i.e. $0.00306

Option B, both versions in full (30,000 tokens) and a request to find the differences:

• input: 30,000 + 500 for the instruction = 30,500 tokens
• claude-sonnet-5-5: 30,500 × 0.4 + 1,500 × 2 = 12,200 + 3,000 = 15,200, i.e. $0.0152
• claude-opus-5-5: 30,500 × 0.8 + 1,500 × 4 = 24,400 + 6,000 = 30,400, i.e. $0.0304

Option A is roughly three times cheaper under these assumptions and more reliable in accuracy. For a hundred document pairs a month, option A on claude-sonnet-5-5 gives 100 × 0.0046 = $0.46. Whether a mid-range model is enough for your contracts, test on a few pairs with known changes. Current prices: GET https://api.zukko.pro/v1/models or the "Models" page in the dashboard.

To try it, create a key in the dashboard↗ and compare two versions of any non-confidential document.

FAQ

Is it enough to send the model both versions in full?

For short texts perhaps, but the model may miss a small edit or invent a non-existent one. It is more reliable to find the changes with an algorithm and give the model only the interpretation.

Does the method work for PDF and Word?

Yes, if you extract the text first. Scans need recognition. Make sure numbering and tables are not lost during extraction.

Can I trust a "no impact" rating?

Check it by sampling. The model may mistake a semantic edit for an editorial one, so read important documents in full using the report.

Is it safe to send contracts to a model?

Send only what you are entitled to pass to an external service. Anonymize personal data and trade secrets. Your organization decides what is acceptable.