Website analysis with AI: auditing pages, texts and structure via API

An AI for website analysis does not visit a page by itself and does not know how a search engine sees it. You need to hand it the data, so the working scheme is this: a script collects headings, meta tags and text from each page, and the model turns the set of fields into a readable report and suggestions. Below is such an audit in Python, how to keep token spending under control, and what the method cannot do. AI for SEO is a helper here, not a replacement for tools that see rankings and traffic.

What you can check with a model

Content-based tasks that are hard to describe with rules fit:

• Headings and their hierarchy. Whether there is one H1, whether H2s repeat, whether they reflect the content.
• Title and meta description. Match with the page text, repeats between pages, readability.
• Texts. Vague wording, repetition, paragraphs without specifics, failure to answer the page's main question.
• Structure. Whether there are twin pages that should be merged, whether there are enough internal links.
• Match to the query. Whether the page answers the query it was written for (you name the query yourself).

Rule-based checks are better done in code, without spending tokens: title length, alt text on images, a canonical tag, the number of H1s. The model is needed where meaning is needed.

What the AI will not see

• Real rankings, traffic, impressions and clicks. Analytics and webmaster tools provide those, not a model.
• Load speed and technical server errors, indexing, broken links across the whole site: you need crawlers and monitoring.
• Pages built by a script in the browser. If content is loaded by JavaScript, a simple HTML request will not get it.
• External links to your site.
• Competitors: without their data the model cannot say anything reliable.

So AI for website analysis complements a regular SEO audit rather than replacing it. Treat the model's optimization conclusions as hypotheses to verify. Analyze only your own site or one where you have permission.

Step 1. Collect the page data

Don't send raw HTML to the model: a page contains lots of service markup and scripts, and tokens cost money. Extract only what is needed:

import requests
from bs4 import BeautifulSoup

def page_summary(url: str) -> dict:
    r = requests.get(url, timeout=20, headers={"User-Agent": "site-audit-script"})
    r.raise_for_status()
    soup = BeautifulSoup(r.text, "html.parser")

    def attr(tag, name):
        return tag.get(name, "").strip() if tag else ""

    for t in soup(["script", "style", "noscript", "svg"]):
        t.decompose()
    text = " ".join(soup.get_text(" ").split())

The libraries install with pip install requests beautifulsoup4. Before a mass crawl, check robots.txt and do not create load: pause between requests.

Step 2. Rule-based checks

Count the simple things yourself; it is free and reliable:

def rule_checks(s: dict) -> list[str]:
    issues = []
    if not 30 <= len(s["title"]) <= 65:
        issues.append(f"title length {len(s['title'])}")
    if not 70 <= len(s["meta_description"]) <= 170:
        issues.append(f"description length {len(s['meta_description'])}")
    if len(s["h1"]) != 1:
        issues.append(f"H1 count on page: {len(s['h1'])}")
    if s["images_without_alt"]:
        issues.append(f"images without alt: {s['images_without_alt']}")
    return issues

The thresholds here are arbitrary; choose your own. They are not search engine requirements.

Step 3. A semantic report from the model

We send the model compact JSON and the page's main query. An OpenAI-format endpoint; details are in the Python documentation↗:

import json
from openai import OpenAI

client = OpenAI(base_url="https://api.zukko.pro/v1", api_key="fetch_your_key")

SYSTEM = """You audit a website page using the data you are given.
Do not invent anything that is not in the data and do not draw conclusions about rankings or traffic.
Return JSON: {"headings_assessment": "...", "title_description_assessment": "...",
"text_problems": ["..."], "fix_first": ["..."]}.
Base every item on a quotation from the data. JSON only."""

def audit(summary: dict, query: str, model: str = "deepseek-v4.1-flash"):
    user = f"Main query of the page: {query}\nPage data:\n" + json.dumps(summary, ensure_ascii=False)
    resp = client.chat.completions.create(

For a list of pages, run a loop with pauses or several requests in parallel. If the model sometimes returns invalid JSON, see the ways to get structured JSON output↗.

AI for website analysis for free: what is realistic

We do not promise free access to paid models through the API. But an audit costs very little. Prices as of October 18, 2026, $ per 1M tokens (input / output): deepseek-v4.1-flash 0.013 / 0.038, claude-sonnet-5-5 0.4 / 2.

Calculation for a 50-page site. A compact page summary with the instruction takes about 1,500 tokens, the report 500 tokens. In total 75,000 input tokens and 25,000 output.

• deepseek-v4.1-flash: 75,000 × 0.013 + 25,000 × 0.038 = 975 + 950 = 1,925, i.e. $0.001925
• claude-sonnet-5-5 (OpenAI format, no cache discount): 75,000 × 0.4 + 25,000 × 2 = 30,000 + 50,000 = 80,000, i.e. $0.08

Now compare with sending raw HTML. Suppose (an illustrative figure; it varies between sites) a raw page takes 40,000 tokens: that is 50 × 40,000 = 2,000,000 input tokens. On claude-sonnet-5-5 that is 2,000,000 × 0.4 / 1,000,000 = $0.80 for input alone. Saving tokens by cleaning first is the main lever here. More ways: how to reduce token usage↗. Whether a cheap model is enough for your reports, test on a few pages. Current prices: GET https://api.zukko.pro/v1/models.

To try it, create a key in the dashboard↗, set a small daily limit on it and run the script on 5 pages of your own site.

FAQ

Can AI replace an SEO specialist?

No. It quickly parses texts and structure, but it does not see rankings, traffic, indexing or competitors. Use the result as a list of hypotheses to verify.

Can I just give the model a link to the site?

That depends on the tool. Through a regular API the model does not open the page itself, so the data is collected by a script and passed in the request.

Will the method work for a JavaScript site?

For pages where content is loaded by a script, a plain HTML request will not get the content. You need a tool that executes the page in a browser, which is a separate setup.

Which model is needed for an audit?

For summaries on headings and meta tags a cheap model is usually enough; for a deeper analysis of texts, compare it with a stronger one on a few pages.