Methodology
How we measure what AI engines actually read
Every number in an audit comes from a saved, dated fetch of your page. This page explains how we take those fetches, how we read them, and what we refuse to claim.
Last updated Oct 2, 2026
01 · The two-fetch diff
One URL. Two fetches.
We fetch your URL as a browser with JavaScript on. That render runs through Firecrawl and gives us the page your buyers see. If the page is still empty after the first wait, we retry once with a 7 second wait.
Then we fetch the same URL as each AI agent, with no JavaScript. That is the raw HTML an agent receives when someone asks ChatGPT, Claude or Perplexity about you.
What Google sees
What AI engines fetchWhat we compare
- HTTP status for every user agent.
- Response size in bytes.
- Readable main text: the page's main content with scripts, styles and markup removed, counted in characters.
- Passages: sentences from the rendered page, checked one by one against what the agent received.
02 · Who we fetch as
Ten fetches per URL
Each probe sends nine crawler user agents plus one browser control, one after another, from the same servers. Each agent is also checked against your robots.txt by its exact token.
| Class | User agent | Stands for | Role |
|---|---|---|---|
| Live agents | ChatGPT-User | ChatGPT | Fetch while someone is chatting. These drive the verdict. |
| Claude-User | Claude | ||
| Perplexity-User | Perplexity | ||
| Search crawlers | OAI-SearchBot | ChatGPT search | Decide whether a page can be a retrieval candidate. |
| Claude-SearchBot | Claude search | ||
| PerplexityBot | Perplexity search | ||
| Bingbot | Bing index, upstream of ChatGPT search and Copilot | ||
| Rendering crawler | Googlebot | Google and Gemini | Renders JavaScript. Gemini relies on it. |
| Training crawler | GPTBot | OpenAI training | Comparator only. Never changes the verdict. |
| Browser control | Browser | Control | A normal desktop browser fetch from the same servers. |
A note on Googlebot. Gemini relies on Googlebot, which renders JavaScript. A probe cannot verify itself as Google, so sites that verify bots by IP may refuse our Googlebot fetch. We do not present that result as Google's view.
03 · How we read a result
Seven possible readings
Percentages compare the agent's readable main text with the rendered main text from the same probe run. Each agent keeps its own measurement. We never borrow ChatGPT's number for another engine.
| Reading | Rule | What it means |
|---|---|---|
| Blocked | The agent gets HTTP 401, 403, 451 or 503, another non-2xx, a challenge page, or a robots.txt disallow, while the browser control gets the page. | The refusal is tied to the agent. |
| Unmeasurable from our servers | The browser control from the same servers is refused too, or no identity reached the page. | We cannot tell whether the site treats AI agents differently, so we say so and do not call it blocked. |
| Thin read | The agent receives less than 50% of the rendered main text. | Most of what buyers read never reaches the engine. A redirect stub lands here. |
| Partial read | 50% to 89% of the rendered main text. | Some lines are missing. The redacted page shows which. |
| Full read | 90% or more of the rendered main text. | The agent receives the page buyers read. |
| Rate limited | HTTP 429 or 430 on a fetch. | Flagged as possibly caused by our own probe. It does not fail a check on its own. |
| Not yet measured | No saved fetch or no readable text count for that agent. | Shown as Not yet measured, never as a pass. |
Cloudflare 520 to 527 responses mean the origin was unreachable. They are never counted as a block.
04 · The redacted page
A black bar means not found
We split the rendered text into sentences and keep those with at least 5 words, dropping repeats. Up to 80 passages are shown per page.
Before matching, both sides are lowercased, punctuation is turned into spaces and runs of whitespace collapse to one. A passage counts as received if it appears whole in the agent's saved text. For passages of 16 words or more, it also counts if the first 8 words and the last 8 words both appear.
A black bar means the passage was not found in the saved fetch. It is not a judgement about the passage.
We store up to 64 KB of each saved response and up to 20 KB of readable main text. Readable text counts are measured on responses up to 200,000 characters. When a response is larger than what we stored, we label the comparison as possibly cut off, so passages beyond the cap are not shown as missing without that warning.
05 · Scoring
51 graded surfaces, weighted by severity
Our signal registry holds 44 distinct signals, graded across 51 surfaces. Each carries a severity (critical, high, medium or low) and a provenance tag: live, proxy or narrated. Only live and proxy signals can drive a measured before and after.
Schema markup is capped at medium severity in code and is never treated as a citation lever. The reason: Ahrefs, 11 May 2026, n=1,885 treated pages vs 4,000 controls, found AI Mode +2.4%, ChatGPT +2.2% and AI Overviews -4.6%. That is not a citation lever.
llms.txt is kept as a check with zero citation weight.
The Bull AI score
Four inputs, weighted toward readability. If an input is missing, it is left out, the remaining weights rescale, and the input shows as Not yet measured.
| Input | Weight |
|---|---|
| Readability | 35% |
| Visibility | 30% |
| Sentiment | 20% |
| Competitor pressure (inverted) | 15% |
Visibility projections are modeled, not measured, and are always labeled as projected.
06 · Live answers
Real questions, real engines
The prompt bench sends each question to the engines' own APIs with web search turned on: ChatGPT through OpenAI with web search, Claude with its web search tool, Perplexity Sonar, and Gemini with Google Search grounding. Answers and their cited sources are saved and reused for 24 hours.
We mark a question as live-fetch when it carries a marker that pushes a model to fetch instead of answering from memory: a freshness marker such as a current year or "latest", a comparison such as "vs" or "alternatives to", a price or availability question, a request for real experience such as reviews, or local intent such as "near me". The marker is a label only and never feeds a score.
07 · Evidence rules
Saved, dated, re-checked
- Public examples on our homepage come from pinned, dated probe runs.
- Screenshots are captured separately and carry their own date.
- After a fix ships, we fetch the page again to verify it.
08 · Limits
What we do not claim
- AI engines' own verified-bot networks may be treated differently from our probe.
- A readable page is required for a citation. It does not guarantee one.
- We do not see inside an engine's ranking.
- A saved fetch describes the page on its probe date, not today.
FAQ
Common questions
- What is the two-fetch diff?
- We fetch the same URL twice: once as a browser with JavaScript on, rendered through Firecrawl, and once as each AI agent with no JavaScript. Then we compare HTTP status, response size, readable main text and passages.
- Which AI agents decide the verdict?
- The live agents: ChatGPT-User, Claude-User and Perplexity-User. Search crawlers tell us whether a page can be a candidate. GPTBot is a training comparator and never changes the verdict.
- Why is Googlebot not shown as Google's view?
- A probe cannot verify itself as Google. Sites that verify bots by IP may refuse our Googlebot fetch, so we do not present that result as what Google sees.
- What does a black bar on the redacted page mean?
- The passage was not found in the saved fetch. It does not mean the engine read and rejected it.
- Does a readable page guarantee a citation?
- No. A page the agent cannot read cannot be quoted, but a readable page can still go uncited. We do not see inside an engine's ranking.