CitableReport

How to do an AI visibility audit: a step-by-step checklist

A clipboard checklist with ticks, a pencil, a stopwatch and a robot

To do an AI visibility audit, you check six things in order: whether AI crawlers can get into your site, whether your text is there without JavaScript, whether your pages answer questions in their opening lines, whether your facts are specific and sourced, whether it is clear who is behind the site, and whether you are measuring any of it. You need a browser, a terminal and free tools from Google and Bing, and most steps take under an hour on a small site. This guide goes through each step, with what to look for and what to do about it, and ends with a checklist template you can download.

Key takeaways

  • You can audit a site's AI visibility yourself in six steps: crawler access, the raw page, openings and structure, facts and sources, identity, and measurement.
  • Do the steps in that order. If AI search crawlers are blocked, nothing after step 1 matters.
  • Test the firewall as well as robots.txt: request a page with an AI crawler's name and compare the result with a normal browser.
  • Most of the checks need only a browser, a terminal and free tools from Google and Bing.
  • Re-run the access checks after any change to your hosting, CDN or CMS, and the full audit every three months.

What do you need before you start?

Before you start, pick the pages you will test. Auditing every page is slow and rarely changes the findings; a handful of the right ones tells you nearly everything:

You also need a way to send a request from the command line. On a Mac or Linux, curl is already installed in Terminal. Windows 10 and 11 include it too: type curl.exe rather than curl, because in PowerShell plain curl runs a different command. Finally, have your analytics and Google Search Console logins ready for step 6. If you have never read what an AI visibility audit is, that guide explains why each layer matters.

The six steps of the audit in order: crawler access, the raw page, openings, facts and sources, identity, measurement. If step 1 fails, stop and fix it first.
The six steps. Each one only counts if the one before it passes.

Step 1: Can AI crawlers get into your site?

Step 1 is the one that decides everything else, because a page an AI crawler cannot fetch cannot be quoted. OpenAI is explicit about it: its crawler documentation says sites that opt out of OAI-SearchBot will not appear in ChatGPT search answers.

Read your robots.txt

Open /robots.txt on your domain and read it as a crawler would. Rules are grouped under User-agent lines, and there is one detail that catches people out: under the robots.txt standard, RFC 9309, a crawler that finds a group with its own name follows that group only and ignores the * group entirely. So a group that names PerplexityBot to block one folder also cancels every general rule you wrote for all crawlers.

Separate the crawlers that search from the ones that collect training data. Blocking GPTBot, ClaudeBot or CCBot keeps your content out of future training sets; it does not stop ChatGPT, Claude or Perplexity from citing you, provided their search crawlers are allowed.

A robots.txt file allowing OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot and Perplexity-User, and disallowing GPTBot, ClaudeBot, CCBot and Google-Extended.
One way to let AI search in and keep training out. Name the crawlers you mean; do not rely on the * group.

Test your firewall

robots.txt is a polite request; a firewall is a locked door, and it can be locked even when robots.txt says come in. Test it by asking for your home page twice, once with an AI crawler's name and once as an ordinary browser:

curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; OAI-SearchBot/1.0)" https://example.com/
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (Macintosh)" https://example.com/

If the browser gets 200 and the crawler gets 403 or 503, something is turning AI crawlers away by name. Repeat with PerplexityBot and Claude-SearchBot. Treat a block as a strong warning rather than proof: some firewalls let the real crawler through after checking where the request comes from. Either way, open your CDN or hosting dashboard and read its bot and AI crawler settings, because several now block AI crawlers by default.

Check the sitemap and the page tags

Make sure /sitemap.xml exists, lists your key pages and is named on a Sitemap: line in robots.txt. Then view the source of each key page and search for noindex, nosnippet and max-snippet. Any of these on a page you want quoted is a problem.

Step 2: Is your text in the raw HTML?

Your text needs to be in the HTML the server sends, because most AI crawlers read that and do not run JavaScript. Google can render scripts, but its own JavaScript guidance recommends server-side rendering or pre-rendering anyway, noting that "not all bots can run JavaScript."

To test a page, open it and choose View Page Source (Ctrl+U on Windows, Option+Command+U on a Mac). Do not use Inspect, which shows the page after scripts have run. Search the source for a sentence from the middle of the page. If you cannot find it, neither can an AI crawler.

On the left, a page with a headline, text and an image as a browser shows it. On the right, the same page's raw HTML: an empty div and a script tag.
A page built entirely in JavaScript, as you see it and as most AI crawlers receive it.

If the text is missing, the fix belongs with whoever built the site: server-side rendering, pre-rendering or static generation, depending on the framework. While you are in the source, check that each page has a rel="canonical" tag pointing to itself, and that its address loads directly over https without a chain of redirects.

Step 3: Do your pages answer the question in the first two sentences?

Assistants quote passages, so the two sentences under each headline do most of the work. A quick way to judge them: copy the first two sentences from each key page into a blank document, and read them without the rest of the page. Each should answer the headline on its own.

For example, imagine a bakery page headed "Do you make gluten-free cakes?" that opens: "At Rosie's, we believe everyone deserves cake." Nothing there can be quoted. Rewritten: "Yes. We make a gluten-free version of all eight of our celebration cakes, baked in a separate kitchen, with 48 hours' notice." Now an assistant has an answer it can lift and credit.

While you are reading, check that:

Step 4: Are your facts specific, sourced and current?

Take your three most important pages and highlight every claim. For each one, ask whether it has a number, a name or an example, and whether a reader could check where it came from. "We're one of the leading suppliers" fails both tests; "We have supplied 1,200 schools in Victoria since 2014" passes the first. This matters for AI answers in particular: a study published at KDD 2024 found that adding statistics, quotations and citations to content raised its visibility in AI-generated answers by up to 40%.

Then check dates. Articles should show when they were published or updated. Your sitemap's <lastmod> dates should change only when a page really does: Google says in its sitemap documentation that it uses the value only when it is consistently and verifiably accurate. A sitemap where every page shows today's date is telling crawlers nothing.

Step 5: Is it clear who is behind the site?

AI systems are more willing to cite a source they can identify. Check that your home page has Organization structured data with your name, address, logo and links to your profiles; that about and contact pages exist and are linked from every page; and that articles name an author who has a page of their own.

Then look for inconsistency, which is surprisingly common. Compare how your business name appears in your page titles, your footer, your structured data, your Google Business Profile and your social accounts. "Smith & Co", "Smith and Co Pty Ltd" and "SmithCo" may be obviously the same to you; to a model they are three possible companies. A useful test is to ask ChatGPT, Claude and Perplexity "What is [your business name]?" and see whether they describe you, someone else, or nothing.

Step 6: Could you tell if it is working?

Set up measurement before you change anything, or you will never know which fix helped. Three things are worth doing:

The AI visibility audit checklist

Here is every check from the six steps in one place. You can also download it as a spreadsheet template (CSV, opens in Excel, Numbers or Google Sheets), with columns for your result and notes.

1. Crawler access

2. The raw page

3. Openings and structure

4. Facts and sources

5. Identity

6. Measurement

How often should you do an AI visibility audit?

Do the full audit every three months, and the step 1 access checks whenever something changes underneath the site: a new host, a new CDN or firewall setting, a CMS migration or a redesign. Those are the moments crawlers get locked out without anyone noticing. The monthly questions from step 6 tell you in between whether anything has moved.

Is an audit for a brand different from one for a small business?

The steps are the same; the emphasis is not. A small business should spend most of its time on steps 1 to 3, because a single firewall setting or a slow opening paragraph can be the whole problem, and both can be fixed in an afternoon. A larger brand usually passes those and has more to gain from step 5 and from what happens off its own site: reviews, press coverage, Wikipedia and Wikidata entries, and consistency across several domains. Remember that robots.txt applies per host, so each subdomain needs checking on its own.

What if you would rather not do it by hand?

The six steps above take an afternoon for a small site. Our checker runs the same checks automatically, 40 of them across up to 20 pages, in about a minute, and shows the evidence for each result. The score and six checks are free; the full report with every fix costs US$9. Whichever way you do it, start with step 1.

See how your site does

Your AI Visibility Score and six of the 40 checks, free, in about a minute.

Free score in 60 seconds. No signup. Full report US$9.