How AI search reads your website

Research, guides and creatives on generative engine optimisation (GEO) and answer engine optimisation (AEO). Use them to learn, or share them with the people who run your website.

Research

Our own studies, with the method and the data behind every figure.

Guides

One question each, answered properly.

Creatives

Ready to share with your team or post as they are, each with its source.

Stat card: the split between OpenAI's training crawler and its search crawler, with the study's figure.
Blocking GPTBot doesn't block ChatGPT search Stat card Download PNGFrom: Who lets AI in? robots.txt at the world's top 1,000 websites
Stat card: how many llms.txt files point AI at a page that no longer exists.
Dead links in llms.txt files Stat card Download PNGFrom: Readable to AI? What a crawler finds on 2,000 homepages
Stat card: the share of top homepages that send little or none of their text without JavaScript.
How many homepages need JavaScript for their text Stat card Download PNGFrom: Readable to AI? What a crawler finds on 2,000 homepages
Stat card: the share of homepages with Organization markup, and how many link their other profiles.
How many homepages say who they are Stat card Download PNGFrom: Readable to AI? What a crawler finds on 2,000 homepages
Stat card: the share of top websites blocking AI training crawlers, against those blocking AI search.
How many top sites block AI training Stat card Download PNGFrom: Who lets AI in? robots.txt at the world's top 1,000 websites
A seven-slide carousel of what a crawler finds on 2,000 homepages, with a checklist to finish.
Readable to AI? The homepage study in seven slides Carousel7 slides Download PDFFrom: Readable to AI? What a crawler finds on 2,000 homepages
Checklist card: five things to check in your robots.txt and CDN so AI search can reach you.
The five-minute robots.txt check Checklist Download PNGFrom: Letting AI crawlers in: robots.txt for the bots that matter
Checklist card: six things to check on your homepage so AI crawlers can read and recognise it.
The homepage AI-readiness check Checklist Download PNGFrom: Readable to AI? What a crawler finds on 2,000 homepages
Illustration: a homepage in a browser beside the same page read without JavaScript, with the study's figure.
What you see against what a crawler sees Illustration Download PNGFrom: Readable to AI? What a crawler finds on 2,000 homepages
A seven-slide carousel of the robots.txt study's findings, with a five-minute check to finish.
Who lets AI in? The study in seven slides Carousel7 slides Download PDFFrom: Who lets AI in? robots.txt at the world's top 1,000 websites

See what an answer engine sees on your site

Run an audit and get your scores, the evidence behind them and the files to fix them.

Start your audit