Developer checks AI crawlers against their own site: all eight returned 200

On 30 September 2026, a developer ran an AI crawler access check on his own site. The test works by requesting a page under each bot's name and reading the status code that comes back, which shows whether the crawlers behind ChatGPT, Claude, Perplexity and similar services can fetch the pages at all. Eight crawler names were tried, and all eight requests returned 200. Nothing was keeping the machines out.

The account was published on 3 October 2026 in the AI section of DEV Community; the author notes it first appeared on yappatyih.com. The write-up leads with the result: eight crawler names, eight answers of 200.

What the same audit did find sat elsewhere on the site. A single line in robots.txt — Disallow: /og/ — was blocking the preview image on every page. So the one problem the check surfaced was not about crawler access at all, but about robots.txt shielding the per-page preview images.

The piece is framed as a self-check developers can run on their own sites: probe the site with each bot's name, read the status codes, and see whether AI crawlers can reach the pages. The reported outcome of this particular pass is two-part: unrestricted access for all eight crawler names, and a robots.txt rule that hides the preview images.