Checked for new stories 9m ago

Updates on Robots Txt

Every AI story we track on Robots Txt — 2 stories so far, each summarized in our own words and linked back to the publisher that reported it.

Pulled from 123 sources

This month

AI Research8 min read

Only 8.9% of sites block AI crawlers, but 94.8% are never cited in AI answers

Hacker News
Social Media3 min read

Patreon stops asking AI bots not to scrape — and starts blocking them

TechCrunch
That's everything we have on Robots Txt right now