Cloudflare has introduced a new default setting that blocks AI web crawlers from accessing its clients' websites unless explicit permission is granted or a fee is paid. The company launched a marketplace feature called "Pay per Crawl," enabling website owners, including publishers and content creators, to charge AI bots for scraping their content. This initiative aims to give creators more control over their content and establish a revenue stream amid concerns over AI companies scraping data without compensation. The system uses HTTP 402 payment requests combined with Cloudflare billing for settlements. Major publishers such as Condé Nast support this permission-based model. Cloudflare, which powers nearly 20% of the internet, positions this move as a step toward a more sustainable web ecosystem that balances content creators' rights with AI innovation. The new policy affects all new websites set up with Cloudflare and is designed to facilitate negotiations between AI companies and website owners, while also leveraging Cloudflare's experience in blocking unauthorized bots. This development has been described as potentially reshaping how AI models access web content and training data.