Cloudflare has introduced a new default setting that blocks AI web crawlers from accessing its clients' websites unless they obtain permission or pay a fee. This initiative, called "Pay per Crawl," establishes a marketplace where website owners, including publishers and content creators, can charge AI companies for scraping their content. The move aims to give creators greater control over their digital content and generate revenue from AI firms that use web data for training and other purposes. Cloudflare's CEO, Matthew Prince, emphasized that this permission-based approach seeks to balance empowering content creators while supporting AI innovation. The company, which powers nearly 20% of the internet, is the first major internet infrastructure provider to implement such a system by default. The Pay per Crawl model uses HTTP 402 payment requests and integrates with Cloudflare's billing system for periodic settlements. Major publishers like Condé Nast have expressed support for the new model. Cloudflare's initiative responds to growing concerns about AI companies scraping content without compensation, potentially reshaping how AI models access and use web data. The system also offers manual controls to whitelist or ban specific AI bots and is designed to facilitate good-faith negotiations between AI firms and website owners. This development has been described as a potential turning point for content monetization and data rights in the AI era.