Anubis is a protective measure implemented by website administrators to prevent AI companies from scraping content. It employs a Proof-of-Work scheme to increase the cost of scraping while aiming to improve the identification of headless browsers. Users may need to disable certain plugins to access the site properly.

The Wikimedia Foundation reports a 50% increase in bandwidth consumption due to web-scraping bots that are primarily used to train AI models, leading to significant costs for the organization. With 65% of traffic for expensive content generated by these bots, the Foundation aims to reduce scraper traffic by 20% and prioritize human users in its resource allocation. Concerns about aggressive AI crawlers have prompted discussions about implementing better protective measures, although current methods, such as robots.txt directives, are often ineffective.

The article discusses the challenges of dealing with aggressive data-scraping bots that collect information to train large language models (LLMs). It explores various strategies for mitigating their impact, such as serving them dynamic content or "garbage," which can be more efficient and cheaper than traditional anti-bot measures like blocking IPs or implementing paywalls. Ultimately, the author concludes that feeding these bots nonsensical data is a practical solution to manage server traffic without incurring significant costs.