Websites claim that AI startup Anthropic is circumventing their anti-scraping regulations and protocols.
A freelancer has accused Anthropic, the AI startup responsible for the Claude large language models, of disregarding the “do not crawl” robots.txt protocol to scrape data from its websites. Additionally, iFixit CEO Kyle Wiens stated that Anthropic has violated the website’s policy against using its content for AI model training. Freelancer CEO Matt Barrie described Anthropic’s ClaudeBot as the most aggressive scraper, with the company’s crawler allegedly generating 3.5 million visits to his website in just four hours. Wiens also reported that Anthropic’s bot accessed iFixit’s servers a million times…
Read More