AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt - Ars Technica - ervin's web review

4172 shaares

4172 shaares

Filters

Links per page

20 50 100

AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt - Ars Technica

Maybe at some point the big providers will get the message and their scrapers will finally respect robots.txt? Let's hope so.

tech · ai · machine-learning · gpt · criticism

January 29, 2025 at 9:28:40 AM GMT+1 * · permalink

·

·

https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and-trick-ai-scrapers-that-ignore-robots-txt/

·

Filters

Links per page

20 50 100