If you've been watching logs the past few years, you know that LLM data scrapers care less about robot directives than the scummiest of scraper bots of yore.
Your choices are: 1) give up 2) spend your days trying to detect and block agents and IPs that are known LLMs 3) try to spoil the pot with generated junk or 4) make it easier for them to scrape
1) is the easiest and frankly - not to be nihilistic - the only logical move
Your choices are: 1) give up 2) spend your days trying to detect and block agents and IPs that are known LLMs 3) try to spoil the pot with generated junk or 4) make it easier for them to scrape
1) is the easiest and frankly - not to be nihilistic - the only logical move