New York is set to become the first state to impose guardrails on “stealth crawlers,” or unauthorized software that trawls news sources to scrape content.
I hope those tools are exempt because, just like a browser, they respond only to specific commands issued by a human user. They don’t “crawl” pages in the way we describe bots that jump from page to page.
Soooo, that means archive.is too. This fucking sucks.
I hope those tools are exempt because, just like a browser, they respond only to specific commands issued by a human user. They don’t “crawl” pages in the way we describe bots that jump from page to page.
Isn’t there something like 340 news sites that actively block them already?