Navigation
Search
|
Perplexity is Using Stealth, Undeclared Crawlers To Evade Website No-Crawl Directives, Cloudflare Says
Monday August 4, 2025. 05:20 PM , from Slashdot
![]() Cloudflare engineers tested the behavior by creating new domains with robots.txt files prohibiting all automated access. Despite the restrictions, Perplexity provided detailed information about the protected content when queried, while the stealth crawler generated 3-6 million daily requests across tens of thousands of domains. The undeclared crawler rotated through multiple IP addresses and network providers to evade detection. Read more of this story at Slashdot.
https://tech.slashdot.org/story/25/08/04/1459240/perplexity-is-using-stealth-undeclared-crawlers-to-...
Related News |
25 sources
Current Date
Aug, Wed 6 - 08:45 CEST
|