zlacker

[parent] [thread] 0 comments
1. jeroen+(OP)[view] [source] 2022-02-18 13:06:07
Crawlers are often permitted to get more web visibility. I think the Web Archive also crawls the web as a browser sometimes, so they can't always distinguish between crawlers.

That said, with NoScript on I can just read the economist article, no banners or anything.

[go to top]