zlacker

>>ColinW+(OP)
How does WEI work with non-browsers, like curl or python requests? I was wondering if there is some motive here to monopolize web scraping (especially with respect to harvesting AI training data)?

>>andy99+cw
I mean that’s part of the point. It’s there to exactly lock out scrapers. Or crawlers, for that matter. What a happy little coincidence.

>>AndroT+jy
This is just silly, there exist frameworks like selenium that allow you to run any browser of choice and emulate actual user behavior(clicks, keystrokes). If they go further the emulation layer will have to be moved higher, above the virtual machine running the browser for example. The truth is, this has nothing to do with scraping, scrapers will find a way. This is to stop the majority of people from using ad block.