zlacker

>>ColinW+(OP)
How does WEI work with non-browsers, like curl or python requests? I was wondering if there is some motive here to monopolize web scraping (especially with respect to harvesting AI training data)?

>>andy99+cw
I mean that’s part of the point. It’s there to exactly lock out scrapers. Or crawlers, for that matter. What a happy little coincidence.

>>AndroT+jy
This is just silly, there exist frameworks like selenium that allow you to run any browser of choice and emulate actual user behavior(clicks, keystrokes). If they go further the emulation layer will have to be moved higher, above the virtual machine running the browser for example. The truth is, this has nothing to do with scraping, scrapers will find a way. This is to stop the majority of people from using ad block.

>>Roark6+ez
>If they go further the emulation layer will have to be moved higher, above the virtual machine running the browser for example.

Your hypothetical change of emulation tactics won't work. You're analyzing at the wrong abstraction level.

The "attestation tokens" to validate the integrity of the web browser environment would come from a 3rd-party (e.g. Google Play services).

For example... Today, hacks like youtube-dl work because implementing client-side code to "solve javascript puzzle challenges" is still inside the "world" that Google-server-to-browser-client present to each other. Same for client-side solvers for Cloudflare captchas. The "3rd-party attestation token" breaks those types of hacks.