zlacker

"You can bet that the Google crawler will have its own "attestation token" but if you want to crawl the Web with your own code you'll be SOL."

Let's be real here and note that while most web properties welcome Google crawlers, there are many, many other scrapers/crawlers that offer zero value to web operators while costing resources.

>>llm_ne+(OP)
Ok... but the result of fixing that by whitelisting only Google is a de facto full-on monopoly for Google search.

>>saurik+uf
That isn't remotely what I said, now is it? The classic downvote/snark reply is such a rote cliche here.