zlacker

Really? I disagree. Running a ubiquitous service that is good for users does not require that you allow every random person to scrape your site and incur that cost.

You simply setup API deals with those who you want to have your data, those that benefit your business, aka Google etc…

Then you close everything else up. This saves cost and complexity and real users, the target of your advertisers, don’t even notice.

This isn’t a sign that engineering is slipping.

It’s a sign that a in a company which struggles to make money, someone is paying attention and trying new things to fix the money problem.

>>shon+(OP)
> Running a ubiquitous service that is good for users does not require that you allow every random person to scrape your site and incur that cost.

What I said is that they must handle the problem transparently to their valuable users. That includes (requires, usually) targeted techniques to block high-volume scraping.