So, why keep returning search results that the end user can't use without registration or purchase? It's essentially "page cloaking" when the rendered page doesn't match google sees.
To me, if you want a paywall, that should come with the consequence that your site isn't included in search results for the general public.
Edit: It's also getting irritating here on HN. I might have a subscription or login to one or two sites, but HN regularly shares stuff from Medium, WSJ, NYT, Wired, and so on. I have to imagine that most people following these posted stories hit the reg/pay-wall.
why does google allow this? as you say it is 100% cloaking to have the entire article indexed but not present it in the subsequent page.
Sure, publishers feel they need paywalls for revenue purposes; have at it. That should not absolve them from the "rules" everyone else has to follow.
Cloaking refers to the practice of presenting different content or URLs to human users and search engines. Cloaking is considered a violation of Google's Webmaster Guidelines because it provides our users with different results than they expected. [0]
[0] - https://developers.google.com/search/docs/advanced/guideline...
It is really frustrating as a user, and undoubtedly Google knows this. So an impending lawsuit is the only reason I can see for them not blocking nyt/other sites that do this.