zlacker

[return to "Google to explore alternatives to robots.txt"]
1. pentag+s41[view] [source] 2023-07-08 16:10:28
>>skille+(OP)
The other day I was trying to search a comment of a YouTube video that I remembered, but wasn't able to find it with the google search "site:youtube.com [phrase of the comment]", later to find out that YouTube disallows search engines to index comments trough robots.txt https://www.youtube.com/robots.txt

>Disallow: /comment

So I guess that works for them.

◧◩
2. Distra+Qd1[view] [source] 2023-07-08 17:02:33
>>pentag+s41
In fairness, I'd expect YouTube comments to largely be noise and not worth indexing.
[go to top]