zlacker

>>monort+(OP)
Does this respect the 'unlisted' setting for a video? I recently uploaded some videos and set them all to unlisted, and yet some of them received views despite me not viewing them or handing out a link. I meant to dig into that more but forgot after getting distracted. Can unlisted videos be found by a program like this, which I assume is using the API?

>>chrsst+Qc1
We [0] index YouTube actively and see way over 5.5B videos [1] at this point. We catch a lot of unlisted videos and we did try to figure out how is that possible in the past.

It seems that a lot of users will upload video which is by default published with the default settings and thus is visible from the outside. Even if they change the settings fairly quickly, automated systems like ours will already know about the existence of that video.

There could be other reasons but this seems the most likely, especially as a video that is being uploaded can be published fairly swiftly.

[0] https://pex.com

[1] https://blog.pex.com/what-content-dominates-on-youtube-39081...

>>doh+bv1
It sounds like you are aware that you are scraping videos that are later re-labeled as "unlisted", but you don't mention what you do to mitigate this problem.

Even if it may not be illegal, at the very least it would seem un-ethical to link to private videos like this, and it would seem trivial for you to "re-scrape" your database every now and then to check whether any existing videos have changed from listed -> unlisted, and if they have, remove them.

>>hnarn+vy1
This logic would require them to re-scrape every video forever, which is unreasonable.

I think a better approach for everyone involved would be to only store references to videos which were posted more than x minutes ago. I'm not sure if they have that information when scraping though.