These are metrics one might use even if there’s no algorithm, in fact historically they have. TV shows used to use Neilsen data for similar purposes long before there was YouTube. TV producers would measure audience dropoff and then use that to help writers write more gripping episodes.
Google’s hope with their search for decades was that their algorithm was ungameable and that the way to get your site to the top of any result was to make it the best. That’s why they made it a black box and changed it whenever SEO caught on and used it to push junk to the top.
That’s had mixed results on the web for sure but it’s probably worked much better with video because you can track these metrics in a way you can’t with text. Also with the web, the page you land on may make Google further money (with ad sense, inspiring more Googling, using a Google product directly, etc.) or it may not, they don’t always own the ad service at wherever you land when you click a search result link. They don’t have the pure financial incentive of just showing you what you want, something you want a little less might make them more money.
With YouTube they own it all. The more you watch YouTube the more they make. You’re only clicking ads to other YouTube videos.
Everybody on YouTube knows you want a compelling lead in to get the click over to your video, a hook to keep them watching, etc. He’s codifying what they all already know and do. He just is better at it.