After submitting a bunch of links, some of them making a front page, I noticed that there is a fairly strong effect of social proof.
People check /newest, see that some post already has 1 or 2 upvotes, check it instead of some without any upvotes (someone upvoted, might be good!). The upvoted one gets even more upvotes (because more people are reading it), and it's on the homepage.
A bunch of my submissions made the homepage and from what I've noticed, the threshold is about 7-10 upvotes in the first hour. So can we fairly say that a dozen of people decide what's on the homepage? Maybe.
While the sample size is really really small, 8 of 9 links I submitted and got more than 4 upvotes, made the front page. But I guess with a greater sample size, the general assumption would still hold true.
The obvious disadvantage of hiding score is that it's harder to tell what is worth attention (especially during peak hours when there is a new submission every minute), but maybe it would help to bring more good content to the front page (as opposed to content that a dozen of people thought is good).