Short answer is HN is basically deal flow for YC. To that goal it obviously works very well.
PG and YC need "everybody" but yet they need "nobody".
As a business HN works to achieve the greater goals of YC. I don't think it is about anything else and I haven't been around long enough to know if things were ever different. They may have been back at the start because I guess if you were to arbitrary you would never build up an audience or fan base.
On a personal level I've always thought that people who get up to the karma level you are at actually deserve some special treatment because to me the top commenters are some of the glue that makes a place like HN special. In any business I have ever run you always give special treatment to the better customers.