Really bad comments are not the root of the problem. Simply having large number of mediocre comments crowds out and discourages thoughtful discussion from starting at all.
I'd say:
* create some real cost to making comments
* make bad comments disappear/not display at all with time
* make things less democratic -- to encourage good behavior identify users who have this behavior and make this behavior more prominent programmaticly
I used to participate in the comments because the conversations were stimulating and the community was small. The community's too large for everyone to talk now, but HN has been the best tool for the intellectually curious to date and that doesn't have to change. The bar for commenting just needs to be higher.
When I read the top comments nowadays I'm expecting them to be written by:
- the author of the submitted article - the subject(s) of the article - employees or close relations to the subject(s) of the article - experts in the subject matter
At the bottom I expect to find comments such as product feedback or links to the print version of the article and minor but useful stuff like that. Smart people with interesting things to say shouldn't leave comments here--the community's too big for that now.
I'm not sure how you programmatically enforce that. It might be as simple as changing the commenting policies and have the users adjust their self policing.