When you hover the up arrow button a tooltip should say "This comment ADDS to the discussion" and on a down "This comment DOESNT ADD to the discussion". Too often I think people just click the arrows based on (1) the username (2) "oh ya I agree, I hate that too!".
Up/down voting should be an extension of the community's ability to assess whether someone's opinion is adding to the community thought process. We often forget that (I do myself).