How something could be well utilized, versus how it is utilized, may be two different things.
There's an obvious conflict of interest in reducing another user's karma risks their ability to also downvote. (Of course, just because it's imperfect doesn't mean it's a bad system.) If this is the intended usage then perhaps it would be fair to expand downvoting rights to other users?