This approach may work if there is additional points given by the system to the upvoter (or downvoter) if there is significant number of other users doing the same thing as that upvoter (or downvoter) too, which means his/her action is indeed valid and objective.
Users would then be more willing to vote, with the hope that other users will do the same thing as him/her too, and reap more points than what he/she spent for upvoting/downvoting. This enforces more thinking and evaluation of comments or stories before even upvoting or downvoting them.