Not exactly, you would have some capital to start. If you start with 50 points you get 50 upvotes or 25 downvotes, plus it evolves every day from your visits and the upvotes you received.
The main point is to force people to be more cautious with their upvotes and downvotes. My theory is that if you have to pay from your reputation (your karma), you'll mainly upvote/downvote what you're really convinced about.