zlacker

You only present users with the bare number of options to establish the ranking. "Was your most recent purchase better or worse than purchase B? Was it better or worse than purchase F?" If the software keeps an up to date ranking, you don't have to ask more than a handful of questions to accurately assess where the new entry goes.

If an entry causes a logical fallacy, that is an opportunity to represent the data in a different order and see if the user changes their ranking. This will actually help to keep the data fresh. And you can retain "fuzzy" rankings in certain areas without threat to the accuracy of the overall database.

If you want to have a multivariable structure, users could rank more than facet at once. So for a car, you could compare if a Honda Civic is better or worse than a Toyota Corolla on handling, comfort, features, etc. Combine this with non-subjective data (price, 0-60, etc) and users can choose if they want an aggregate ranking or weighted based on their criteria.