zlacker

>>benhur+(OP)
If you've got an automated vetting process with a 99.999% success rate, but are dealing with billions of accounts, that's still tens of thousands of false positives.

At that level, "percentage" is an insufficient measure. You want "permillionage", or maybe more colloquially "DPM" for "Defects Per Million" or even "DPB".

You'll still get false positives though, so you provide an appeal process. But what's to prevent the bad actors from abusing the appeal process while leaving your more clueless legitimate users lost in the dust?

(As the joke goes: "There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists" [1])

Can you build any vetting process, and associated appeal process, that successfully keeps all the bad actors out, and doesn't exclude your good users? What about those on the edge? Or those that switch? Or those who are busy, or wary?

There's a lot of money riding on that.

[1] https://www.schneier.com/blog/archives/2006/08/security_is_a...

>>AceJoh+YM1
The problem with unjustified bans due to some algorithm is also: These cases might not even be a close calls like: “oh yeah this person did something that is in the grey area of what our policies state. I will ban him but he might interpret things differently.”

No if you enforce your policies strictly by (machine learning) algorithms it could just be a matter of misinterpreting a different language, slang, irony or something else. Which makes these bans even more infuriating.