zlacker

>>evanwe+(OP)
2010: "At any moment, Justin Bieber uses 3% of our infrastructure. Racks of servers are dedicated to him"

https://gizmodo.com/5632095/justin-bieber-has-dedicated-serv...

>>molecu+qx
Bieber-related conversation might reasonably have been 3%. Updating his friend count was a constant source of lock contention on a particular mysql.

There were never explicitly dedicated databases/servers, unless you count the hotspot caused by whatever shard he hashed into.

>>fizx+Lx
You would think a HyperLogLog would be approximately good enough when follower counts reach tens of millions.

>>lwansb+sz
Hyperloglog was only just invented, and hadn't really made its way into the popular developer consciousness.

Anyhow, the solution is easier than that (implement lock striping, i.e. https://stackoverflow.com/questions/16151606/need-simple-exp...), but I was reorg'd away before my code hit production, so I'm not sure what happened there.

>>fizx+qB
Approximate counters are much much older than HyperLogLog. Here’s something from 1978: https://www.inf.ed.ac.uk/teaching/courses/exc/reading/morris.... Of course this solves a different problem because HyperLogLog is an, uh, interesting way to count followers (why would you need to do a count-distinct query? You can’t follow someone multiple times). In any case, the Flajolet Martin sketch dates back to 1985 and solves the same problem as HyperLogLog: https://en.wikipedia.org/wiki/Flajolet%E2%80%93Martin_algori...

>>lorenz+sE
Here is nice presentation covering those: (link: https://www.cs.princeton.edu/~rs/talks/AC11-Cardinality.pdf)