zlacker

>>evanwe+(OP)
I haven't seen anyone touch on this, but I remember reading about this in Data Intensive Applications[1]. The way that they solved the celebrity feed issue was to decouple users with high amounts of followers from normal users.

Here is a quick excerpt, this book is filled to the brim with these gems.

> The final twist of the Twitter anecdote: now that approach 2 is robustly implemented,Twitter is moving to a hybrid of both approaches. Most users’ tweets continue to be fanned out to home timelines at the time when they are posted, but a small number of users with a very large number of followers (i.e., celebrities) are excepted from this fan-out. Tweets from any celebrities that a user may follow are fetched separately and merged with that user’s home timeline when it is read, like in approach 1. This hybrid approach is able to deliver consistently good performance.

Approach 1 is a global collection of tweets, the tweets are discovered and merged in that order.

Approach 2 involves posting a tweet from each user into each follower's timeline, with a cache similar to how a mailbox would work.

[1] https://www.amazon.com/Designing-Data-Intensive-Applications...

>>collin+LG
It’s an oft overlooked inequality in these systems. People get so wrapped up in some whiz bang thing that they don’t stop to think if they should.

At the end of the day, one of the most important aspects of your information architecture is how many times is each write to the system actually observed? That answer can dictate a lot about your best processing strategies.