zlacker

>>DonHop+(OP)
The analogy helped me understand, but in factorio belts are required primarily because everything takes physical space. What are real world use cases for kafka? Is it primarily if your data is too much to simply store in a db?

>>chrisc+GC
A database is a huge API to share with another team, and queries from other people add load with a principle-agent problem [1]

Give people a firehouse of events or deltas, though, and if they want to query it, performance is now their problem - they build the database, update it, index it, etc.

This is part of scaling organisationally, not just removing the database as a bottleneck.

Events are also a route to making cache invalidation - one of the hardest problems in CS, as we know - tractable. Build your caches as services that consume the firehouse, and invalidate based on a join between events and the cached data.

[1] https://en.m.wikipedia.org/wiki/Principal%E2%80%93agent_prob...