zlacker

>>pul+(OP)
Must be something about Kafka to attract these kind of explanations. Another one few months back was a children's book on Kafka [1] . For me it just look like solution looking for actual problems.

I wonder if Kafka represents an existential angst in these Kubernetized Microservice times. Or is it more simply I am just too dumb to learn and use this shit correctly.

1. https://news.ycombinator.com/item?id=27541339

>>geodel+nkb
When you are wondering whether you might need Kafka, it is certain that you don't need it.

But there are times when you have a problem, and amongst the possible solutions is Kafka.

I've come across Kafkaesque problems only three times in the last seven years: a hosting platform that had to parse logs of over 700 WordPress sites for security and other businesslogic. Putting all events of a financial app backend into datalakes and filtering and parsing all openstreetmap changesets live.

>>berkes+yob
I work in the oil and gas industry where legacy systems runs on their last breath. Kafka is a fantastic tool and solves a shit ton of problem. We have millions of sensors on an offshore installation, these all send data into kafka, where we generate events on new topics from different timeseries. Other data services consume these topics and get data updated in near realtime.

No more daily SQL dumps from offshore to onshore and big batch procedures to genereate outdated events.

>>olavgg+ECb
I'm in the same situation in the paper making industry. Kafka is an almost perfect match for our needs: high volume, durable storage, decoupled stream processing.