zlacker

Any thoughts on NATS' JetStream? Looks quite compelling, to me.

https://docs.nats.io/using-nats/developer/develop_jetstream

>>vosper+(OP)
Jetstream is the Kafka or Kinesis bit built out on top of the NATS core protocol. It's much easier to work with than either of them in my experience

>>protoc+ch
Thanks, do you know if it's a good idea to use JetStream's KV or Object stores to implement a join? I'm thinking of "fan-in", where multiple streams carrying different pieces of data about the same object all meet in the same place, and the output is a unified object carrying all those bits of data. Seems the store could be used by the joining consumer to hold onto the various pieces of data related to an object until it has the full set, then emit the object into the next stream?

>>vosper+CB
At this point, where you are doing KSQL level stuff, isn't it better to just use Kafka, NATS is very appealing for it's simplicity but if I reach a point of re-implementing KSQL or spending 3 hours setting up zookeeper and understanding how to do Kafka distributed for a devops point then Im choosing the latter

>>vosper+CB
You could do something in that vein, but it might be easier to have those streams all actually deliver to a set of consumers and write a bit of code to join them back together. https://natsbyexample.com/examples/jetstream/multi-stream-co... has an example (also a great resource for learning about NATS use cases!)

>>yes299+cE
Yes you may well be right, that's kind of what I'm wondering (not using either Kafka or NATS today). Correct me if I'm wrong, but Kafka itself doesn't do joins either, right? I would need something like Flink's Window Join?

https://nightlies.apache.org/flink/flink-docs-master/docs/de...

>>vosper+(OP)
Jetstream (includes Streams (better as in Kafka IMO), a powerful KV Store and also an Object Store to store files) is really powerful. It earns much more attention than it currently has.

Jetstream is also much easier to operate than Kafka IMO. Just a simple single binary with an easy to understand config.