zlacker

[parent] [thread] 3 comments
1. vosper+(OP)[view] [source] 2023-07-24 02:05:28
Thanks, do you know if it's a good idea to use JetStream's KV or Object stores to implement a join? I'm thinking of "fan-in", where multiple streams carrying different pieces of data about the same object all meet in the same place, and the output is a unified object carrying all those bits of data. Seems the store could be used by the joining consumer to hold onto the various pieces of data related to an object until it has the full set, then emit the object into the next stream?
replies(2): >>yes299+A2 >>protoc+69
2. yes299+A2[view] [source] 2023-07-24 02:24:15
>>vosper+(OP)
At this point, where you are doing KSQL level stuff, isn't it better to just use Kafka, NATS is very appealing for it's simplicity but if I reach a point of re-implementing KSQL or spending 3 hours setting up zookeeper and understanding how to do Kafka distributed for a devops point then Im choosing the latter
replies(1): >>vosper+ow
3. protoc+69[view] [source] 2023-07-24 03:29:15
>>vosper+(OP)
You could do something in that vein, but it might be easier to have those streams all actually deliver to a set of consumers and write a bit of code to join them back together. https://natsbyexample.com/examples/jetstream/multi-stream-co... has an example (also a great resource for learning about NATS use cases!)
◧◩
4. vosper+ow[view] [source] [discussion] 2023-07-24 07:35:41
>>yes299+A2
Yes you may well be right, that's kind of what I'm wondering (not using either Kafka or NATS today). Correct me if I'm wrong, but Kafka itself doesn't do joins either, right? I would need something like Flink's Window Join?

https://nightlies.apache.org/flink/flink-docs-master/docs/de...

[go to top]