zlacker

> A Rama operation does not return a value to its caller. It emits values to its continuation. This is a critical distinction, as part of what makes Rama operations more general than functions is how they can emit multiple times, not emit at all, or emit asynchronously.

>>beders+(OP)
Which is obviously very similar to how transducers already work in Clojure, but they still lack some of concurrency options of reducers. Getting all this on a smart, distributed runtime seems very promising.

>>thom+C7
> Getting all this on a smart, distributed runtime seems very promising.

Hopefully it is.

This CPS article is the first of the Rama blog posts where it seemed like there might be something there. The earlier posts - "I built Twitter scale Twitter in 10Kloc" - were never really all that convincing. The thing they claimed to have built was too ambitious a claim.

>>mschae+0b
Oh I think there’s a lot of good stuff baked in there. The big idea downstream is that you have incrementally calculated, indexed data structures to query all the results of this fancy CPS logic. It’s all slightly esoteric even coming from a Clojure background but it ticks every box I want from a modern data platform, short of speaking SQL.

>>thom+C7
There is a library called tesser [1] (by Jepsen/Riemann author) that behaves like parallel transducers with more Clojure "native" syntax. With transducers, you have to use "comp," and with tesser, you use "->>" as you'd use with lazy functions.

Sadly, tesser is not advertised as it should; I find it much more flexible than transducers. E.g. you could parallelize tesser code over Spark/Hadoop cluster.

[1] https://github.com/aphyr/tesser

>>dig1+Wk
See the injest library as well, I've had pretty good experience with ita performance.

https://github.com/johnmn3/injest

I can squeeze more performance out of tesser, but injest gives me a surprising boost with very little ceremony most of the time.

>>dig1+Wk
As I understand it Tesser doesn't support ordering which makes it tricky for many types of logic. I'd like to be able to control exactly which parts of a pipeline are parallelisable without too much ceremony, but I can't say I've used Rama in anger or that it makes this ergonomic.