zlacker

>>mpweih+(OP)
Does anyone know a good, low overhead out-of-process message queue, that's lightweight enough that it can be useful for communicating between processes on the same machine, but if necessary it can scale beyond it? In case of a single-machine product that comprises of several services, a message queue can sometimes be useful for pull model, but adding RabbitMQ to the stack makes installation and ops much more complex than customers deem acceptable.

I know some people use Akka with Persistence module, but I would welcome other alternatives.

>>polski+BF
"Low overhead"? Redis.

Reliable and durable? Emphatically not Redis, and at that point you probably want to just start looking at SQS.

(I've had good luck shipping products as docker-compose files, these days. Even to fairly backwards clients.)

>>eroppl+OJ
Can you explain why Redis is not durable? Looking into it for a project but this comment worries me.

>>tjanks+RL
Intentionally so. It's not a deficiency or a footgun, it's a design decision to be aware of. Redis is an in-memory database first.

You can configure Redis for durability. The docs[1] page for persistence has a good examination of the pros and cons.

[1]: https://redis.io/topics/persistence

>>chucks+lV
antirez put a lot of work in around Redis 3.0 (iirc) to make the persistence reliable and strong. As long as your server is configured and used correctly (obviously a large caveat, but you can only hold someone's hand so much), I don't think there is any reason to doubt Redis's persistence anymore.

It's important to make this distinction because there are commonly-used systems that offer a best-effort style persistence that usually works fine, but explicitly warn developers not to trust it, and developers rarely understand that.

We badly need to get better at distinguishing between true, production-level data integrity and "probably fine".