zlacker

>>mpweih+(OP)
Does anyone know a good, low overhead out-of-process message queue, that's lightweight enough that it can be useful for communicating between processes on the same machine, but if necessary it can scale beyond it? In case of a single-machine product that comprises of several services, a message queue can sometimes be useful for pull model, but adding RabbitMQ to the stack makes installation and ops much more complex than customers deem acceptable.

I know some people use Akka with Persistence module, but I would welcome other alternatives.

>>polski+BF
"Low overhead"? Redis.

Reliable and durable? Emphatically not Redis, and at that point you probably want to just start looking at SQS.

(I've had good luck shipping products as docker-compose files, these days. Even to fairly backwards clients.)

>>eroppl+OJ
Can you explain why Redis is not durable? Looking into it for a project but this comment worries me.

>>tjanks+RL
In my experience Redis is plenty durable (especially in Elasticache with replicas + multi-region + backups). Redis _is_ an in-memory data store, so if the server crashes, you lose the data, but if you have replicas it'll fail over, if you have backups you can restore, and if you have multi-region you can failover to the other region. IMO, the idea that Redis is not durable enough is outdated.

It's something to be aware of and to have backup plans for, but we've been using Redis as our primary datastores for over a year with only one or two instance failures which were quickly resolved within minutes by failing over to replicas, with no data loss.

>>Jemacl+eY
Redis won't lose data from just a server crash. It has persistence with streaming AOF mode and snapshot RDB mode, and they can be combined. fsync is also configurable. It can be set to write every operation to disk but most set it to every 1 second which is safe enough with replicas.