zlacker

[parent] [thread] 1 comments
1. njharm+(OP)[view] [source] 2017-09-20 18:15:56
> by selecting primed jobs in bulk and feeding them into another store like Redis

Doesn't this just mean bunch of lost jobs when redis fails.

Why not keep jobs with job state wait, done, etc in the reliable ACID store.

replies(1): >>brandu+tb
2. brandu+tb[view] [source] 2017-09-20 19:32:34
>>njharm+(OP)
> Doesn't this just mean bunch of lost jobs when redis fails.

You need to be a little careful about configuration, but believe it or not, as long as Redis is configured with an append only file [1] it will provide you with a durability guarantee just like Postgres would. If it crashes, any jobs that were in there are recovered the next time it starts up.

From the docs:

> From now on, every time Redis receives a command that changes the dataset (e.g. SET) it will append it to the AOF. When you restart Redis it will re-play the AOF to rebuild the state.

---

[1] https://redis.io/topics/persistence

[go to top]