zlacker

[return to "Show HN: WakaQ - a Python distributed task queue"]
1. zamuba+UL[view] [source] 2022-09-06 04:54:38
>>welder+(OP)
Why not just use Redis streams for the queue over the Redis list impl.?
◧◩
2. welder+yN[view] [source] 2022-09-06 05:08:22
>>zamuba+UL
New: Streams are append only, but I push eta tasks onto the front of a queue when their time is ready to run. Looks like streams could work and might even be better suited instead of lists, but I would have to figure out some way to run eta tasks before other pending tasks in a queue.

Old when I confused streams with pubsub: Streams aren't durable... a worker goes offline for a second and all of a sudden you lost all your pending tasks. Redis lists are durable, so you can add tasks first then boot up a worker later and the worker will process those waiting tasks. With streams, the worker wouldn't see any pending tasks when it boots.

[go to top]