zlacker

[parent] [thread] 1 comments
1. alecr9+(OP)[view] [source] 2022-09-07 21:24:30
> TaskTiger only processes one

Generally yes, though it also supports "batching" tasks where tasks are `.delay`'d normally but the task is given batches of N sets of args at a time and the task's code has to understand batching logic.

:+1: I'd definitely believe the task handling code is slower than celery. Honestly I haven't personally done too much optimization work on it (though perhaps previous maintainers have, I'm not sure). Performance of TaskTiger itself hasn't really been problematic at all for us in production with hundreds of workers. Usually it's been external things (like databases, third party API's, etc) that impact our task throughput the most. Or wanting to shift the architecture entirely to avoid using memory-bound Redis as a queue with an overflow risk.

replies(1): >>welder+f3
2. welder+f3[view] [source] 2022-09-07 21:42:32
>>alecr9+(OP)
> Or wanting to shift the architecture entirely to avoid using memory-bound Redis as a queue with an overflow risk.

I wanted to use SSDB[1] instead of Redis for that reason, but it doesn't support the necessary data structures.

[1] https://github.com/ideawu/ssdb

[go to top]