zlacker

[parent] [thread] 2 comments
1. qaq+(OP)[view] [source] 2023-11-20 17:26:30
Postgres based job queues work fine if you have say 10K transaction per second and jobs on average do not take significant time to complete (things will run fine on fairly modest instance). They also give guarantees that traditional job queues do not.
replies(1): >>Rapzid+Iw2
2. Rapzid+Iw2[view] [source] 2023-11-21 08:04:01
>>qaq+(OP)
Probably order of magnitude more or perhaps a multiple of that depending on the hardware and design.

In theory an append-only and/or HOT strategy leaning on Postgres just ripping through moderate sized in-mem lists could be incredibly fast. Design would be more complicated and perhaps use case dependent but I bet could be done.

replies(1): >>qaq+5n3
◧◩
3. qaq+5n3[view] [source] [discussion] 2023-11-21 14:25:11
>>Rapzid+Iw2
Yep that's why I specifically mentioned "fairly modest instance" on reasonably fast box you can get magnitude more. You can partition the tasks table to reduce the number of rows skip locked has to run through to grab next task.
[go to top]