zlacker

[parent] [thread] 4 comments
1. radica+(OP)[view] [source] 2023-11-20 20:25:22
I don't know why people even use libraries in those languages - assuming you stick to a database engine you understand well then the (main)-database-as-queue pattern is trivial to implement. Any time spent writing code is quickly won back by not having to debug weird edge cases, and sometimes you can highly optimize what you're doing (for example it becomes easy to migrate jobs which are data-dominated to the DB server which can cut processing time by 2-3 orders of magnitude).

It's particularly suited to use cases such background jobs, workflows or other operations which occur within your application and scales well enough for what 99.9999% of us will be doing.

replies(1): >>AtlasB+aJ4
2. AtlasB+aJ4[view] [source] 2023-11-21 23:45:33
>>radica+(OP)
I thought the first rule of queues was never use a database as a queue.
replies(1): >>22c+J05
◧◩
3. 22c+J05[view] [source] [discussion] 2023-11-22 01:30:17
>>AtlasB+aJ4
The "problem" is that a database as a queue doesn't infinitely scale, but nothing infinitely scales. If you're starting to reach the limits of Postgres-as-a-queue then you've either done something very wrong or very right to get to that point.

It's probably more correct to say that the engineering effort required to make a Postgres-as-a-queue scale horizontally is a lot more than the engineering effort required to make a dedicated queueing service scale horizontally. The trade-off is that you're typically going to have to start scaling horizontally much sooner with your dedicated queuing service than with a Postgres database.

The argument for Postgres-as-a-queue is that you might be able to get to market much quicker, which can be significantly more important than how well you can scale down the track.

replies(1): >>AtlasB+8B6
◧◩◪
4. AtlasB+8B6[view] [source] [discussion] 2023-11-22 14:19:19
>>22c+J05
I thought Kafka could relatively infinitely scale
replies(1): >>22c+097
◧◩◪◨
5. 22c+097[view] [source] [discussion] 2023-11-22 16:45:53
>>AtlasB+8B6
I wouldn't categorise Kafka as a database.
[go to top]