zlacker

>>abelan+(OP)
I love your vision and am excited to see the execution! I've been looking for exactly this product (postgres-backed task queue with workers in multiple languages and decent built-in observability) for like... 3 years. Every 6 months I'll check in and see if someone has built it yet, evaluate the alternatives, and come away disappointed.

One important feature request that probably would block our adoption: one reason why I prefer a postgres-backed queue over eg. Redis is just to simplify our infra by having fewer servers and technologies in the stack. Adding in RabbitMQ is definitely an extra dependency I'd really like to avoid.

(Currently we've settled on graphile-worker which is fine for what it does, but leaves a lot of boxes unchecked.)

>>kcorbi+Nd
Thank you, appreciate the kind words! What boxes are you looking to check?

Yes, I'm not a fan of the RabbitMQ dependency either - see here for the reasoning: >>39643940 .

It would take some work to replace this with listen/notify in Postgres, less work to replace this with an in-memory component, but we can't provide the same guarantees in that case.

>>abelan+9g
I come to this only as an interested observer, but my experience with listen/notify is that it outperforms rabbitmq/kafka in small to medium operations and has always pleasantly surprised me. You might find out it's a little easier than you think to slim your dependency stack down.

>>jagged+Ce1
How do you handle things when no listeners are available to be notified?

>>hosh+sb2
Presumably there'd be a messages table that you listen/notify on, and you'd replay messages that weren't consumed when a listener rejoins. But yeah, this is the overhead I was referencing.

>>abelan+1K2
Yep, but practically speaking, you need those records anyway even if you're using another queue to actually distribute the jobs. At least every system I've ever built of a reasonable size has a job audit table anyway. Plus it's an "Enterprise Feature™" so you can differentiate on it if you like that kind of feature-based pricing

>>jagged+TZ2
Postgres's LISTEN/NOTIFY doesn't keep those kinds of records. The whole point of using SKIP LOCK is so that you can make updates to rows to keep those kinds of messages with concurrent consumers.