zlacker

>>hyzyla+(OP)
I think one of the biggest advantages of using Redis for job queing vs Postgres comes down to library support.

For example Python has Celery and Ruby has Sidekiq. As far as I know there's no libraries in either language that has something as battle hardened with comparable features for background tasks using Postgres as a backend.

There's a big difference between getting something to work in a demo (achievable by skimming PG's docs and rolling your own job queue) vs using something that has tens of thousands of hours of dev time and tons of real world usage.

I'm all for using PG for things like full text search when I can because it drastically reduces operation complexity if you can avoid needing to run Elasticsearch, but Redis on the other hand is a swiss army knife of awesome. It's often used for caching or as a session back-end so you probably have it as part of your stack already. It's also really easy to run, uses almost no resources and is in the same tier as nginx in terms of how crazy efficient it is and how reliable it is. I don't see not using Redis for a job queue as that big of a win.

>>nickjj+Tg
But in Postgres you could write functions in your schema to handle job queueing/dequeueing with the additional benefit of being able to use it in any language that can connect to Postgres and being able to reuse the same SQL/interface across all languages.

>>RedShi+ph
I'm totally with you in this regard and I'd like to see that too but the reality of the situation is a job queue is more than slapping together a few SQL queries.

A good job queue will have all or most of these features:

    Prioritize jobs (queues, weighs to certain jobs, etc.)
    Scheduled jobs (running them once but X time in the future)
    Periodic jobs (running them every 2nd Tuesday at 3:33am)
    Static and dynamic configuration (CRUD'ing jobs at runtime, like adding new scheduled tasks)
    Re-try jobs (customizable strategy, such as exponential back off)
    Rate limit jobs
    Expire jobs
    Cancel jobs
    Unique jobs
    Batch executing
    Handling graceful shutdown (integration with your app server)
    Get metrics (status, health, progress, etc.)
    Browse job history
    Web UI (nice to have)

And I'm sure I'm missing things too. These are only off the top of my head based on features I tend to use in most applications. In other words, this isn't a laundry list of "nice to haves in theory", most of these are core or essential features IMO. I use them in nearly every web app.

Rolling all of these things on your own would be a massive undertaking. Tools like Celery and Sidekiq have been actively developed for ~10 years now and have likely processed hundreds of billions of jobs through them to iron out the kinks.

Even if you managed to do all of that and created it as a Postgres extension (which I think is doable on paper), that's only half the story. Now you'd have to write language specific clients to interface with that so you can create jobs in your application using a nice API. This would be a very welcome project but I think we're talking a year+ of full time development time to release something useful that supports a few languages, assuming you're already an expert with writing pg extensions, have extension knowledge about job queues and know a few popular programming languages to release the initial clients.