zlacker

>>abelan+(OP)
Can somebody explain why would I use it instead a simple Redis/SQS/Postgres queue implemented in 50 LOC (+ some grafana panel for monitoring) (which is pretty much mandatory even for a wrapper of this or any other service)? I'm not trying to mock it, it's a serious question. What is implied by "task queue" that makes it worth bothering to use a dedicated service?

>>krick+RD1
You're right, if all you need is a queue with a small number of workers connected at low volume, you don't need Hatchet or any other managed queue - you can get some pretty performant behavior with something like: https://github.com/abelanger5/postgres-fair-queue/blob/main/....

The point of Hatchet is to support more complex behavior - like chaining tasks together, building automation around querying and retrying failed tasks, handling a lot of the fairness and concurrency use-cases you'd otherwise need to build yourself, etc - or just getting something that works out of the box and can support those use-cases in the future.

And if you are running at low volume and trying to debug user issues, a grafana panel isn't going to get you the level of granularity or admin control you need to track down the errors in your methods (rather than just at the queue level). You'd need to integrate your task queue with Sentry and a logging system - and in our case, error tracing and logging are available in the Hatchet UI.

>>abelan+Pj2
That caught my attention. Retrying failed tasks isn’t easy. There are all kinds of corner cases that pop up one by one. If you have some nice way to handle the common failure modes ("text me" or "retry every ten minutes" or "retry 5 times, then give up" or "keep retrying, but with exponential backoff") then that’s something I’d love to use.

(Wiring together 40+ preemptible TPUs was a nice crucible for learning about all of these. And much like a crucible, it was as painful as it sounds. Hatchet would’ve been nice.)

Thanks for making this!