zlacker

[parent] [thread] 0 comments
1. abelan+(OP)[view] [source] 2024-06-27 19:32:35
While the execution model is very similar to Airflow, we're primarily targeting async jobs which are spawned from an application, while Airflow is primarily for data pipelines. The connector ecosystem of Airflow is very powerful and not something that we're trying to replace.

That's not to say you can't use Hatchet for data pipelines - this is a common use-case. But you probably don't want to use Hatchet for big data pipelines where payload sizes are very large and you're working with payloads that aren't JSON serializable.

Airflow also tends to be quite slow when the task itself is short-lived. We don't have benchmarks, but you can have a look at Windmill's benchmarks on this: https://www.windmill.dev/docs/misc/benchmarks/competitors#re....

[go to top]