zlacker

>>bo0tzz+(OP)
What a strange design. If a job is dependent on an extant transaction then perhaps the job should run in the same code that initiated the transaction instead of a outside job queue?

Also you pass the data a job needs to run as part of the job payload. Then you don't have the "data doesn't exist" issue.

>>hipade+Xb
I agree. This design is incredibly strange, and seems to throw away basically all distributed systems knowledge. I'm glad folks are playing with different ideas, but this one seems off.

>>zackki+yd
No, this is a fairly common pattern called having an 'outbox' where the emission/enquing of your event/message/job is tied to the transaction completion of the relevant domain data.

We use this to ensure Kafka events are only emitted when a process succeeds, this is very similar.

>>eximiu+3g
So when the the business data transaction commit a notify event is raised and a job row is inserted. Out of bound job broker listens to a notify event of the job-table or polls the table skipping rows and takes work for processing?

>>iskela+yT
Basically.

For our particular use case, I think we're actually not using notify events. We just insert rows into the outbox table and the poller re-emits as kafka events and deletes successfully emitted events from the table.