zlacker

[return to "Idempotency keys for exactly-once processing"]
1. AdieuT+iPg[view] [source] 2025-12-06 06:04:27
>>defly+(OP)
From the article:

  In distributed systems, there’s a common understanding that 
  it is not possible to guarantee exactly-once delivery of 
  messages.
This is not only a common understanding, it is a provably correct axiom. For a detailed discussion regarding the concepts involved, see the "two general's problem"[0].

To guarantee exactly once processing requires a Single Point of Truth (SPoT) enforcing uniqueness shared by all consumers, such as a transactional persistent store. Any independently derived or generated "idempotency keys" cannot provide the same guarantee.

The author goes on to discuss using the PostgreSQL transaction log to create "idempotency keys", which is a specialization of the aforementioned SPoT approach. A more performant variation of this approach is the "hi/low" algorithm[1], which can reduce SPoT allocation of a unique "hi value" to 1 in 2,147,483,648 times when both are 32-bit signed integers having only positive values.

Still and all, none of the above establishes logical message uniqueness. This is a trait of the problem domain, in that whether two or more messages having the same content are considered distinct (thus mandating different "idempotentcy keys") or duplicates (thus mandating identical "idempotency keys").

0 - https://en.wikipedia.org/wiki/Two_Generals'_Problem

1 - https://en.wikipedia.org/wiki/Hi/Lo_algorithm

◧◩
2. threat+xgh[view] [source] 2025-12-06 12:27:13
>>AdieuT+iPg
It should be noted that when you have bad actors in your system almost all guarantees of all kinds go out the window.
[go to top]