zlacker

[parent] [thread] 25 comments
1. Twisol+(OP)[view] [source] 2021-06-12 09:02:38
For other readers, the `UPDATE` step is an exact anlogue of the "compare-and-set" atomic instruction [0]. It's really cool to see how you've realized it in SQL!

As a capability, compare-and-swap has an infinite consensus number [1], meaning it's sufficient to implement wait-free consensus algorithms with an arbitrary number of participants. That makes it a perfect fit for managing a scalable pool of workers that need to coordinate on consuming from a queue!

[0] https://en.wikipedia.org/wiki/Compare-and-swap

[1] https://en.wikipedia.org/wiki/Consensus_(computer_science)#C...

replies(2): >>chx+k2 >>halifa+La
2. chx+k2[view] [source] 2021-06-12 09:31:43
>>Twisol+(OP)
Yes, the UPDATE command is the exact equivalent of LOCK CMPXCHG (the SELECT can be seen as computing the memory address). So the discussion about that in the comments of https://stackoverflow.com/a/59022356/308851 totally applies: if two queue runner threads pick the same item exactly one will succeed so it can't happen both tries and tries the same item. So there's no busy wait (reminder: a busy wait is where it does nothing just tests a condition), it just goes over every candidate until one succeeds.
replies(1): >>abhish+5v
3. halifa+La[view] [source] 2021-06-12 11:05:50
>>Twisol+(OP)
Postgres’s transactional semantics are really useful when building a queue, because of how it interacts with the various pieces.

Connection 1

  LISTEN 'job-updates';
Connection 2

  BEGIN;
  INSERT INTO jobs ('a-uuid', …);
  SELECT PG_NOTIFY('job-update', 'json blob containing uuid and state change info');
  COMMIT;
Connection 3 (used when Connection 1 is notified)

  BEGIN;
  SELECT id, … FROM jobs WHERE id = 'a-uuid' FOR UPDATE SKIP LOCKED;
  UPDATE 'jobs' SET state = 'step1_completed' WHERE is = 'a-uuid';
  SELECT PG_NOTIFY('job-update', 'json blob containing uuid and state change info');
  -- do the thing here: computation, calling external API, etc. If it fails then rollback.
  COMMIT;
Because notify has transactional semantics, the notify only goes out at transaction commit time. You want to use a dedicated connection for the notify.

The only downsides I immediately think of are you will have every worker contending to lock that row, and you’ll need to write periodic jobs to cleanup/retry failures.

◧◩
4. abhish+5v[view] [source] [discussion] 2021-06-12 14:47:34
>>chx+k2
I have been asked this question in interviews for how to prevent two people from booking the same seat. Interviewers don’t seem to be satisfied by the answer that I will let the query go to the database and see who ends up booking. They want some advanced locking mechanism using redis. Redis does serialize queries using lua thus avoiding two people from booking the same seat.
replies(4): >>bryanr+9z >>agf+az >>kccqzy+G31 >>phamil+6K2
◧◩◪
5. bryanr+9z[view] [source] [discussion] 2021-06-12 15:25:17
>>abhish+5v
based on experience arriving very last minute to flight, nobody has problem with letting two people book the same seat.
replies(3): >>kyleee+GE >>chx+q31 >>abhish+Nv2
◧◩◪
6. agf+az[view] [source] [discussion] 2021-06-12 15:25:22
>>abhish+5v
I think this being a good answer hinges on the resulting user experience. If one person spends several minutes going through the process of putting in their payment and other info only to be told you don't have a ticket, they're going to be pissed. So you need a good answer on how to avoid that experience, however you implement the locking.

Saying this as someone who has been on both sides of that interview question, and also evaluated performance in it as a hiring manager.

replies(3): >>lstamo+CG >>spockz+GU >>abhish+tv2
◧◩◪◨
7. kyleee+GE[view] [source] [discussion] 2021-06-12 16:18:48
>>bryanr+9z
Let the check-in attendant handle it; that's what I call a no-code solution! You're hired
replies(3): >>Tostin+xT >>lootsa+kv1 >>abhish+2w2
◧◩◪◨
8. lstamo+CG[view] [source] [discussion] 2021-06-12 16:35:57
>>agf+az
Send bookings to the database early, with an expiry time and a counter visible on the screen?

Thing is, because purchasing is often shared across third-parties, even then you can’t guarantee that you’ll have the reservation until it’s paid and entered into the shared booking backend… unless the third party backend has a locking mechanism, at which point you’re probably not using your own Postgres to do this, but their mainframe instead.

◧◩◪◨⬒
9. Tostin+xT[view] [source] [discussion] 2021-06-12 18:14:02
>>kyleee+GE
That's solid middle management thinking right there!
◧◩◪◨
10. spockz+GU[view] [source] [discussion] 2021-06-12 18:23:32
>>agf+az
Is there any other solution other than going full STM on this?
replies(1): >>mathfa+DF1
◧◩◪◨
11. chx+q31[view] [source] [discussion] 2021-06-12 19:44:18
>>bryanr+9z
mmmm no

They allow overbooking the flight but you can't book the same seat twice.

Overbooking can make financial sense -- even if you need to hand a little compensation from time to time it's still better than flying with empty seats. Of course it sucks big time for the travelers especially if there are no volunteers. IDB is miserable. https://www.transportation.gov/individuals/aviation-consumer...

replies(2): >>mianos+4k1 >>bryanr+Dq1
◧◩◪
12. kccqzy+G31[view] [source] [discussion] 2021-06-12 19:46:38
>>abhish+5v
I had a similar question asked of me in an interview. The interviewer was also surprised by my solution of letting the database decide the winner.
replies(1): >>abhish+bv2
◧◩◪◨⬒
13. mianos+4k1[view] [source] [discussion] 2021-06-12 22:42:20
>>chx+q31
They obviously don't have the transaction semantics worked out. I have been at the gate and the person in front of me had the same seat number. They read it out. When I presented my pass the girl politely took my ticket and printed another one. If the person was not right in front of me I would never have known why. It is not unusual.
◧◩◪◨⬒
14. bryanr+Dq1[view] [source] [discussion] 2021-06-12 23:44:43
>>chx+q31
>They allow overbooking the flight but you can't book the same seat twice.

ok, but how do you overbook a flight without booking at least one of the seats twice?

replies(1): >>Jailbi+oy1
◧◩◪◨⬒
15. lootsa+kv1[view] [source] [discussion] 2021-06-13 00:37:42
>>kyleee+GE
Reminds me of Starbucks does not use two phase commit. https://news.ycombinator.com/item?id=6229004
◧◩◪◨⬒⬓
16. Jailbi+oy1[view] [source] [discussion] 2021-06-13 01:09:44
>>bryanr+Dq1
You don't assign all the seats up front. Some are marked to be assigned at the gate.
replies(1): >>abhish+pw2
◧◩◪◨⬒
17. mathfa+DF1[view] [source] [discussion] 2021-06-13 02:48:06
>>spockz+GU
Yes a simple SCRM will do.
replies(1): >>abhish+zv2
◧◩◪◨
18. abhish+bv2[view] [source] [discussion] 2021-06-13 13:45:49
>>kccqzy+G31
What exactly is the right answer here?
◧◩◪◨
19. abhish+tv2[view] [source] [discussion] 2021-06-13 13:48:11
>>agf+az
So what do you propose here? I see that stackoverflow mentions some preemptive locking for a seat in cache for a few minutes. During that time the seat is taken out of the pool for potential booking. If the time lapses without the seat getting booked then the seat returns to the pool. This avoids going to the DB and see who wins.
◧◩◪◨⬒⬓
20. abhish+zv2[view] [source] [discussion] 2021-06-13 13:48:58
>>mathfa+DF1
What is SCRM?
◧◩◪◨
21. abhish+Nv2[view] [source] [discussion] 2021-06-13 13:50:12
>>bryanr+9z
Well, sure. But what if you want to avoid such a scenario? Two people booking vaccine slots shouldn’t be allowed the exact same slot. They are going to be pissed realising one of them reached the center but didn’t get the vaccine.
◧◩◪◨⬒
22. abhish+2w2[view] [source] [discussion] 2021-06-13 13:52:08
>>kyleee+GE
Wouldn’t the attendant be in trouble if two people at the gate start fighting for the same window seat as printed on their ticket?
◧◩◪◨⬒⬓⬔
23. abhish+pw2[view] [source] [discussion] 2021-06-13 13:54:12
>>Jailbi+oy1
Well then the problem moves a little to the left. If there are x seats to be booked up front and y seats to be allocated later. How many times can each of the x seats be booked and how do you prevent that n and n+1 over the limit booking for some seat? The problem is still there.
replies(1): >>chx+qC6
◧◩◪
24. phamil+6K2[view] [source] [discussion] 2021-06-13 15:39:26
>>abhish+5v
The "what actually happens" answer is often "let them both book successfully and then rebook the person with lower loyalty status".
◧◩◪◨⬒⬓⬔⧯
25. chx+qC6[view] [source] [discussion] 2021-06-14 21:13:43
>>abhish+pw2
Airlines have been data mining for a very, very long time now and have a reasonably guess of the ratio of people booking vs showing up at the gate to fly. So you set your buckets to oversell by that much and assign seats later. If more people show up than expected than you throw them a voucher to go away. Hopefully enough will.

Sometimes it doesn't work out and that's when airlines bleed. I was flying home one day from some US city (was it Denver? the years and cities blend together) when a Drupal conference and some major sports thing ended the same day and the airport was a total gong show. United first offered a paltry 120 USD but when they raised to 400 USD to go home the next day instead my hand went up -- and I was such a greenhorn I didn't know they will pay for the hotel, too. A buddy pocketed no less than 1200 USD from Delta.

replies(1): >>abhish+V88
◧◩◪◨⬒⬓⬔⧯▣
26. abhish+V88[view] [source] [discussion] 2021-06-15 12:23:44
>>chx+qC6
This looks like one solution but I don’t think you want this design in medicine slot booking or stock selling. You want it to be much more definite if there is no way for a conflict to be resolved. This solution won’t work there.
[go to top]