zlacker

[return to "Postgres is a great pub/sub and job server (2019)"]
1. colinc+be[view] [source] 2021-12-18 00:19:11
>>anonu+(OP)
Author here! A few updates since this was published two years ago:

- The service mentioned (now called https://webapp.io ) eventually made it into YC (S20) and still uses postgres as its pub/sub implementation, doing hundreds of thousands of messages per day. The postgres instance now runs on 32 cores and 128gb of memory and has scaled well.

- We bolstered Postgres's PUBLISH with Redis pub/sub for high traffic code paths, but it's been nice having ACID guarantees as the default for less popular paths (e.g., webhook handling)

- This pattern only ever caused one operational incident, where a transaction held a lock which caused the notification queue to start growing, and eventually (silently) stop sending messages, starting postgres with statement_timeout=(a few days) was enough to solve this

Previous discussion: https://news.ycombinator.com/item?id=21484215

Happy to answer any questions!

◧◩
2. boomsk+Ye[view] [source] 2021-12-18 00:24:46
>>colinc+be
> doing hundreds of thousands of messages per day

> The postgres instance now runs on 32 cores and 128gb of memory and has scaled well.

Am I the only one?

◧◩◪
3. johnis+jg[view] [source] 2021-12-18 00:34:49
>>boomsk+Ye
Are you implying that given the specs, hundreds of thousands of messages per day is not good enough? I think you are, or at least that is what I was thinking myself.
◧◩◪◨
4. marcos+Qi[view] [source] 2021-12-18 00:56:16
>>johnis+jg
Only for hundreds of thousands of messages per day, that's way too big of a server. But if you look on the rest of the thread, it doesn't do only that.

Anyway, for a server that only does pub/sub with ACID guarantees, those specs are so large that there is certainly a bottleneck before they matter. So it wouldn't be strange if somebody gets one that can't even handle that, it just would mean that there is some issue somewhere we don't see.

◧◩◪◨⬒
5. edmund+XD[view] [source] 2021-12-18 04:19:50
>>marcos+Qi
Is your point that the server has room to grow? Or that you just “ain’t impressed by that”?
[go to top]