zlacker

[return to "Postgres is a great pub/sub and job server (2019)"]
1. colinc+be[view] [source] 2021-12-18 00:19:11
>>anonu+(OP)
Author here! A few updates since this was published two years ago:

- The service mentioned (now called https://webapp.io ) eventually made it into YC (S20) and still uses postgres as its pub/sub implementation, doing hundreds of thousands of messages per day. The postgres instance now runs on 32 cores and 128gb of memory and has scaled well.

- We bolstered Postgres's PUBLISH with Redis pub/sub for high traffic code paths, but it's been nice having ACID guarantees as the default for less popular paths (e.g., webhook handling)

- This pattern only ever caused one operational incident, where a transaction held a lock which caused the notification queue to start growing, and eventually (silently) stop sending messages, starting postgres with statement_timeout=(a few days) was enough to solve this

Previous discussion: https://news.ycombinator.com/item?id=21484215

Happy to answer any questions!

◧◩
2. moneyw+0C[view] [source] 2021-12-18 03:57:45
>>colinc+be
Silly question but how does this compare to SQS? More cost friendly I assume?
◧◩◪
3. rmbyrr+2w1[view] [source] 2021-12-18 15:06:02
>>moneyw+0C
SQS vs "Postgres Queue", I think mainly:

- Closed/lock-in vs. Open/lock-free

- Rigid data access pattern vs. Very flexible SQL access

-Managed by AWS vs. Managed by you/your team (although you could use one of those managed Postgres services to reduce ops burden)

- Integrates well with other AWS services (e.g. Lambda, SNS, DynamoDB, etc) vs. No integrations with AWS ecossystem out of the box

[go to top]