zlacker

Postgres Postmaster does not scale

submitted by davidg+(OP) on 2026-02-04 16:30:51 | 124 points 86 comments
[view article] [source] [go to bottom]

NOTE: showing posts with links only show all posts
◧◩
2. evanel+U22[view] [source] [discussion] 2026-02-05 04:41:24
>>vel0ci+o12
Also check out ProxySQL [1][2], it's an extremely powerful and battle-tested proxy. Originally it was only for MySQL/MariaDB, where it is very widely used at scale, even despite MySQL already having excellent built-in scalable threaded connection management. But ProxySQL also added Postgres support too in 2024 and that has become a major focus.

[1] https://proxysql.com/

[2] https://github.com/sysown/proxysql

8. haki+k42[view] [source] 2026-02-05 05:00:09
>>davidg+(OP)
Some a prime example of a service that naturally peaks at round hours.

We have a habbit of never scheduling long running processes at round hours. Usually because they tend to be busier.

https://hakibenita.com/sql-tricks-application-dba#dont-sched...

◧◩◪
10. btown+c52[view] [source] [discussion] 2026-02-05 05:10:49
>>eatonp+Y32
You don't need multiple postmasters to spawn connection processes, if you have a set of Postgres proxies each maintaining a set pool of long-standing connections, and parceling them out to application servers upon request. When your proxies use up all their allocated connections, they throttle the application servers rather than overwhelming Postgres itself (either postmaster or query-serving systems).

That said, proxies aren't perfect. https://jpcamara.com/2023/04/12/pgbouncer-is-useful.html outlines some dangers of using them (particularly when you might need session-level variables). My understanding is that PgDog does more tracking that mitigates some of these issues, but some of these are fundamental to the model. They're not a drop-in component the way other "proxies" might be.

◧◩
15. lfittl+u62[view] [source] [discussion] 2026-02-05 05:24:11
>>vivzke+B32
Specifically on the cost of forking a process for each connection (vs using threads), there are active efforts to make Postgres multi-threaded.

Since Postgres is a mature project, this is a non-trivial effort. See the Postgres wiki for some context: https://wiki.postgresql.org/wiki/Multithreading

But, I'm hopeful that in 2-3 years from now, we'll see this bear fruition. The recent asynchronous read I/O improvements in Postgres 18 show that Postgres can evolve, one just needs to be patient, potentially help contribute, and find workarounds (connection pooling, in this case).

◧◩◪
36. jabl+zs2[view] [source] [discussion] 2026-02-05 08:49:45
>>lfittl+u62
Would be nice if the OrioleDB improvements were to be incorporated in postgresql proper some day.. https://www.slideshare.net/slideshow/solving-postgresql-wick...
◧◩◪
47. citrin+yO2[view] [source] [discussion] 2026-02-05 12:00:17
>>abrook+J52
I use fqdn_rand [1] in puppet for most cron jobs - it allows to run cron jobs at different time on different hosts (with different FQDN) but with the consistent interval between job runs. I would expect any modern configuration management system to have something like that.

[1] https://github.com/puppetlabs/puppet/blob/main/lib/puppet/pa...

◧◩◪
67. yencab+Ut3[view] [source] [discussion] 2026-02-05 16:18:05
>>abrook+J52
systemd timers have this, and more.

https://www.freedesktop.org/software/systemd/man/latest/syst...

[go to top]