zlacker

> I am sick and tired of systems engineering being grandized, when all you have to do is sit down somewhere quiet and think about the problem — with a bit of tea, and some way to access reference material.

> Reddit is not a hard or interesting problem.

Exactly, tech is full of this weird hubris that everything has to be super complicated and over-engineered.

Heck, while you exclaim disdain for Python, I've seen large web services run on Django and a few servers behind load balancers with very few problems.

>>Maxion+(OP)
Care to show some proof? Create random data, store it somewhere, make sure it's about the size of Reddit. Off-the-cuff let's estimate that read traffic is 100x more than write traffic. Create some load generators that generate this synthetic read and write traffic. The load should follow a Zipf distribution of topics. Make sure it can handle huge traffic surges for events or abuse attacks. Show us your read and write performance. Do a small writeup on the architecture you ended up on, the number and types of servers you allocate, etc. You shouldn't be stopping at an order of magnitude short as scaling challenges change as the magnitude of scale changes.

Unlike Reddit, you'll have the benefit of the hindsight of 2023 instead of managing 20 years of tech debt.

>>Karrot+o4
Instagram has used python and django at scale. They have written about it in their engineering blog[0]. Not sure what their current stack is.

They did resort to all kinds of tricks. But your overal point still stands. The performance of python is lacking memory and it's embarrassingly slow. I hope python4 will have scripted for developing and compiled for production, like Dart. And a great compiler like Rust.

[0]https://instagram-engineering.com/static-analysis-at-scale-a...

>>Attumm+48
My question isn't about Python. It's about Reddit being trivial to recreate. I work on an API team at a Big Tech company and, funny enough, a lot of our legacy is in Python and we've scaled it using lots of pretty gross tricks. We may or may not be Instagram (:

The keyword here of course is "at scale". At what scale? Any commenter that believes what was written upthread should create a system and demonstrate that it can scale to Reddit levels.

>>Karrot+o4
> Care to show some proof?

I said large, not huge :P

I'm afraid I don't want to dox myself so I can't post publicly stuff from my employer. And I don't really have time to do what ask and write it up in my free time.

I doubt something the size of reddit would run properly on Python, but I think both mine and the commenter I replied to had the point that most sites on the internet WOULD run fine without all the bloatware and overengineering complexity. Very very few sites have the traffic that reddit does. Most websites belong to the long tail, and for those almost any tech stack would work - so why choose a needlessly complex one?

>>Karrot+o9
They also think what Reddit does is "only serve large amounts of text" oh where should I start with how wrong this is

I'm sure not even HN "does only that" and even that it does with a lot of help from caching, etc

>>Attumm+48
python4? Is that coming?

No... not again... I can't.