zlacker

This is what everybody thinks, until they grow large enough to care about the amount of money they’re burning and get tired of it. Then they try to not lose so much money, and their loyal users turn out not to be so loyal.

>>Nifty3+(OP)
All reddit is doing is storing text and serving it to people.

This not expensive or a hard problem. You grab a bunch of servers, you set them up properly, and then you write your app properly.

No resume-driven bullshit; no hype-driven bullshit; no “we need to be galaxy scale now” bullshit. No email notifications, besides basic “thanks for registering, here is your login” and “here’s a password reset link.” No cloud-based bullshit. Don’t use fucking python. Use a real systems language to eek out as much performance as you can from the hardware. Actually understand databases and how your specific databases work. Use Postgres unless you have a very good reason not to.

Just a few thousand dollars a month, and a brief reprieve from short-term mania to actually think, and you too can literally serve 1 billion pages a day.

Why does everyone run into problems with this? Because they have personal hang-ups and delude themselves (or simply don’t care). This path has been tread numerous times before. The mistakes have been made thousands of times. The people who made those mistakes are available to help you out (for the right price, or if you’re good enough company).

I am sick and tired of systems engineering being grandized, when all you have to do is sit down somewhere quiet and think about the problem — with a bit of tea, and some way to access reference material.

Reddit is not a hard or interesting problem.

>>rgbgra+W1
> I am sick and tired of systems engineering being grandized, when all you have to do is sit down somewhere quiet and think about the problem — with a bit of tea, and some way to access reference material.

> Reddit is not a hard or interesting problem.

Exactly, tech is full of this weird hubris that everything has to be super complicated and over-engineered.

Heck, while you exclaim disdain for Python, I've seen large web services run on Django and a few servers behind load balancers with very few problems.

>>Maxion+Qc
Care to show some proof? Create random data, store it somewhere, make sure it's about the size of Reddit. Off-the-cuff let's estimate that read traffic is 100x more than write traffic. Create some load generators that generate this synthetic read and write traffic. The load should follow a Zipf distribution of topics. Make sure it can handle huge traffic surges for events or abuse attacks. Show us your read and write performance. Do a small writeup on the architecture you ended up on, the number and types of servers you allocate, etc. You shouldn't be stopping at an order of magnitude short as scaling challenges change as the magnitude of scale changes.

Unlike Reddit, you'll have the benefit of the hindsight of 2023 instead of managing 20 years of tech debt.

>>Karrot+eh
Instagram has used python and django at scale. They have written about it in their engineering blog[0]. Not sure what their current stack is.

They did resort to all kinds of tricks. But your overal point still stands. The performance of python is lacking memory and it's embarrassingly slow. I hope python4 will have scripted for developing and compiled for production, like Dart. And a great compiler like Rust.

[0]https://instagram-engineering.com/static-analysis-at-scale-a...

>>Attumm+Uk
My question isn't about Python. It's about Reddit being trivial to recreate. I work on an API team at a Big Tech company and, funny enough, a lot of our legacy is in Python and we've scaled it using lots of pretty gross tricks. We may or may not be Instagram (:

The keyword here of course is "at scale". At what scale? Any commenter that believes what was written upthread should create a system and demonstrate that it can scale to Reddit levels.

>>Karrot+eh
> Care to show some proof?

I said large, not huge :P

I'm afraid I don't want to dox myself so I can't post publicly stuff from my employer. And I don't really have time to do what ask and write it up in my free time.

I doubt something the size of reddit would run properly on Python, but I think both mine and the commenter I replied to had the point that most sites on the internet WOULD run fine without all the bloatware and overengineering complexity. Very very few sites have the traffic that reddit does. Most websites belong to the long tail, and for those almost any tech stack would work - so why choose a needlessly complex one?

>>Karrot+em
They also think what Reddit does is "only serve large amounts of text" oh where should I start with how wrong this is

I'm sure not even HN "does only that" and even that it does with a lot of help from caching, etc

>>rgbgra+W1
"I could build a Reddit clone in a weekend" says person who hasn't taken over from Reddit. "Cost a few thousand a month, it's not hard" they continued, turning down an easy shot at Reddit's $10Bn valuation with admirable restraint. "It's just self delusion" they said, describing "numerous" failed attempts and "thousands and thousands" of previous mistakes they have convinced themselves they wouldn't fall into should they try, which they carefully avoid having to by dismissing it as "not interesting".

"Take my word for it, I'm just superior" the comment, which would be as fitting on r/SneerClub today SlashDot 15 years ago or Usenet 30 years ago as a dismissive geek putdown-cum-status grab, could have been summarised as.

>>Attumm+Uk
python4? Is that coming?

No... not again... I can't.