zlacker

> 429 Too Many Requests

Aight, level with me: Is every mastodon server running on a Raspberry Pi?

>>NavinF+(OP)
Looks like we hugged it to death

>>NavinF+(OP)
The server itself seems to work fine. It only seems to be this specific post that's being 429'd. I'm guessing it's some kind of anti-DDoS setup kicking in.

Mastodon is also quite heavy to host, my single user instance will easily gobble up several gigabytes of memory if you let it. There are more efficient ActivityPub servers but specifically Mastodon seems to be written for running efficiently on huge servers.

>>NavinF+(OP)
From the instance admins:

> We just started serving a http 429 error on the exact url of the post. So everything should go back to normal now.

>>NavinF+(OP)
Maybe, but the admin commented it was intentional for that specific post, it was slowing down the entire site.

>>i-am-g+N
Sites like HN and that other one should track this. It could work like flagging - if enough people mark it as hugged to death (HTD), it would say so next to the link. Maybe it could even redirect to an archive if it’s currently HTD.

>>i-am-g+N
lightly blew in its general direction to death

>>tethys+n1
If you go to the user's profile and then to the post, it seems to be okay. So perhaps also looking at Referrer.

>>NavinF+(OP)
Aren’t federated services grand?

>>NavinF+(OP)
chaos.social is run by the chaos computer club, you can assume that they configured it that way on purpose.

my profile, on the same server, loads fine.

>>NavinF+(OP)
for the time being, archive.org has a snapshot of it: https://web.archive.org/web/20230504185520/https://chaos.soc...

>>NavinF+(OP)
Not really: hhttps://leah.is/posts/scaling-the-mastodon/ They just got 6 times the normal requests: https://chaos.social/@ordnung/110312089838674624

>>isclev+v1
> slowing down the entire site

This is mind-blowing. Last I checked, the front page of HN sends tens of requests per second to each link. There are humans who can pack envelopes faster than the typical mastodon server can answer GETs. I'd love to see someone benchmark the top servers for a few seconds to see what it takes to break a reasonable latency SLA.

>>NavinF+(OP)
No trouble viewing it from another Mastodon server:

https://hachyderm.io/@jonty@chaos.social/110307532115312279

EDIT: Ah I guess if you're not logged into a hachyderm.io account, you get forwarded. So probably don't use the above link.

>>neogod+a5
That just redirected me to too many requests.

>>NavinF+15
It has been 20+ years since slashdotting with the requisite hardware and connection upgrades and still things fall over.

>>NavinF+15
Clearly they're not microservicing hard enough.

>>NavinF+(OP)
one alt-social crumbling from throwing rocks at another alt-social

This is peak Web 5.0 right here.

>>NavinF+(OP)
This post is on the front of HN. Many a larger website have succumbed to HN's warm embrace.

>>NavinF+(OP)
Because it is webscale.

>>tethys+n1
So to answer GP's question, yes.

>>weglas+f4
unfortunately this is exactly why mastodon won't work

I already don't trust mastodon links because 9 times out of 10 they simply don't work. Everyone's tiny hobby server falls over when one post gets big, and obviously not everyone is going to scale their servers to support the load of a viral post that might happen once every 6 months and will be 100x their base load

>>NavinF+(OP)
You can also see the same "toot" as a "tweet": https://twitter.com/jonty/status/1653915932677271552

>>crooke+n6
I suggest making the current team explain to 4 new teams how to port it to something fast, like elixir and rust (both!)

Probably messaging with pulsar and the build system from python 4, too.

I read this in a whitepaper. Let’s do this, guys! ;)

>>NavinF+(OP)
Just look at it failing the HN hug of death. If this one can't survive techies on a orange site rushing to the site then it cannot possibly survive a lotus of users from Twitter or TikTok rushing into any post on Mastodon, bringing it flat on to the floor.

I can only see Mastodon centralizing to cope with the load. But a server going down on this load from HN tells us it is no where near ready to handle an insurmountable amount of users or even begin to challenge Twitter which hosts 220M+ users every single day.

>>rvz+Zh
Mastodon user count has mostly been a steady growth. So far it hasn't really failed at a high level. We aren't in Bitcoin territory, the network isn't really slower than it was a year ago even if number of user is much higher. It's mostly distributed among many instances.

>>rvz+Zh
Twitter is pretty sluggish, to be fair. 7 seconds to load and render a single tweet on mobile.

https://pagespeed.web.dev/analysis/https-twitter-com-realDon...

Mastodon.social is actually much faster on this particular benchmark. So maybe there is hope.

>>shawab+Nd
Mastodon is by design about small niche communities rather than centralised twitter alternative.

>>chrish+c2
Doing such a thing never requests to the page that is linked, so it makes sense that nothing is blocking it

>>shawab+Nd
That is the benefit of centralization, the experience for the end user can be controlled completely. Maybe a Mastodon friendly web cache that anyone running a semi-serious instance could easily opt into (for a fee) is needed. As a hedge to keep your Raspberry Pie instance online if something goes viral.

As a community effort where no one is expecting to get rich it might work.

>>neorom+9n
That is not the point.

If someone sent any link or post that is from that Mastodon instance and it went viral, the entire instance will be sent to the ground and out for hours, making the post unavailable to be viewed.

The worst part is journalists and the media have to be told that posting a link from a 'small niche community' on Mastodon will send a flood of traffic that will knock it down offline also giving the impression to others that it is not ready for mainstream at all or even ready to onboard on tens or hundreds of millions of users, daily like Twitter.

>>jszymb+5b
Isn't HN pretty small? This post has <400 upvotes over 3 hours. There can't be 1000x that amount of lurkers can there?

>>yreg+Ky
HN has millions of page views per day (maybe @dang can give a more accurate and updated number), and things frequently gets reposted elsewhere. Happens many times that things on the frontpage gets brought down to its knees, this wouldn't be the first nor the last.

>>popey+eh
And the original tooter is apparently Google Translate?

>>NavinF+(OP)
Ironically, I've seen pages make it to the front of HN that were run on RPis or weaker.

There was a guy running a site off of a single core 32bit ARM SoC that was able to handle the HN frontpage.

>>capabl+jD
Source on the millions?

This person says they got 12k visitors over a day:

https://nicklafferty.com/blog/what-happens-when-you-re-on-th...

The websites hugged to death by this forum are usually tiny hobby projects.

>>rvz+St
It's a fair criticism but I don't feel so fatalistic. This would look a lot different if everyone was opening this post on their own home server instead of chaos.social.

Unfortunately there's no way to construct a link that references the post but opens where it belongs for you. I think there needs to be a fediverse URL protocol to solve for this, ie this HN post would link to `fedi://@jonty@chaos.social/110307532009155432`, then when people clicked it they wouldn't have to talk to chaos.social, because it would be opened at their home server.

Another option could be a 'copy link for public access' that generates a static page for the purpose of sharing widely.

Journalists and media could also run their own server which is scaled for the traffic they expect, and mirror the post there. The main problem is linking to the source of the post instead of linking to a federated representation of it.

>>jeroen+b1
Or running efficiently never maybe?

>>yreg+Ky
I’ve seen other people that posted about the HN embrace talk about 50k extra visitors. I guess this is a single page, so 50k pageviews?

>>rvz+St
Seems like a correct impression, then.

>>NavinF+(OP)
chaos.social is run on four dedicated servers

https://leah.is/posts/scaling-the-mastodon/

>>NavinF+(OP)
This was both funny and ironic at the same time.

Upon clicking reply fastidiously, i got the hn 429

"Sorry, we're not able to serve your requests this quickly."

Wow.

>>yreg+Ky
I can provide some statistics myself.

One of my blog posts was submitted to HN that had 194 points and 149 comments[1]. All dates are in UTC.

  1 - Unique visitors per day - Including spiders
  Hits       h%  Vis.     v%  Tx. Amount Data
  ------ ------ ----- ------ ----------- ----
   14439  1.49%  1148  1.19%  106.42 MiB 21/Jan/2023
   17043  1.75%  1754  1.81%  184.69 MiB 20/Jan/2023
   33560  3.45%  3267  3.37%  491.32 MiB 19/Jan/2023
   46568  4.79%  5816  6.01%  637.54 MiB 18/Jan/2023
  323797 33.32% 28928 29.88%    4.06 GiB 17/Jan/2023  <- Resubmitted on HN and websites started copy-pasting the article from the big website with the same mistakes, never checking my post which had a note about these mistakes :)
   24330  2.50%  3341  3.45%  360.48 MiB 16/Jan/2023  <- Put in a second-chance pool by a moderator and an article with a lot of mistakes published by some big website
   17074  1.76%  3348  3.46%  243.44 MiB 15/Jan/2023  <- Published on HN
    1041  0.11%   120  0.12%    3.70 MiB 14/Jan/2023
    1666  0.17%   171  0.18%    8.40 MiB 13/Jan/2023  <- Post published
     991  0.10%   123  0.13%  374.78 KiB 12/Jan/2023

  2 - Requested Files (URLs)
  Hits      h%  Vis.     v%  Tx. Amount Mtd      Proto    Data
  ----- ------ ----- ------ ----------- -------- -------- ----
  57604  5.93% 31427 32.46%  260.97 MiB GET      HTTP/2   /en/2023/01/13/msi-insecure-boot/
  31179  3.21% 11263 11.63%  245.20 MiB GET      HTTP/1.1 /en/2023/01/13/msi-insecure-boot/

  11 - Referring Sites (depends on Referer header, not very accurate for reasons)
  Hits       h%  Vis.     v% Tx. Amount Data
  ------ ------ ----- ------ ---------- ----
  446781 45.97% 29686 30.66%   5.95 GiB dawidpotocki.com
   14834  1.53%  9485  9.80%  79.85 MiB news.ycombinator.com
  (news sites with very low hundreds or even under, nobody checks sources)

[1]: https://news.ycombinator.com/item?id=34388533

>>yreg+Ky
Indeed there are. Tens of us!

Maybe you underestimate how many people want to keep up on things but not interact?

>>Aeolun+3J
It will definitely "never" (barring fairly significant changes) run efficiently, you're right. It's extremely unnecessarily heavy in all kinds of ways that are made in ways that makes it run overall better on a large setup, though. All the instructions are also there to front it by proper caching, but setting it up in a proper resilient way is more effort.

>>shawab+Nd
When someone finds an annoyance, often even anecdotal, that is no evidence of why "Mastodon (or the fediverse) won't work".

It's an annoyance, often anecdotal at most. Not the foundation of why a platform cannot ever "work".

>>yreg+xF
Not the OP but they were referring to the whole site. So definitely not millions, but the number is probably higher than you think.

From the blog you linked, the number of interest is 18k. 12k are only those with HN referrer headers. In reality, many setup strips that header so you can't track it exactly right. The author did mention they averaged 50 views before.

A big part of it are reposts. From my own submissions, posting to HN resulted in tons of different origins. Public ones like reddit, twitter and private ones like newsletters, dashboard & chat messages. You'll also be surprised by the wide variety of clients people use to access HN.

They also used Google analytics to track the numbers. Most people in HN block it either through the browser or an extension [0]. In reality it's probably double the traffic.

Don't forget to account for scraping & crawling bots. That's another big source of traffic that the author didn't track.

[0] https://plausible.io/blog/google-analytics-adblockers-missin...

>>dicknu+hV
More than tens I'd say. I suspect for each person that interacts, there are dozens that don't. If I have to bet, I'd put the ratio at 1:100. So 100 lukers for every 1 active user.

>>yreg+xF
Dang from 6 months ago https://news.ycombinator.com/item?id=33454140

> There's no stats page but last I checked it was around 5M monthly unique users (depending on how you count them), perhaps 10M page views a day (including a guess at API traffic), and something like 1300 submissions (stories) and 13k comments a day.

>>yreg+xF
Hacker News has 3.4 million users per month and 350,000 users per day, with 4 million pageviews a day. There are just under 1 million registered accounts, with several hundred added each day. Users post around 1,000 articles and 6,000 comments to the site per day. https://blog.samaltman.com/2017-yc-annual-letter

>>mal-2+MG
I think this is an absolutely absurd take. A post is the same post no matter who views it - it belongs on the instance where it was posted. Sure it might show up in your timeline or comments somewhere else but for the post itself there should only be one canonical link. If mastodon can't manage to show a simple text post with a small image to anonymous visitors without falling over then it's mastodon that needs to change and not how people interact with it. Most people don't even have a fediverse account ffs.

>>accoun+yZ1
Where is the canonical location to access an email, or read an XMPP message? It's not just that it 'might show up' in my timeline, seeing it on my home server is where I want it to be - that's where I can take actions on it like replying, starring, or boosting. The post belongs in my client because that representation is the one that's relevant to me. I agree that the mastodon software could do better to optimize for public anonymous read, but it's not the most important functionality for the server to do.

> Most people don't even have a fediverse account ffs.

This is why you won't see a Bluesky post linked on HN, no one can open it. Imagine if you could sign up on your choice of thousands of servers and get the same access to the content rather than a central site, that's fediverse, it's not that complex.

>>capabl+Ur1
Forgot to mention, but "I was on the HN frontpage for X hours and got X views" doesn't always translate to the same happening for everything. Some topics are more interesting to the people just browsing HN, than others. I'd expect an article titled "I spent $6 Million On Google Ads Last Year" to be significantly less interesting than "MSFT is forcing Outlook and Teams to open links in Edge and IT admins are angry" for example, where the latter would surely gather a magnitude of more visits than the earlier, even if they would spend the same amount of hours on the frontpage.

Some content is simply more interesting for a broader audience.