zlacker

Cloudflare outage on December 5, 2025

submitted by meetpa+(OP) on 2025-12-05 15:35:43 | 761 points 551 comments
[view article] [source] [go to bottom]

NOTE: showing posts with links only show all posts
22. da_gri+Y4[view] [source] 2025-12-05 15:54:43
>>meetpa+(OP)
It's not an outage, it's an Availability Incident™.

https://blog.cloudflare.com/5-december-2025-outage/#what-abo...

72. rachr+Wc[view] [source] 2025-12-05 16:24:27
>>meetpa+(OP)
Time for Cloudflare to start using the BOFH excuse generator. https://bofh.d00t.org/
◧◩◪◨⬒
103. esseph+kl[view] [source] [discussion] 2025-12-05 17:00:11
>>beanju+od
Half your medical devices are probably opening up data leakage to China.

https://www.csoonline.com/article/3814810/backdoor-in-chines...

Most hospital and healthcare IT teams are extremely under funded, undertrained, overworked, and the software, configurations and platforms are normally not the most resilient things.

I have a friend at one in the North East right now going through a hell of a security breach for multiple months now and I'm flabbergasted no one is dead yet.

When it comes to tech, I get the impression most organizations are not very "healthy" in the durability of systems.

123. cpncru+Qo[view] [source] 2025-12-05 17:14:59
>>meetpa+(OP)
I've noticed that in recent months, even apart from these outages, cloudflare has been contributing to a general degradation and shittification of the internet. I'm seeing a lot more "prove you're human", "checking to make sure you're human", and there is normally at the very least a delay of a few seconds before the site loads.

I don't think this is really helping the site owners. I suspect it's mainly about AI extortion:

https://blog.cloudflare.com/introducing-pay-per-crawl/

◧◩
134. ignora+is[view] [source] [discussion] 2025-12-05 17:29:25
>>flamin+q3
> this sounds like the sort of cowboy decision

Ouch. Harsh given that Cloudflare's being over-honest (to disabling the internal tool) and the outage's relatively limited impact (time wise & no. of customers wise). It was just an unfortunate latent bug: Nov 18 was Rust's Unwrap, Dec 5 its Lua's turn with its dynamic typing.

Now, the real cowboy decision I want to see is Cloudflare [0] running a company-wide Rust/Lua code-review with Codex / Claude...

cf TFA:

  if rule_result.action == "execute" then
    rule_result.execute.results = ruleset_results[tonumber(rule_result.execute.results_index)]
  end

  This code expects that, if the ruleset has action="execute", the "rule_result.execute" object will exist ... error in the [Lua] code, which had existed undetected for many years ... prevented by languages with strong type systems. In our replacement [FL2 proxy] ... code written in Rust ... the error did not occur.
[0] >>44159166
◧◩◪◨⬒
135. Pharao+3t[view] [source] [discussion] 2025-12-05 17:33:08
>>flamin+Nk
https://react.dev/blog/2025/12/03/critical-security-vulnerab...

Privately Disclosed: Nov 29 Fix pushed: Dec 1 Publicly disclosed: Dec 3

◧◩
137. archon+Bt[view] [source] [discussion] 2025-12-05 17:35:37
>>miyuru+E3
I recently ran into an issue with the Cloudflare API feature that if you want to roll back requires contacting the support team because there's no way to roll it back with the API or GUI. Even when the exact issue was pointed out, it took multiple days to change the setting and to my knowledge there's still no API fix available.

https://www.answeroverflow.com/m/1234405297787764816

◧◩
178. nova22+3I[view] [source] [discussion] 2025-12-05 18:40:22
>>paradi+q5
Speaking of fintech

https://www.henricodolfing.ch/case-study-4-the-440-million-s...

◧◩◪
189. cpncru+TK[view] [source] [discussion] 2025-12-05 18:52:00
>>james2+Br
>You call it extortion of the AI companies, but isn’t stealing/crawling/hammering a site to scrape their content to resell just as nefarious?

You can easily block ChatGPT and most other AI scrapers if you want:

https://habeasdata.neocities.org/ai-bots

◧◩◪◨
210. james2+ZQ[view] [source] [discussion] 2025-12-05 19:16:18
>>cpncru+TK
This is just using robots.txt and asking "pretty please, don’t scrape me".

Here is an article (from TODAY) about the case where Perplexity is being accused of ignoring robots.txt: https://www.theverge.com/news/839006/new-york-times-perplexi...

If you think a robots.txt is the answer to stopping the billion-dollar AI machine from scraping you, I don’t know what to say.

◧◩◪
230. markus+mX[view] [source] [discussion] 2025-12-05 19:46:48
>>65+Kw
TBF they are still hiring a lot of eng people from US/UK/EU:

https://www.cloudflare.com/careers/jobs/?department=Engineer...

◧◩◪
235. jameso+9Z[view] [source] [discussion] 2025-12-05 19:55:57
>>chatma+t8
The bad change wasn't even a deployment as such, just an entry in the global KV store https://blog.cloudflare.com/introducing-quicksilver-configur...

Actual deployments take hours to propagate worldwide.

(Disclosure: former Cloudflare SRE)

◧◩◪◨⬒⬓
236. toomuc+sZ[view] [source] [discussion] 2025-12-05 19:57:46
>>lockni+CT
If you are a customer of Cloudflare, and not happy, I encourage you to evaluate other providers more to your liking. Perhaps you'll find someone more fitting to your use case and operational preferences, but perhaps not. My day job org pays Cloudflare hundreds of thousands of dollars a year, and am satisfied with how they operate. Everyone has choice, exercise it if you choose. I'm sure your account exec would be happy to take the feedback. Feedback, including yours, is valuable and important to attempt to improve the product and customer experience (imho; i of course do not speak for Cloudflare, only myself).

As a recovering devops/infra person from a lifetime ago (who has, much to my heartbreak, broken prod more than once), perhaps that is where my grace in this regard comes from. Systems and their components break, systems and processes are imperfect, and urgency can lead to unexpected failure. Sometimes its Cloudflare, other times it's Azure, GCP, Github, etc. You can always use something else, but most of us continue to pick the happy path of "it works most of the time, and sometimes it does not." Hopefully the post mortem has action items to improve the safeguards you mention. If there are no process and technical improvements from the outage, certainly, that is where the failure lies (imho).

China-nexus cyber threat groups rapidly exploit React2Shell vulnerability (CVE-2025-55182) - https://aws.amazon.com/blogs/security/china-nexus-cyber-thre... - December 4th, 2025

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

◧◩◪◨⬒⬓⬔
289. XCSme+Ph1[view] [source] [discussion] 2025-12-05 21:30:35
>>al_bor+Bg1
Today a client is having some issue with Zoom because of some artificial rate limits they impose. Their support is not responding, the account can't be used, courses can not be held and there's not much we can do.

We already started looking into moving away from Zoom, I suggested self-hosting http://jitsi.org Based on their docs, self-hosting is well supported, and probably a $50-$100 server is more than enough, so a lot cheaper than Zoom.

◧◩◪
312. Techni+1o1[view] [source] [discussion] 2025-12-05 22:04:36
>>cnnliv+oh1
Thing is, these things are automated... Internally.

Which makes it feel that much more special when a service provides open access to all of the infrastructure diagnostics, like e.g. https://status.ppy.sh/

◧◩◪◨⬒⬓⬔⧯▣
318. XCSme+Yp1[view] [source] [discussion] 2025-12-05 22:17:32
>>carl_d+oo1
Artifical limits, because they have 40 paid licenses that they can not use, because of a non-disclosed assignment limit that is NOT mentioned in the pricing page nor in the ToS.

A lot of people are angry about this, and I think it's borderline illegal: https://devforum.zoom.us/t/you-have-exceeded-the-limit-of-li...

You pay for something, and you can't use it.

◧◩◪
345. hashst+Oy1[view] [source] [discussion] 2025-12-05 23:13:51
>>trippl+IX
See 1.1 and 1.2; you get some credit back for when they fail to deliver.

https://www.cloudflare.com/business-sla/

◧◩◪◨
353. yearol+bA1[view] [source] [discussion] 2025-12-05 23:22:36
>>bostik+Ck1
Critical high-level stats such as errors should be scraped more frequently than 30 seconds. It’s important to have multiple time granularity scraping intervals, a small set of most critical stats should be scraped closer to 10s or 15s.

Prometheus has as an unaddressed flaw [0], where rate functions must be at least 2x the scrape interval. This means that if you scrape at 30s intervals, your rate charts won’t reflect the change until a minute after.

[0] - https://github.com/prometheus/prometheus/issues/3746

378. telefo+rH1[view] [source] 2025-12-06 00:21:40
>>meetpa+(OP)
Internet packet switching based architecture was originally design to withstand this type of outages [1].

Some people even go further by speculating that the original military DARPA network precursor to the modern Internet was originally designed to ensure the continuity of command and control (C&C) of the US military operation in the potential event of all out nuclear attack during the Cold War.

This the time when Internet researchers need to redefine the Internet application and operation. The local-first paradigm is the first step in the right direction (pardon the pun) [2].

[1] The Real Internet Architecture: Past, Present, and Future Evolution:

https://press.princeton.edu/books/paperback/9780691255804/th...

[2] Local-first software You own your data, in spite of the cloud:

https://www.inkandswitch.com/essay/local-first/

◧◩◪◨⬒⬓
406. winddu+FN1[view] [source] [discussion] 2025-12-06 01:18:42
>>andrew+4A1
https://w3techs.com/technologies/overview/proxy, they are tiny compared to CF, their revenue is high because they focus on large enterprise clients.
◧◩◪◨⬒
420. rossju+iQ1[view] [source] [discussion] 2025-12-06 01:40:31
>>roguec+CQ
Relying on code ninja ego backfires way sooner, and way more often.

https://security.googleblog.com/2025/11/rust-in-android-move...

432. qoutea+mW1[view] [source] 2025-12-06 02:45:22
>>meetpa+(OP)
It's (at least) the second time Couldflage gets bitten by React. Last time an useEffect caused an incident.

https://blog.cloudflare.com/deep-dive-into-cloudflares-sept-...

◧◩◪◨⬒⬓⬔
450. cpncru+db2[view] [source] [discussion] 2025-12-06 05:44:44
>>albedo+w82
Actually, it looks like all the major ones do honour robots.txt including perplexity. They seemingly get around it using google serps, so theyre not actually crawling or hammering the site servers (or even cloudflare).

https://www.ailawandpolicy.com/2025/10/anti-circumvention-re...

◧◩
512. psycho+8S2[view] [source] [discussion] 2025-12-06 14:41:03
>>mixedb+Xv1
That's a reflect of social organisation. Pushing for hierarchical organisation with a few key centralising nodes will also impact business and technological decisions.

See also https://en.wikipedia.org/wiki/Conway%27s_law

◧◩◪◨⬒⬓
537. kortil+Cn3[view] [source] [discussion] 2025-12-06 18:52:41
>>morphe+hI1
https://creators.spotify.com/pod/profile/epicompliance/episo...
◧◩◪
551. cpncru+n24[view] [source] [discussion] 2025-12-07 00:47:51
>>chamom+CF3
Looking into this more, it does indeed seem to be a cloudflare problem. It looks like cloudflare made a significant error in their bot fingerprinting, and Perplexity wasn't actually bypassing robots.txt.

https://www.perplexity.ai/hub/blog/agents-or-bots-making-sen...

To be honest I find cloudflare a much more scammy company than Perplexity. I had a DDoS attack a few years ago which originated from their network, and they had zero interest in it.

[go to top]