zlacker

[return to "GitHub experience various partial-outages/degradations"]
1. llama0+t8[view] [source] 2026-02-02 22:02:47
>>bhoust+(OP)
Looks like Azure as a platform just killed the ability for VM scale operations, due to a change on a storage account ACL that hosted VM extensions. Wow... We noticed when github actions went down, then our self hosted runners because we can't scale anymore.

Information

Active - Virtual Machines and dependent services - Service management issues in multiple regions

Impact statement: As early as 19:46 UTC on 2 February 2026, we are aware of an ongoing issue causing customers to receive error notifications when performing service management operations - such as create, delete, update, scaling, start, stop - for Virtual Machines (VMs) across multiple regions. These issues are also causing impact to services with dependencies on these service management operations - including Azure Arc Enabled Servers, Azure Batch, Azure DevOps, Azure Load Testing, and GitHub. For details on the latter, please see https://www.githubstatus.com.

Current status: We have determined that these issues were caused by a recent configuration change that affected public access to certain Microsoft‑managed storage accounts, used to host extension packages. We are actively working on mitigation, including updating configuration to restore relevant access permissions. We have applied this update in one region so far, and are assessing the extent to which this mitigates customer issues. Our next update will be provided by 22:30 UTC, approximately 60 minutes from now.

https://azure.status.microsoft/en-us/status

◧◩
2. bob102+vq[view] [source] 2026-02-02 23:07:01
>>llama0+t8
They've always been terrible at VM ops. I never get weird quota limits and errors in other places. It's almost as if Amazon wants me to be a customer and Microsoft does not.
◧◩◪
3. dgxyz+qz[view] [source] 2026-02-02 23:47:48
>>bob102+vq
Amazon isn't much better there. Wait until you hit an EC2 quota limit and can't get anyone to look at it quickly (even under paid enterprise support) or they say no.

Also had a few instance types which won't spin up in some regions/AZs recently. I assume this is capacity issues.

◧◩◪◨
4. pauldd+pT[view] [source] 2026-02-03 01:52:38
>>dgxyz+qz
The cloud isn’t some infinite thing.

There’s a bunch of hardware, and they can’t run more servers than they have hardware. I don’t see a way around that.

◧◩◪◨⬒
5. Imusta+ET1[view] [source] 2026-02-03 10:25:12
>>pauldd+pT
Really prefer Hetzner in this sense because they actually talk about limits. I recently got myself a hetzner account (after shilling it for so much, hearing positivity, I felt like it was time for me to discover it)

I wanted to try out the most cheapest option out of frugality & that was actually limited (but kudos to them that they mentioned that these servers have limits) so no worries I went and picked the 5.99 euro instead of the 3.99 euro option instead.

They also have limits option itself as a settings iirc and it shows you all the limits that are imposed in a transparent manner and my account's young so I can't request for limit increases but after some time, one definitely can.

Essentially I love this idea because essentially Cloud is just someone's else's hardware and there is no infinitium. But I feel as if it can come pretty close with hetzner (and I have heard some great things about OVH and have a good personal experience with netcup vps but netcup's payments were really PITA to setup]

◧◩◪◨⬒⬓
6. direwo+092[view] [source] 2026-02-03 12:21:02
>>Imusta+ET1
Hetzner is a dedicated server (meaning monthly contract, 1 month setup fee and up to 1 week delivery time) company that branched out into cloud, so it's not that surprising they treat cloud a bit like that. While Amazon wants you to think they have an infinite capacity pool, and any failure to get a server is an unexpected error, Hetzner seems to not hide they have a finite number of servers in a finite number of racks, since that's how their main business works.
◧◩◪◨⬒⬓⬔
7. Imusta+NP2[view] [source] 2026-02-03 16:03:54
>>direwo+092
I guess its understandable now the reasons why Amazon might want to do this.

Similar to hetzner, I haven't used OVH but does it also have limits or how do they follow?

Out of pure curiosity, Is there anything aside from the three hyperscaler trifecta which doesn't show limits too?

[go to top]