Not loading for me at all, but status page shows green across the board.
edit: The outage is now acknowledged on the status page https://www.githubstatus.com/
edit: EU folks appear to have things working so it looks like a regional network fault
route-views>sh bgp 140.82.113.0
BGP routing table entry for 140.82.113.0/24, version 62582026
Paths: (19 available, best #4, table default)
Edit: deleted traceroute
Just flipped to red.
See here: https://www.githubstatus.com/incidents/gqx5l06jjxhp
>Investigating - We are currently experiencing an outage of GitHub products and are investigating.
>Jun 29, 2023 - 17:52 UTC
Maybe you want human confirmation on historic figures, but the live thing might as well be live.
Lunch time.
Edit: Front page still loads and I am logged in. Everything is as normal. Status page shows everything is down. Lol.
6 lag-26.nycmny837aw-bcr00.netops.charter.com (24.30.201.130) 158.033 ms
lag-16.nycmny837aw-bcr00.netops.charter.com (66.109.6.74) 29.575 ms
lag-416.nycmny837aw-bcr00.netops.charter.com (66.109.6.10) 30.077 ms
7 lag-1.pr2.nyc20.netops.charter.com (66.109.9.5) 81.351 ms 37.879 ms 27.877 ms
8 * * *(Not directed at GitHub specifically, but at bogus status pages.)
For the 30 seconds where you wait for failover to complete: that is a 30 second outage. It's not necessarily profitable to admit to it, but showing it as a 30 second outage would be accurate
In case anyone questions whether centralizing literally everything onto GitHub is a good idea, at least as a mirror for things you depend on
Haha! Happy to see impostor syndrome goes all the way to the top of the hierarchy.
lb-140-82-121-3-fra.github.com
which from the distance and name I would assume is a frankfurt based load balancer. I get there from BT -> Zayo
I can reach that IP from Washington too, but github returns 140.82.114.3 and 140.82.114.4 from DNS at 1.1.1.1 on a Level3 handoff in Washington
Spot checks around the place show the first returned IP as pingable across the world
Bangkok, Dhaka, Jakarta - 20.205.243.166
Seoul - 20.200.245.247
Nairobi - 20.87.225.212
Kabul, Dakar, Amman, Amman, Cairo - 140.82.121.3
Moscow, Riga, Istanbul - 140.82.121.4
Miami - 140.82.114.3
I am the lead dev on two projects.
I think I'll believe them when they say it's down, no offense.
I'm always evangelizing Argo or Flux and some self-hosted Gitlab or gitea, but seems like they all prefer to throw their money at Github as of now.
It's not fake, it's just a human process. And automating this would be error prone just the same.
I know he was still right in a way, who knows when git.lain.faith will just disappear. But still. I'm going to send some texts to bother him right now, hahaha.
We have identified the root cause of the outage and are working toward mitigation Posted 4 minutes ago. Jun 29, 2023 - 18:02 UTC
Very infuriating, that.
There's stuff like this that can't be automated well. The automated result is far worse than the human-based alternative.
this, ladies and gentlemen, is why you should always self host critical infrastructure
-False positives -Short outages that last a minute or three
Ultimately, SLA's and uptime guarantees. That way, a business can't automatically tally every minute of publicly admitted downtime against the 99.99999% uptime guarantee, and the onus to prove a breach of contract is on the customer
Sorry to bother!
If you are down for 1 hour a year on self hosting, but Office 364 is down 3 days a year, your CEO is going to be more understanding of the Office outage as all his golf buddies have the same problem, and he reads about it in the NYT.
But in any case zero downtime is difficult, that's why you need two independent systems. I had a a 500 microsecond outage at the weekend when a circuit failed which caused an business affecting incident, not a big one fortunately, as it was only some singers, but it was still one that was unacceptable -- had it happened at a couple of other events in the last 12 months it would have been far more problematic. Work has started to ensure it doesn't happen next year.
[0] >>35967921
I've also never been fired, so, it isn't always linked to trauma from past firings.
E.g. just yesterday for a short time frame of a few hours maybe half a day or so they had a bug where some closed PRs where shown in the personal which show created _not closed_ PRs.
Or github CI having spurious job cancellations or sometimes on a job failing waits until some (quite long) timeout is reached before reporting it.
Or it temporary being (partial or fully) down for a few hours.
Or it's documentation even through rather complete somehow managing to be often rather inconvenient to use. Oh wait that's not a bug, just subtle bad design, like it's PR overview/filters. Both cases of something which seems right on the first look, but starts being more and more inconvenient the more you use it. A trend I would argue describes GitHub as a whole rather well.
Check out this Ted talk from the co-founder of Atlassian.
https://www.ted.com/talks/mike_cannon_brookes_how_you_can_us...