zlacker

>>ZacnyL+(OP)
This is interesting.

Judging from the screenshot, a huge amount of GET /TweetDetail is generated which triggers some rate limiting, as shown by the 429.

If this is indeed due to the recent decision to enforce authentication for all API calls, it means the curlprit may actually be the API gateway or something similar downstream.

Also, this behavior seem to never stop, which isn't what one would expect from an exponential backoff retry.

I don't claim to be a better engineer than the folks working at Twitter, but it is interesting to see something like this in the wild, all Musk-related considerations aside.

>>arter4+ab
Did Elon pay the AWS bill? That seems like a likely culprit. Twitter instances are being forcibly shutdown.

>>cactus+cs
Twitter operates its own datacenters.

>>amluto+bt
"Twitter and AWS signed a five-and-a-half-year contract in 2020, which AWS is not willing to renegotiate."

https://gritdaily.com/twitter-owes-aws-millions/

>>cactus+4u
Twitter.com and the associated user-facing services do not run on AWS.

>>willia+5v
While it looks like they never started the move over to AWS, the press release makes it sound like they do use some AWS services.

> In addition, Twitter will continue to use AWS services such as Amazon CloudFront (AWS’s fast content delivery network service that securely delivers data, videos, applications, and APIs with low latency and high transfer speeds to customers globally) and Amazon DynamoDB (AWS’s key-value database that delivers single-digit millisecond performance at any scale).

>>vGPU+GR
I worked there. Services running on GCP are a significant part of the internal service infra (ml platform, etc.) and it's not impossible that the abrupt loss of GCP would cause user-facing problems. The GCP spend was many, many times the AWS spend. Unless things changed since last November, AWS is not a meaningful part of the internal or user-facing infra.

With respect to DynamoDB specifically, Twitter has its own custom distributed key-value store: https://blog.twitter.com/engineering/en_us/a/2014/manhattan-... that twitter.com itself runs on.

>>willia+CS
Thanks for weighing in with some actual first-hand knowledge. It is appreciated.

The latest on cloud hosting is from a week ago, and I'm guessing you don't have any more recent info than this:

https://www.reuters.com/technology/twitter-resumes-paying-go...

>>18pfsm+UW
Correct, no more recent (or less public) info than that. Like I say, losing GCP could cause problems users notice, but sounds like that’s not going to happen.