zlacker

I see lots of engineers who argue that queues are bad and cause more problems than they solve. "Queues don't have backpressure" is a common complaint. It wouldn't be a difficult thing to add to a queue interface, but people don't do it despite the relative ease of the task. It is pretty easy to get a cardinality estimate on the queue size of any given queue even as it's rapidly increasing and decreasing in value.

The reason why is that it's probably a bad idea in the first place to mix your control and data interfaces into an inextricable knot. Since you're not going to be able to institute rate limiting at a sub-millisecond latency in a distributed system anyways, why isn't your control plane separate and instrumented?

Once you have a separate control plane, you can introduce back-pressure in many different ways and do so with a better understanding of the dissemination of those throttling values throughout your system.

So what I see are engineers who are actually ignoring the the architectural decision with at least as many implications as "queue vs diffuse interface," namely "control and data or just one monolithic system."