zlacker

[parent] [thread] 4 comments
1. sunsho+(OP)[view] [source] 2024-01-19 23:53:51
For the limited case of data-parallel computations as you're describing, you don't need async.

However, many real-world programs are inherently complicated state machines, where each individual state waits on one or more nested state machines to progress and/or complete -- async is often the most reasonable way to express that.

I wrote a post detailing a non-trivial production example use of async that might be interesting, regarding the cargo-nextest test runner that I wrote and maintain: https://sunshowers.io/posts/nextest-and-tokio/.

Nextest is not "web-scale" (c10k), since the number of tests one is concurrently running is usually bounded by the number of processor hyperthreads. So the fact that async tasks are more lightweight than OS threads doesn't have much bearing in this case. But even beyond that, being able to express state machines via async makes life so much simpler.

Over a year after switching nextest to using async, I have no regrets. The state machine has gotten around twice as complicated since then--for example, nextest now handles SIGTSTP and SIGCONT carefully--and async has coped admirably with it.

(There are currently somewhat serious issues with Rust async, such as a really messy cancellation story. But that's generally not what gets talked about on places like HN.)

replies(2): >>mplanc+rn >>Twenty+KD2
2. mplanc+rn[view] [source] 2024-01-20 03:56:19
>>sunsho+(OP)
Not who you’re replying to, but this is great context, and I want to thank you for including it. As another heavy async user (for a network service that handles loads of requests and does loads of DB reads and writes), I am also a big fan of Rust’s async at scale. We’re currently in the process of seeing where we can get rid of async_trait with 1.75, which has not been particularly drop-in in many cases but which is still exciting.

Anyway, I have been meaning to try out nextest for our big honking monorepo workspace at work. The cargo test runner has always been essentially fine for our needs, but speeding up test execution in CI could be a huge win for us.

replies(1): >>sunsho+5q
◧◩
3. sunsho+5q[view] [source] [discussion] 2024-01-20 04:28:47
>>mplanc+rn
I'd love to hear how it goes!
4. Twenty+KD2[view] [source] 2024-01-20 22:40:06
>>sunsho+(OP)
Thank you, this was a very insightful blog post!
replies(1): >>sunsho+jX2
◧◩
5. sunsho+jX2[view] [source] [discussion] 2024-01-21 01:12:38
>>Twenty+KD2
No problem! I wasn't totally convinced of the value of async myself, before I went through the exercise of building something that is in many ways more complicated than most web services.
[go to top]