zlacker

The bane of my existence: Supporting both async and sync code in Rust

submitted by lukast+(OP) on 2024-01-19 22:00:01 | 183 points 141 comments
[view article] [source] [go to bottom]

NOTE: showing posts with links only show all posts
◧◩◪
18. the__a+1g[view] [source] [discussion] 2024-01-19 23:27:31
>>bpye+Rf
Here's one: https://github.com/David-OConnor/stm32-hal

Device IC code generally ends as files in my firmware directly.

◧◩
24. herman+Ph[view] [source] [discussion] 2024-01-19 23:39:44
>>Laaas+Hf
The article has a whole section about the problems they ran into with two crates:

https://nullderef.com/blog/rust-async-sync/#_duplicating_the...

◧◩◪◨
31. sunsho+Gj[view] [source] [discussion] 2024-01-19 23:53:51
>>Twenty+6f
For the limited case of data-parallel computations as you're describing, you don't need async.

However, many real-world programs are inherently complicated state machines, where each individual state waits on one or more nested state machines to progress and/or complete -- async is often the most reasonable way to express that.

I wrote a post detailing a non-trivial production example use of async that might be interesting, regarding the cargo-nextest test runner that I wrote and maintain: https://sunshowers.io/posts/nextest-and-tokio/.

Nextest is not "web-scale" (c10k), since the number of tests one is concurrently running is usually bounded by the number of processor hyperthreads. So the fact that async tasks are more lightweight than OS threads doesn't have much bearing in this case. But even beyond that, being able to express state machines via async makes life so much simpler.

Over a year after switching nextest to using async, I have no regrets. The state machine has gotten around twice as complicated since then--for example, nextest now handles SIGTSTP and SIGCONT carefully--and async has coped admirably with it.

(There are currently somewhat serious issues with Rust async, such as a really messy cancellation story. But that's generally not what gets talked about on places like HN.)

◧◩
48. Britto+Zr[view] [source] [discussion] 2024-01-20 01:10:01
>>bruce3+Ko
There is this:

https://lunatic.solutions/

◧◩
49. stevek+4s[view] [source] [discussion] 2024-01-20 01:10:36
>>toolto+sh
I gave two talks about this:

* An overview of terminology, and a description of how various languages fit into the various parts of the design space https://www.infoq.com/presentations/rust-2019/

* A deep dive into what Rust does https://www.infoq.com/presentations/rust-async-await/

◧◩◪◨
58. sodali+su[view] [source] [discussion] 2024-01-20 01:31:12
>>doakes+Zt
I was calling await on ws_stream.next() - awaiting for there to be a new message.

Code: https://gist.github.com/sigaloid/d0e2a7eb42fed8c2397fbf84239...

In the example you give, yes, it's just sequential and everything relies on a previous thing. But say you are making a spotify frontend - you want to render a list of playlists, the profile info, etc - you call await on all of them and they can complete simultaneously.

59. SethML+dv[view] [source] 2024-01-20 01:37:26
>>lukast+(OP)
Nice! This is similar to the solution here: https://github.com/python-trio/unasync
◧◩◪
71. hedgeh+HA[view] [source] [discussion] 2024-01-20 02:40:20
>>boustr+4k
In the early days of Rust there was a debate about whether to support "green threads" and in doing that require runtime support. It was actually implemented and included for a time but it creates problems when trying to do library or embedded code. At the time Go for example chose to go that route, and it was both nice (goroutines are nice to write and well supported) and expensive (effectively requires GC etc). I don't remember the details but there is a Rust RFC from when they removed green threads:

https://github.com/rust-lang/rfcs/blob/0806be4f282144cfcd55b...

81. Arnavi+3F[view] [source] 2024-01-20 03:28:17
>>lukast+(OP)
Another option is to implement your API in a sans-io form. Since k8s-openapi was mentioned (albeit for a different reason), I'll point out that its API gave you a request value that you could send using whatever sync or async HTTP client you wanted to use. It also gave you a corresponding function to parse the response, that you would call with the response bytes your client gave to you.

https://github.com/Arnavion/k8s-openapi/blob/v0.19.0/README....

(Past tense because I removed all the API features from k8s-openapi after that release, for unrelated reasons.)

◧◩◪◨⬒⬓
97. lifthr+wU[view] [source] [discussion] 2024-01-20 06:58:11
>>thiht+qS
While the original use of "function colors" was purely syntactic [1], they can be easily remapped to cooperative vs. preemptitive multitasking. This remapping is important because they change programmer's mental model.

For example, the common form of `await` calls implies cooperative multitasking and people will have a good reason to believe that no other tasks can't affect your code between two `await` calls. This is not generally true (e.g. Rust), but is indeed true for some languages like JS. Now consider two variants of JS, where both had `await` removed but one retains cooperative multitasking and another allows preemptitive tasks. They will necessarily demand different mental models, even though it is no longer syntactically distinguishable. I believe this distinction is important enough that they still have to be considered to have a function color, which is only uniform within a single language.

Zig's approach in comparison is often called "color-blind", because while it provides `async` and `await`, those keywords only change the return type to a promise (Zig term: async frame) and do not guarantee that it will do anything different. Instead, users are given the switch so that most libraries are expected to work equally well regardless of that switch. You can alternatively think this as follows: all Zig modules are implicitly parametrized via an implicit `io_mode` parameter, which affect the meaning of `async` and `await` and propagate to nested dependencies. There is definitely a color here, but it's no longer a function color because functions can no longer paint themselves. So I think it's reasonable to call this to have no function color.

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

◧◩◪◨⬒
113. SkiFir+Rh1[view] [source] [discussion] 2024-01-20 12:00:55
>>lifthr+ME
Last time I checked Zig still had a subtle form function coloring for function pointers. See for example https://github.com/ziglang/zig/issues/8907
◧◩◪
141. winrid+P16[view] [source] [discussion] 2024-01-22 01:34:11
>>nullde+Uv4
It's a type-safe macro that allows you to use async or sync system calls transparently: https://nim-lang.org/blog/2016/09/30/version-0150-released.h...
[go to top]