zlacker

[parent] [thread] 21 comments
1. toolto+(OP)[view] [source] 2024-01-19 23:36:57
The comment threads here discuss async in many different languages: Rust, Go, JavaScript, Python. Can somebody knowledgeable describe how they are subtly different between languages? Why are they painful in some but not in others?

Is there already an article that describes this well?

replies(4): >>vlovic+R1 >>boustr+C2 >>stevek+Ca >>Too+fD
â—§
2. vlovic+R1[view] [source] 2024-01-19 23:49:54
>>toolto+(OP)
They’re painful in all contexts because of function coloring. They’re slightly less painful in Go and JS because there’s a single opinionated async runtime built in. In Rust they have yet to standardize a bunch of stuff that would remove the pain:

async traits in std instead of each runtime having their own,

a pluggable interface so that async in code doesn’t have to specify what runtime it’s being built against

potentially an effect system to make different effects composable more easily (eg error effects + async effects) without needing to duplicate code to accomplish composition

Keyword generics as the current thing being explored instead of an effect system to support composition of effects

With these fixes async rust will get less annoying but it’s slow difficult work.

replies(3): >>yawara+pj >>anonym+Nj >>pcwalt+6n
â—§
3. boustr+C2[view] [source] 2024-01-19 23:57:54
>>toolto+(OP)
I think the biggest underlying difference is that Rust does not have a language runtime, whereas the other three you've listed do. Since the language runtime can preempt your code at any time, it becomes a lot easier to make async work - at the expense that now data races are easier to create.

I'm not going to pretend I'm an expert but would be happy if someone could expand further.

replies(1): >>hedgeh+fj
â—§
4. stevek+Ca[view] [source] 2024-01-20 01:10:36
>>toolto+(OP)
I gave two talks about this:

* An overview of terminology, and a description of how various languages fit into the various parts of the design space https://www.infoq.com/presentations/rust-2019/

* A deep dive into what Rust does https://www.infoq.com/presentations/rust-async-await/

â—§â—©
5. hedgeh+fj[view] [source] [discussion] 2024-01-20 02:40:20
>>boustr+C2
In the early days of Rust there was a debate about whether to support "green threads" and in doing that require runtime support. It was actually implemented and included for a time but it creates problems when trying to do library or embedded code. At the time Go for example chose to go that route, and it was both nice (goroutines are nice to write and well supported) and expensive (effectively requires GC etc). I don't remember the details but there is a Rust RFC from when they removed green threads:

https://github.com/rust-lang/rfcs/blob/0806be4f282144cfcd55b...

â—§â—©
6. yawara+pj[view] [source] [discussion] 2024-01-20 02:41:51
>>vlovic+R1
There's no function colouring in Go. Async functions don't have any special colour.
replies(1): >>SkiFir+831
â—§â—©
7. anonym+Nj[view] [source] [discussion] 2024-01-20 02:45:24
>>vlovic+R1
Go doesn't have function coloring. Greenlet, Lua, and libco solve the problem without function coloring by adding a stack-switching primitive. Zig solves the problem without forcing function coloring on all consumers of functions by having the compiler monomorphize functions based on whether they end up being able to suspend.
replies(1): >>lifthr+kn
â—§â—©
8. pcwalt+6n[view] [source] [discussion] 2024-01-20 03:24:28
>>vlovic+R1
For Go I'd say there's a single synchronous runtime built-in. People say that Go is async because the implementation of goroutines is async internally, but the implementation of threads on every OS is async internally too. The only real difference as far as sync/async is concerned† between goroutines and threads is that Go's implementation of goroutines is in userspace, while the implementation of OS threads is in kernel space. Both are equally asynchronous under the hood.

† Yes, there are other differences between goroutines and typical OS threads, such as stack sizes, but I'm only talking about I/O differences here.

â—§â—©â—ª
9. lifthr+kn[view] [source] [discussion] 2024-01-20 03:26:05
>>anonym+Nj
A more accurate description would be that Go has a single function color, that is namely green. This distinction is important because, for example, C also has no function coloring problem only because it doesn't care about lightweight threading, i.e. its function color is always red. Only Zig's approach, and hopefully Rust's keyword generics if accepted, can be considered to have no function color.
replies(2): >>thiht+YA >>SkiFir+p01
◧◩◪◨
10. thiht+YA[view] [source] [discussion] 2024-01-20 06:23:32
>>lifthr+kn
I don’t understand the difference between "single color" and "no color", can you explain? What makes Zig’s approach colorless?
replies(1): >>lifthr+4D
◧◩◪◨⬒
11. lifthr+4D[view] [source] [discussion] 2024-01-20 06:58:11
>>thiht+YA
While the original use of "function colors" was purely syntactic [1], they can be easily remapped to cooperative vs. preemptitive multitasking. This remapping is important because they change programmer's mental model.

For example, the common form of `await` calls implies cooperative multitasking and people will have a good reason to believe that no other tasks can't affect your code between two `await` calls. This is not generally true (e.g. Rust), but is indeed true for some languages like JS. Now consider two variants of JS, where both had `await` removed but one retains cooperative multitasking and another allows preemptitive tasks. They will necessarily demand different mental models, even though it is no longer syntactically distinguishable. I believe this distinction is important enough that they still have to be considered to have a function color, which is only uniform within a single language.

Zig's approach in comparison is often called "color-blind", because while it provides `async` and `await`, those keywords only change the return type to a promise (Zig term: async frame) and do not guarantee that it will do anything different. Instead, users are given the switch so that most libraries are expected to work equally well regardless of that switch. You can alternatively think this as follows: all Zig modules are implicitly parametrized via an implicit `io_mode` parameter, which affect the meaning of `async` and `await` and propagate to nested dependencies. There is definitely a color here, but it's no longer a function color because functions can no longer paint themselves. So I think it's reasonable to call this to have no function color.

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

replies(1): >>thiht+HO
â—§
12. Too+fD[view] [source] 2024-01-20 07:00:06
>>toolto+(OP)
JS makes it easier because it was always single threaded and never had any sync IO to begin with.

This means, before async existed, any library doing IO had to be based on callbacks. Then came Promises, which are essentially glorified callbacks and then came async which can be seen syntax sugar for Promises.

So you will never see synchronous code that depends on an asynchronous result. The concept of sync code waiting for something just never existed in JavaScript. Instead you wake up your sync functions with Promise.then()-callbacks and that same mechanism bridges async functions.

It’s also very rare to have compute heavy sync code in JS so there is rarely any need to run it multi threaded.

replies(1): >>penter+eG
â—§â—©
13. penter+eG[view] [source] [discussion] 2024-01-20 07:48:52
>>Too+fD
> The concept of sync code waiting for something just never existed in JavaScript.

Have you forgotten prompt() and friends?

◧◩◪◨⬒⬓
14. thiht+HO[view] [source] [discussion] 2024-01-20 09:59:35
>>lifthr+4D
Interesting thinking, thanks for the details! I’ll have to look into Zig’s async mode, it seems pretty good but I’m wondering what are the drawbacks of this approach (and specifically, why would you ever set io_mode to "sync")
replies(1): >>anonym+Cd3
◧◩◪◨
15. SkiFir+p01[view] [source] [discussion] 2024-01-20 12:00:55
>>lifthr+kn
Last time I checked Zig still had a subtle form function coloring for function pointers. See for example https://github.com/ziglang/zig/issues/8907
â—§â—©â—ª
16. SkiFir+831[view] [source] [discussion] 2024-01-20 12:22:42
>>yawara+pj
IMO saying there's no function coloring in a language ignores a lot of details.

In Go there's no function coloring because there are only async functions. That's why they don't get any special color, they are the only color. In Go you don't get to use sync functions, which creates problems e.g. when you need to use FFI, because the C ABI is the exact opposite and doesn't have function coloring because it only allows you to use sync functions.

Zig and Rust's async-generic initiative are a bit different in that they want to allow functions to be both sync and async at the same time. Ultimately there are still colors, but you don't have to choose one of them when you write a function. However IMO there are a lot of non-trivial problems to solve to get to that result.

Ultimately Go's approach work well enough, and usually better than other approaches, until you need to do FFI or you need to produce a binary without a runtime (e.g. if you need to program a microcontroller)

replies(3): >>gpdere+LX1 >>65a+1d3 >>yawara+No3
◧◩◪◨
17. gpdere+LX1[view] [source] [discussion] 2024-01-20 18:11:47
>>SkiFir+831
That's a bit of nonsense. As far as I know, all functions are sync in go. The fact that they are implemented async in the runtime with an user-space scheduler is irrelevant (you could otherwise make the point that there are truly no sync functions).

If we call the go programming model async, the word has completely lost all meanings.

replies(1): >>SkiFir+Ux2
◧◩◪◨⬒
18. SkiFir+Ux2[view] [source] [discussion] 2024-01-20 21:45:03
>>gpdere+LX1
What is the difference between a sync and an async function for you then?
replies(1): >>gpdere+u23
◧◩◪◨⬒⬓
19. gpdere+u23[view] [source] [discussion] 2024-01-21 01:43:06
>>SkiFir+Ux2
An async function is on CPS form and return it's result via a return continuation. Typically when invoked from a non CPS function it also forks the thread of execution.

These days async functions are also typically lazily evaluated via partial evaluation and the return continuation is not necessarily provided at the call site.

A sync function provides it's result via the normal return path.

◧◩◪◨
20. 65a+1d3[view] [source] [discussion] 2024-01-21 03:29:24
>>SkiFir+831
Go works fine for C FFI, all of its problems there are caused by its innovation wrt dynamic stack sizes and having a garbage collector. I'd rather write multithreaded Go FFI than deal with JNI again, anyway. There isn't really a language keyword-level concept of async in Go that's comparable to `await` in JS, or Java futures, or Rust async.
◧◩◪◨⬒⬓⬔
21. anonym+Cd3[view] [source] [discussion] 2024-01-21 03:37:27
>>thiht+HO
This post is a bit confused. io_mode is an stdlib feature that changes some blocking stuff in the stdlib to use the stdlib's event loop. There's not such a thing for most libraries.

For typical libraries, functions and structs are provided, and to the extent that these functions call functions provided by the user, they are generic over the async-ness of those functions. That's how the language-level async feature works, for library code that doesn't specify that it is async and doesn't specify that it would like to use a specific non-async calling convention for calling the user's callbacks.

◧◩◪◨
22. yawara+No3[view] [source] [discussion] 2024-01-21 05:53:49
>>SkiFir+831
> IMO saying there's no function coloring in a language ignores a lot of details.

Details which are meant to be ignored. When you use async/await constructs in various languages, you don't care about the fact that they are desugared into callback chains under the hood. You either do async/await in a language or you don't. That's what the concept of 'your function has a colour' means. If you want to change the meaning, OK but then you're talking about something else.

[go to top]