zlacker

>>tehnub+(OP)
This is kind of just a measurement of how representative a language is in the distribution of the tokenizer training. You could have a single token equal to “public static void main”.

>>janals+Xj
You could, but you wouldn't when those keywords can all change in equivalent contexts.

>>muyuu+rk
What do you mean?

`public` might have a token by itself, even though you can have `pub` occurring in other contexts, too.