zlacker

[parent] [thread] 1 comments
1. goose0+(OP)[view] [source] 2026-01-12 09:42:13
Looks like my tokenization review method was incorrect - honestly a little embarrassing on my part. I think it would have been a lot longer before I discovered it, so thanks for the comment!

I did just go through and ran equivalent code samples in the GlyphLang repo (vs the sample code I posted that I'm assuming you ran) through tiktoken and found slightly lower percentages, but still not insignificant: on average 35% fewer than Python and 56% fewer than Java. I've updated the README with the corrected figures and methodology if you want to check: https://github.com/GlyphLang/GlyphLang/blob/main/README.md#a...

replies(1): >>p0w3n3+xi3
2. p0w3n3+xi3[view] [source] 2026-01-13 09:25:32
>>goose0+(OP)
Yeah Java IS verbose. Thanks!
[go to top]