zlacker

Collision is perhaps the wrong word. But llms definitely have trouble disambiguating different symbols of a language that map to similar tokens.

Way back in the gpt3.5 days I could never get the model to do a parse of even the simplest grammar until I replaced the one letter production rules with one word production rules, e.g. S vs Start. A bit like how they couldn't figure out the number of rs in strawberry.