"Language" is a subset of "symbols". I agree with what you said, but it's not representative of the quote in GP.
Just a few days ago was "What do you visualize while programming?", and there's a few of us in the comments that, when programming, think symbolically without language: >>41869237