Based on my experience with toddlers, a rather smart dog, and my own thought processes, I disagree that language is a fundamental component of abstraction. Of sharing abstractions, sure, but not developing them.
When I'm designing a software system I will have a mental conception of the system as layered abstractions before I have a name for any component. I invent names for these components in order to define them in the code or communicate them to other engineers, but the intuition for the abstraction comes first. This is why "naming things" is one of the hard problems in computer science—because the name comes second as a usually-inadequate attempt to capture the abstraction in language.
Remember in CS theory, a language is just a set of strings. If you think in pictures that is STILL a language if your pictures are structured.
So I'm really handwaving the above just to suggest that it all depends on the assumptions that each expert is making in elucidating this debate which has a long history.
Unless we're getting metaphysical to the point of describing quantum systems as possessig a language, there are various continuous analog systems that can compute without a formal grammar. The language system could be the one that thinks in discrete 'tokens'; the conscious system something more complex.
There is no reason to assume consciousness is Turing computable [1].
[1] https://en.m.wikipedia.org/wiki/Church%E2%80%93Turing_thesis
[1] Mark Bickford, Liron Cohen, Robert L. Constable, and Vincent Rahli. 2018. Computability Beyond Church-Turing via Choice Sequences. In Proceedings of the 33rd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS '18). Association for Computing Machinery, New York, NY, USA, 245–254. https://doi.org/10.1145/3209108.3209200