Beyond Semantics: Unreasonable Effectiveness of Reasonless Intermediate Tokens

>>ngruhn+0o
It's also worth pointing out that Winger's (position) paper[0] is really about something that would sound silly today. He's arguing that we should use math to drive physics. Today, many people think these are indistinguishable things and you'll get into arguments about math being invented or discovered. But Winger is talking about how mathematics provides us with a framework where we can drive physics forward through theory instead of relying purely upon experimentation to poke and prod the universe.

It is rather "unreasonable" to think we can explore the world simply through pen and paper, from the comfort of a chair. You'd think you'd need to go out and touch grass, but incredibly this is not necessary.

  | The first point is that the enormous usefulness of mathematics in the natural sciences is something bordering on the mysterious and that there is no rational explanation for it. Second, it is just this uncanny usefulness of mathematical concepts that raises the question of the uniqueness of our physical theories.

Which is exactly why a lot of these other things are overused. Hamming's seems like an extension or corollary[1] and I even think Norvig's (Halevy's) is highly appropriate[2]. It is "unreasonable" to think these things would be effective.

  -------------------------------------

With this paper?

I think is fine. It is being used in a similar way to Winger, with similar context.

I can see two camps. One has always interpreted the COT as analogous to a model's internal dialogue. While the other has always thought there's a much larger gap between the manipulations within latent representations and what has been decoded, not necessarily needing be strongly aligned.[3] To the former, the results here would be shocking, while to the latter it is "yes, and?" Clearly they're addressing the former camp. There were plenty of people that Winger did not need to convince.

I'm of the latter camp[4], and I'm happy people are not just asserting and are demonstrating. Honestly, I'm even frequently upset when works get dismissed because they "demonstrate something we already knew" but no one had ever actually demonstrated. The proofs and evidencing is more important than the answer. Quite often we're highly certain about results but they are difficult to even evidence (let alone prove). I mean it would be quite silly to dismiss a proof that P != NP, even though the vast majority of us have long been convinced that this is the relationship we'll end up with. Yet, no one's done it.

  -------------------------------------

[0] https://web.archive.org/web/20210212111540/http://www.dartmo...

[1] https://math.dartmouth.edu/~matc/MathDrama/reading/Hamming.h...

[2] https://static.googleusercontent.com/media/research.google.c...

[3] Both camps can be further broken down too. Lots of nuances and opinions here and the lines really get fuzzy as we try to make it more accurate. I don't want to pretend there's a hard defining line, but the distinction helps the discussion and I think is reasonably accurate enough. Let me know if you think it is a gross mischaracterization.

[4] I can expand more why this side seems "obvious" to me. But a warning, you can probably guess I'm not good at being terse.

[Note]: I'd even go so far as say we should revisit Winger's argument around AI. I'm certain mathematics can be and will be "unreasonably effective." But not enough time has been dedicated to formulate the right type of math to use. We really do have to invent a new kind here. This may sound weird to non-mathematicians, but even physics uses multiple kinds of mathematics. The operations, fields, and algebras you use in one part may not be appropriate in another part. That's okay. But we don't have a TOE yet either, and that's a critical part of finding a TOE, is bringing all this together.

>>godels+OD
>It's also worth pointing out that Winger's (position) paper[0] is really about something that would sound silly today. He's arguing that we should use math to drive physics.

I think you misinterpret what it's about. He's pointing out how remarkable it is that the universe obeys laws like E=MC^2 exactly as far as we can tell which is not something you would probably expect just from looking at the world. The pre scientific understanding of the world was it was driven my gods and spirits. The mathematical laws were only discovered by scientific investigation.

Or as he puts it:

>The miracle of the appropriateness of the language of mathematics for the formulation of the laws of physics is a wonderful gift which we neither understand nor deserve.

If he was just saying use maths it would be boring and not a famous paper 65 years on.

zlacker