zlacker

How does misalignment scale with model intelligence and task complexity?

submitted by salkah+(OP) on 2026-02-03 00:28:06 | 241 points 79 comments
[view article] [source] [go to bottom]

NOTE: showing posts with links only show all posts
2. gopalv+y4[view] [source] 2026-02-03 00:57:39
>>salkah+(OP)
> Making models larger improves overall accuracy but doesn't reliably reduce incoherence on hard problems.

Coherence requires 2 opposing forces to hold coherence in one dimension and at least 3 of them in higher dimensions of quality.

My team wrote up a paper titled "If You Want Coherence, Orchestrate a Team of Rivals"[1] because we kept finding that upping the reasoning threshold resulted in less coherence - more experimentation before we hit a dead-end to turn around.

So we had a better result from using Haiku (we fail over to Sonnet) over Opus and using a higher reasoning model to decompose tasks rather than perform each one of them.

Once a plan is made, the cheaper models do better as they do not double-think their approaches - they fail or they succeed, they are not as tenacious as the higher cost models.

We can escalate to higher authority and get out of that mess faster if we fail hard and early.

The knowledge of how exactly failure happened seems to be less useful to the higher reasoning model over the action biased models.

Splitting up the tactical and strategic sides of the problem, seems to work similarly to how Generals don't hold guns in a war.

[1] - https://arxiv.org/abs/2601.14351

◧◩◪
21. jagged+xl[view] [source] [discussion] 2026-02-03 02:53:02
>>crabmu+p8
I believe that the issue right now is that we're using languages designed for human creation in an AI context. I think we probably want languages that are optimized for AI written but human read code, so the surface texture is a lot different.

My particular hypothesis on this is something that feels a little bit like python and ruby, but has an absolutely insane overkill type system to help guide the AI. I also threw in a little lispiness on my draft: https://github.com/jaggederest/locque/

25. leahth+uo[view] [source] 2026-02-03 03:18:11
>>salkah+(OP)
It's nice seeing this with Sohl-Dickstein as the last author after reading this blog post from him some time ago: https://sohl-dickstein.github.io/2023/03/09/coherence.html
◧◩
34. dang+av[view] [source] [discussion] 2026-02-03 04:19:19
>>throwp+G5
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

◧◩
66. anupam+xL1[view] [source] [discussion] 2026-02-03 14:19:33
>>anupam+lF
Following up - I built a tool "wobble"[1] to measure this: parses ~/.claude/projects/*.jsonl session transcripts, extracts skill invocations + actual commands executed, calculates Bias/Variance per the paper's formula.

Ran it on my sessions. Result: none of skills scored STABLE. The structural predictors of high variance: Numbered steps without clear default, Options without (default) marker, Content >4k chars (overthinking zone), Missing constraint language

[1] https://github.com/anupamchugh/shadowbook (bd wobble)

[go to top]