zlacker

>>amrrs+(OP)
I think one of the reason we are confused about what LLMs can do is because they use language. And we look at the "reasoning traces" and the tokens there look human, but what is actually happening is very alien to us, as shown by "Biology of Large Language Models"[1] and "Safety Alignment Should Be Made More Than Just a Few Tokens Deep"[2]

I am struggling a lot to see what the tech can and can not do, particularly designing systems with them, and how to build systems where the whole is bigger than the sum of its parts. And I think this is because I am constantly confused by their capabilities, despite understanding their machinery and how they work, their use of language just seems like magic. I even wrote https://punkx.org/jackdoe/language.html just to remind myself how to think about it.

I think this kind of research is amazing and we have to spend tremendous more effort into understanding how to use the tokens and how to build with them.

[1]: https://transformer-circuits.pub/2025/attribution-graphs/bio...

[2]: https://arxiv.org/pdf/2406.05946

>>jackdo+vm1
> build systems where the whole is bigger than the sum of its parts.

Any “product” can be thought of this way.

Of systems there are many systems nested within systems, yet a simple singular order “emerges”, usually it is the designed intended function.

The trick to discerning systems lies in their relationships.

Actors through interfaces have a relationship (usually more than one so think of each relationship as its own system dynamic.)

A relationship is where the magic happens, usually a process with work being done (therefore interface inputs must account for this balance.)

Vectors. Vectors I am thinking are the real intellectual and functional mechanisms. Most systems process inputs of potential (“energy”) control signal (“information”) and assets (other actors for nested systems). Processes do the work of adding vector solutions [for some other problem] for whatever the output is.

That’s the topology as I am seeing it.