zlacker

>>rafael+(OP)
Low level numerical operation optimizations are often not reproduceable. For example: https://www.intel.com/content/dam/develop/external/us/en/doc... (2013)

But it's still surprising that that LLM doesn't work on iPhone 16 at all. After all LLMs are known for their tolerance to quantization.

>>rainco+8h
Yes, "floating point accumulation doesn't commute" is a mantra everyone should have in their head, and when I first read this article, I was jumping at the bit to dismiss it out of hand for that reason.

But, what got me about this is that:

* every other Apple device delivered the same results

* Apple's own LLM silently failed on this device

to me that behavior suggests an unexpected failure rather than a fundamental issue; it seems Bad (TM) that Apple would ship devices where their own LLM didn't work.

>>bri3d+Dh
> floating point accumulation doesn't commute

It is commutative (except for NaN). It isn't associative though.

>>sva_+Ct
I think it commutes even when one or both inputs are NaN? The output is always NaN.

>>ekelse+eD
NaNs are distinguishable. /Which/ NaN you get doesn't commute.

>>addaon+ND
I guess at the bit level, but not at the level of computation? Anything that relies on bit patterns of nans behaving in a certain way (like how they propagate) is in dangerous territory.

>>ekelse+nG
> Anything that relies on bit patterns of nans behaving in a certain way (like how they propagate) is in dangerous territory.

Why? This is well specified by IEEE 754. Many runtimes (e.g. for Javascript) use NaN boxing. Treating floats as a semi-arbitrary selection of rational numbers plus a handful of special values is /more/ correct than treating them as real numbers, but treating them as actually specified does give more flexibility and power.

>>addaon+OJ
Can you show me where in the ieee spec this is guaranteed?

My understanding is the exact opposite - that it allows implementations to return any NaN value at all. It need not be any that were inputs.

It may be that JavaScript relies on it and that has become more binding than the actual spec, but I don't think the spec actually guarantees this.

Edit: actually it turns out nan-boxing does not involve arithmetic, which is why it works. I think my original point stands, if you are doing something that relies on how bit values of NaNs are propagated during arithmetic, you are on shaky ground.

>>ekelse+hP
See 6.2.3 in the 2019 standard.

> 6.2.3 NaN propagation

> An operation that propagates a NaN operand to its result and has a single NaN as an input should produce a NaN with the payload of the input NaN if representable in the destination format.

> If two or more inputs are NaN, then the payload of the resulting NaN should be identical to the payload of one of the input NaNs if representable in the destination format. This standard does not specify which of the input NaNs will provide the payload.