zlacker

[parent] [thread] 0 comments
1. hereon+(OP)[view] [source] 2023-09-12 20:32:22
It runs fantastically well on M2 Mac + llama.cpp, such a variety of factors in the Apple hardware making it possible. The ARM fp16 vector intrinsics, the Macbook's AMX co-processor, the unified memory architecture, etc.

It's more than fast enough for my experiments and the laptop doesn't seem to break a sweat.

[go to top]