zlacker

[parent] [thread] 2 comments
1. ghotli+(OP)[view] [source] 2023-11-21 00:02:44
Do you happen to have a link to where that interwoven layers bit is described? As far as I can tell it's not clear on the model cards.
replies(1): >>valine+ng
◧
2. valine+ng[view] [source] 2023-11-21 01:55:41
>>ghotli+(OP)
The model page is the only info I’ve found on it. As far as I can tell there’s no paper published on the technique.

In the “Merge Process” section they at least give the layer ranges.

https://huggingface.co/alpindale/goliath-120b

replies(1): >>ghotli+to
◧◩
3. ghotli+to[view] [source] [discussion] 2023-11-21 02:47:43
>>valine+ng
Ah, actually reviewing that more closely I found a link to it in the acknowledgements.

https://github.com/cg123/mergekit

[go to top]