zlacker

[parent] [thread] 4 comments
1. api+(OP)[view] [source] 2023-10-24 14:05:54
I'm excited to see attention layers animated like this. I feel like I'm this close to grasping them.
replies(3): >>imjons+e2 >>Greenp+J4 >>Animat+t8o
◧
2. imjons+e2[view] [source] 2023-10-24 14:17:53
>>api+(OP)
You mean you would be excited to see attention animations? The page presents convolutions not attention.
◧
3. Greenp+J4[view] [source] 2023-10-24 14:28:25
>>api+(OP)
I still haven't found "that one visualisation" that makes the attention concept in Transformers as easily understood as these CNNs.

If someone here on HN has a link to a page that has helped them get to the Eureka-point of fully grasping attention layers, feel free to share!

replies(1): >>julian+Rb
◧◩
4. julian+Rb[view] [source] [discussion] 2023-10-24 14:57:10
>>Greenp+J4
I found this video helpful for understanding transformers in general, but it covers attention too: https://www.youtube.com/watch?v=kWLed8o5M2Y

The short version (as I understand it) is that you use a neural network to weight pairs of inputs by their importance to each other. That lets you get rid of unimportant information while keeping what actually is important.

◧
5. Animat+t8o[view] [source] 2023-10-31 22:10:54
>>api+(OP)
Hi! I'm the creator of the site. Good news: I'm currently working on animations and an explainer video on transformers and self-attention. The best way to be notified is probably to subscribe to my YouTube channel and hit the bell icon for notifications.
[go to top]