The many amazing things about Self-Attention and why they work

Self-Attention is the heart of Transformer models, which are one of the most important innovations in Deep Learning in the past decade. In this video I dive deep into the mechanics of Self Attention - what it does, how it does it, and why it does what it does. This is Part 2 of my "Attention to Transformers" series (yes, that's what I am calling it). My first video covered the basics of Attention, describing it from first principles, here is the link for that if you want to see it: • Neural Attention - Thi...
To support the channel and access the Word documents/slides/animations used in this video, consider JOINING the channel on KZhead or Patreon. Members get access to Code, project files, scripts, slides, animations, and illustrations for most of the videos on my channel! Learn more about perks below.
Join and support the channel - www.youtube.com/@avb_fj/join
Patreon - / neuralbreakdownwithavb
Follow on Twitter: @neural_avb
More interesting stuff related to Attention/Transformers:
- The History of NLP in 50 concepts and 17 minutes - • 10 years of NLP histor...
- Segment Anything Model - • Explaining the Segment...
- State of Multimodal AI : • Multimodal AI from Fir...
Timestamps:
0:00 - Intro
1:45 - What is Self Attention
5:26 - How does Self Attention Work
7:08 - Why is it so powerful?
9:25 - Masked Attention
11:40 - Transformers
#deeplearning #ai #machinelearning #nlp