AIdventure.

Speeding up Attention Layers

Multi-head, Multi-Query & Grouped-Query Attention layers clearly explained. How cache works in the Attention layers

SENet - Squeeze and Excitation Networks

Introducing channel attention to improve the performance of image classification tasks

Transformer - Attention Is All You Need

Demystifying the Transformer architecture, explaining the Encoder, Decoder, and Attention mechanisms block by block with PyTorch implementation