AIdventure.

Speeding up Attention Layers

Multi-head, Multi-Query & Grouped-Query Attention layers clearly explained. How cache works in the Attention layers

LoRA - Low-Rank Adaptation of Large Language Models

Fine-tuning large language models via trainable rank decomposition matrices

DistillBERT - A distilled version of BERT

Knowledge distillation compresses BERT: smaller, faster, with almost all performance

MobileNet v1 - Efficient Convolutional Neural Networks

Efficient convolutional neural networks for mobile vision applications

Depthwise Separable Convolutions

Same results as standard convolutions with only a fraction of the computational cost. Explore the tricks behind MobileNet and efficient CNNs