AIdventure.

Speeding up Attention Layers

Multi-head, Multi-Query & Grouped-Query Attention layers clearly explained. How cache works in the Attention layers

TF-IDF & BM25 Explained

TF-IDF and BM25 are two of the most used algorithms in Information Retrieval. In this post we will explain how they work

LoRA - Low-Rank Adaptation of Large Language Models

Fine-tuning large language models via trainable rank decomposition matrices

CLIP - Contrastive Language-Image Pre-training

Contrastive learning for unified vision-language representations in a shared embedding space

DistillBERT - A distilled version of BERT

Knowledge distillation compresses BERT: smaller, faster, with almost all performance

RoBERTa - Optimized BERT Pretraining Approach

Unlocking the true potential of BERT through rigorous optimization and strategic training choices

BERT - Bidirectional Encoder Representations from Transformers

Pre-training bidirectional by jointly conditioning on both left and right context

GPT1 - Improving Language Understanding by Pre-Training

Semi-supervised learning through generative pre-training on unlabeled text and task-specific fine-tuning

Transformer - Attention Is All You Need

Demystifying the Transformer architecture, explaining the Encoder, Decoder, and Attention mechanisms block by block with PyTorch implementation