LoRA - Low-Rank Adaptation of Large Language Models
June 17, 2021 • 5 min read
Fine-tuning large language models via trainable rank decomposition matrices
DINO - Emerging Properties in Self-Supervised Vision Transformers
May 24, 2021 • 9 min read
Unsupervised visual feature learning using knowledge distillation and transformers
CLIP - Contrastive Language-Image Pre-training
February 26, 2021 • 9 min read
Contrastive learning for unified vision-language representations in a shared embedding space
Vision Transformer - An Image is Worth 16x16 Words
October 22, 2020 • 6 min read
Google shows how treating image patches as tokens can revolutionize computer vision
DistillBERT - A distilled version of BERT
October 2, 2019 • 4 min read
Knowledge distillation compresses BERT: smaller, faster, with almost all performance
RoBERTa - Optimized BERT Pretraining Approach
July 26, 2019 • 5 min read
Unlocking the true potential of BERT through rigorous optimization and strategic training choices
BERT - Bidirectional Encoder Representations from Transformers
October 11, 2018 • 5 min read
Pre-training bidirectional by jointly conditioning on both left and right context
GPT1 - Improving Language Understanding by Pre-Training
June 11, 2018 • 4 min read
Semi-supervised learning through generative pre-training on unlabeled text and task-specific fine-tuning
SENet - Squeeze and Excitation Networks
September 17, 2017 • 6 min read
Introducing channel attention to improve the performance of image classification tasks
Transformer - Attention Is All You Need
June 12, 2017 • 19 min read
Demystifying the Transformer architecture, explaining the Encoder, Decoder, and Attention mechanisms block by block with PyTorch implementation
MobileNet v1 - Efficient Convolutional Neural Networks
April 17, 2017 • 6 min read
Efficient convolutional neural networks for mobile vision applications
DenseNet - Densely Connected Convolutional Networks
August 25, 2016 • 6 min read
Connecting each layer to every other layer to maximize information flow and efficiency
