AIdventure.

RoBERTa - Optimized BERT Pretraining Approach

Unlocking the true potential of BERT through rigorous optimization and strategic training choices

BERT - Bidirectional Encoder Representations from Transformers

Pre-training bidirectional by jointly conditioning on both left and right context

GPT1 - Improving Language Understanding by Pre-Training

Semi-supervised learning through generative pre-training on unlabeled text and task-specific fine-tuning