BERT - Bidirectional Encoder Representations from Transformers October 11, 2018 • 5 min read Pre-training bidirectional by jointly conditioning on both left and right context