Transformer Architecture Introduced

Researchers at Google publish 'Attention Is All You Need,' introducing the Transformer architecture. The paper proposes an approach based entirely on attention mechanisms, dispensing with recurrence and convolutions. The Transformer becomes the foundation for subsequent language models including BERT, GPT, and their successors.