transformers-explained
Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.
What is transformers-explained?
Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.
Overview of transformers-explained
What it is
Transformers Explained is a step-by-step educational guide that covers the full Transformer architecture, every attention variant, positional embeddings, and every layer inside a Transformer. It is maintained by Amit Shekhar, founder of Outcome School, and is available as an open-source repository on GitHub.
Key features
- Covers the complete Transformer architecture from tokenization to output - Explains all attention variants: self-attention, multi-head attention, cross attention, causal masking, grouped-query attention, and flash attention - Includes detailed math behind attention (Q, K, V), scaling factor √dₖ, and Rotary Position Embedding (RoPE)
- Describes feed-forward networks, residual connections, and layer normalization - Provides numeric examples and step-by-step walkthroughs - Organized into parts: Architecture, Attention, Position, and Other Layers
Use cases
- Learning how Transformers work from scratch - Understanding the internal components of modern Large Language Models (LLMs)
- Studying attention mechanisms and positional embeddings for research or education - Preparing for interviews or courses on deep learning and NLP
Getting started
See the repository README for links to each blog post and video. No installation is required.
License and pricing
Licensed under Apache-2.0. The project is open source and free to use.
Based on the project's public README and repository metadata.