transformers-explained

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

What is transformers-explained?

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

Overview of transformers-explained

What it is

Transformers Explained is a step-by-step educational guide that covers the full Transformer architecture, every attention variant, positional embeddings, and every layer inside a Transformer. It is maintained by Amit Shekhar, founder of Outcome School, and is available as an open-source repository on GitHub.

Key features

  • Covers the complete Transformer architecture from tokenization to output - Explains all attention variants: self-attention, multi-head attention, cross attention, causal masking, grouped-query attention, and flash attention - Includes detailed math behind attention (Q, K, V), scaling factor √dₖ, and Rotary Position Embedding (RoPE)
  • Describes feed-forward networks, residual connections, and layer normalization - Provides numeric examples and step-by-step walkthroughs - Organized into parts: Architecture, Attention, Position, and Other Layers

Use cases

  • Learning how Transformers work from scratch - Understanding the internal components of modern Large Language Models (LLMs)
  • Studying attention mechanisms and positional embeddings for research or education - Preparing for interviews or courses on deep learning and NLP

Getting started

See the repository README for links to each blog post and video. No installation is required.

License and pricing

Licensed under Apache-2.0. The project is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/amitshekhariitbhutransformers-explained.svg)](https://yunodb.com/tools/amitshekhariitbhutransformers-explained?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/amitshekhariitbhutransformers-explained?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/amitshekhariitbhutransformers-explained.svg" alt="Featured on YunoDB" height="20"></a>