Paper-Replications
From-scratch PyTorch replications of classic and SOTA AI/ML papers: transformers, attention, GANs, diffusion, CLIP, LoRA, MoE and more, with reproducible traini
What is Paper-Replications?
From-scratch PyTorch replications of classic and SOTA AI/ML papers: transformers, attention, GANs, diffusion, CLIP, LoRA, MoE and more, with reproducible training code.
Overview of Paper-Replications
What it is
This repository contains from-scratch PyTorch replications of classic and state-of-the-art AI/ML papers. It covers transformers, attention mechanisms, GANs, diffusion models, CLIP, LoRA, Mixture-of-Experts, and more. Each subfolder corresponds to a specific paper or technique, with code, notes, and sometimes pretrained weights or results. The project aims to provide reproducible training code for influential research.
Key features
- From-scratch PyTorch implementations of a wide range of AI/ML papers.
- Covers attention mechanisms, BERT, GANs (CGANs, CycleGANs, DCGANs, WGANs), CLIP, CLAP, diffusion, LoRA, MoE, and many more.
- Each subfolder is self-contained with code, scripts, and sometimes notebooks.
- Includes replications of modern models like DeepSeekV3, Gemma, Llama, Mixtral, ViT, Whisper, etc.
- Parameter-Efficient Fine-Tuning (PEFT) methods included.
- Distributed Data Parallel (DDP) training experiments.
Use cases
- Researchers and practitioners who want to understand and reproduce key AI/ML papers.
- Students learning deep learning by studying clean, from-scratch implementations.
- Developers looking for reference implementations of popular models and techniques.
- Experimentation with different architectures and training methods.
Getting started
Each folder is self-contained. Please refer to the README or notes within each subfolder for specific instructions on installation and usage. See the repository README for an overview.
License and pricing
- Code and notebooks: Apache License 2.0.
- Documentation, diagrams, generated results and published artifacts: CC BY 4.0.
- The project is open source and free to use.
Based on the project's public README and repository metadata.