| Paper Replications | ML/DL Research Paper Implementations |
Date:
Overview
This repository contains a collection of code implementations and experiments replicating results from a wide range of influential machine learning and deep learning research papers. Each subfolder corresponds to a specific paper, model, or technique, with code, notes, and sometimes pretrained weights or results.
Deployed Models
Visit SmolHub to view the deployed models and interactive demos.
Repository Structure
Language Models & Transformers
- BERT/: Bidirectional Encoder Representations from Transformers
- GPT/: Generative Pretrained Transformer models
- Llama/, Llama4/: Meta’s Llama model replications and experiments
- Gemma/, Gemma3/: Google’s Gemma models
- Mixtral/: Mixture-of-Experts Transformer models
- DeepSeekV3/: DeepSeek model replications and experiments
- Kimi-K2/: Kimi-K2 model replications and training scripts
- Moonshine/: Moonshine model experiments
- Transformer/: Original Transformer model and variants
- Differential Transformer/: Novel differential attention architectures
Vision & Multimodal Models
- ViT/: Vision Transformer models
- CLiP/: CLIP vision-language models
- SigLip/: Sigmoid Loss for Language-Image Pretraining
- Llava/: Large Language and Vision Assistant models
- PaliGemma/: PaliGemma multimodal model replications
Generative Adversarial Networks
- DCGANs/: Deep Convolutional GANs
- WGANs/: Wasserstein GANs
- CGANs/: Conditional Generative Adversarial Networks
- CycleGANs/: Cycle-consistent GANs for image translation
- Pix2Pix/: Image-to-image translation with conditional GANs
Recurrent & Sequence Models
- RNNs/: Recurrent Neural Networks
- LSTM/: Long Short-Term Memory models
- GRU/: Gated Recurrent Unit models
- Seq2Seq/: Sequence-to-sequence models
- Encoder-Decoder/: Encoder-decoder architectures
RLHF & Fine-Tuning Techniques
- DPO/: Direct Preference Optimization
- ORPO/: Online RLHF Preference Optimization
- SimplePO/: Simple Preference Optimization
- LoRA/: Low-Rank Adaptation for efficient fine-tuning
- Fine Tuning using PEFT/: Parameter-Efficient Fine-Tuning methods
Audio & Speech Models
- Whisper/: OpenAI’s Whisper speech recognition model
- TTS/: Text-to-Speech models
- CLAP/: Contrastive Language-Audio Pretraining
Other Architectures
- Attention Mechanisms/: Various attention mechanisms and patterns
- VAE/: Variational Autoencoders
- DDP/: Distributed Data Parallel training experiments
Usage
Each folder is self-contained and includes:
- Implementation code in PyTorch
- Training and inference scripts
- Configuration files
- Documentation and notes
- Sometimes pretrained weights or results
Please refer to the README or notes within each subfolder for specific instructions on running the code.
Key Features
- 30+ Model Implementations: From classic RNNs to modern LLMs and diffusion models
- Production-Ready Code: Clean, documented, and tested implementations
- Training Scripts: End-to-end training pipelines with distributed training support
- Deployed Demos: Interactive demos available at smolhub.com
- Research Citations: Proper attribution and links to original papers
Contributing
Feel free to open issues or pull requests if you have suggestions, improvements, or additional replications to add!
Sponsors
Thank you to everyone for their support and love for this project!
Links
Author
Citation
If you find this repository useful in your research, please cite it:
@misc{singh_paper_replications_2025,
author = {Yuvraj Singh},
title = {Paper-Replications: Replication from Scratch Repository using PyTorch},
year = {2025},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/YuvrajSingh-mist/Paper-Replications}},
commit = {1d7a1b37a82e441cde884f591c9c41fa4e47ddbb}
}