🏆 #9 overall#3 of 1,378 in Language Models

rasbt /LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

$ git clone https://github.com/rasbt/LLMs-from-scratch.git
GitHub social preview for rasbt/LLMs-from-scratch
Stars
108.5K
108,506
This week
+499 ★
+1.8K this month
Forks
16.8K
16,791
Language
Jupyter Notebook
License
Other
Created
Jul 23, 2023
3.3 years old
Last push
Nov 9, 2026

Stars, last 30 days

Oct 15, 2026 → Nov 16, 2026
★ 106.6K▲ +1.9K★ 108.5K

Categories

GitHub topics

More in Language Models

Language Models

jshuadvd/LongRoPE

Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper

🧠 Natural Language Processing✂️ Tokenization & Preprocessing
Pythonno licenseupdated Jul 20, 2024
GitHub ↗★ 154⑂ 13
Language Models

raiyanyahya/how-to-train-your-gpt

▲ +6 ★ this week

Build a modern LLM from scratch. Every line commented. Explained like we are five.

🧠 Natural Language Processing
Jupyter NotebookMITupdated Oct 26, 2026
GitHub ↗★ 4.2K⑂ 505
Language Models

analyticalrohit/llms-from-scratch

▲ +1 ★ this week

Build a ChatGPT like LLM from scratch in PyTorch, explained step by step.

✂️ Tokenization & Preprocessing
Jupyter NotebookMITupdated Jun 8, 2026
GitHub ↗★ 387⑂ 90
Language Models

theaniketgiri/create-llm

▲ +1 ★ this week

The fastest way to build and start training your own LLM. CLI tool that scaffolds production-ready PyTorch training projects in seconds. Like create-next-app but for language models.

🧠 Natural Language Processing
TypeScriptMITupdated Sep 13, 2026
GitHub ↗★ 373⑂ 49
Language Models

BlinkDL/RWKV-LM

▲ +5 ★ this week

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

PythonApache-2.0updated Nov 4, 2026
GitHub ↗★ 14.8K⑂ 1K

Data from GitHub · snapshot Nov 16, 2026