rasbt/LLMs-from-scratch🔥 active
▲ +435 ★ this week
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Language modeling with linear-cost context
$ git clone https://github.com/m-a-n-i-f-e-s-t/retention.git▲ +435 ★ this week
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
▲ +171 ★ this week
🐙 Guides, papers, lessons, notebooks and resources for prompt engineering, context engineering, RAG, and AI Agents.
Code and documentation to train Stanford's Alpaca models, and generate the data.
▲ +18 ★ this week
High-performance In-browser LLM Inference Engine
▲ +2 ★ this week
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
▲ +18 ★ this week
An open source implementation of CLIP.
Data from GitHub · snapshot Nov 13, 2026