🏆 #162 overall#37 of 1,378 in Language Models

BlinkDL /RWKV-LM

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

$ git clone https://github.com/BlinkDL/RWKV-LM.git
GitHub social preview for BlinkDL/RWKV-LM
Stars
14.8K
14,777
This week
+5 ★
+27 this month
Forks
1K
1,027
Language
Python
License
Apache-2.0
Created
Aug 8, 2021
5.3 years old
Last push
Nov 4, 2026

Stars, last 30 days

Oct 15, 2026 → Nov 16, 2026
★ 14.8K▲ +26★ 14.8K

Categories

GitHub topics

More in Language Models

Language Models

microsoft/LoRA

▲ +16 ★ this week

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

PythonMITupdated Dec 17, 2024
GitHub ↗★ 13.9K⑂ 936
Language Models

voidful/TextRL

▲ +2 ★ this week

Implementation of ChatGPT RLHF (Reinforcement Learning with Human Feedback) on any generation model in huggingface's transformer (blommz-176B/bloom/gpt/bart/T5/MetaICL)

🧠 Natural Language Processing
PythonMITupdated Apr 23, 2026
GitHub ↗★ 565⑂ 61
Language Models

rasbt/LLMs-from-scratch

▲ +499 ★ this week

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

✂️ Tokenization & Preprocessing🧠 Natural Language Processing
Jupyter NotebookOtherupdated Nov 9, 2026
GitHub ↗★ 108.5K⑂ 16.8K
Language Models

OptimalScale/LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

PythonApache-2.0updated Aug 10, 2026
GitHub ↗★ 8.5K⑂ 820
Language Models

raiyanyahya/how-to-train-your-gpt

▲ +6 ★ this week

Build a modern LLM from scratch. Every line commented. Explained like we are five.

🧠 Natural Language Processing
Jupyter NotebookMITupdated Oct 26, 2026
GitHub ↗★ 4.2K⑂ 505
Language Models

mallahyari/llm-hub

A curated collection of interesting applications, repos, and tutorials using large language models (LLM) like GPT-3

🧠 Natural Language Processing
OtherMITupdated Jun 7, 2023
GitHub ↗★ 153⑂ 14

Data from GitHub · snapshot Nov 16, 2026