An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
$ git clone https://github.com/EleutherAI/gpt-neox.gitRWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Chinese-LLaMA 1&2、Chinese-Falcon 基础模型；ChatFlow中文对话模型；中文OpenLLaMA模型；NLP预训练/指令微调数据集
A curriculum for learning about foundation models, from scratch to the frontier
Data from GitHub · snapshot Sep 24, 2026