🏆 #233 overall#58 of 1,378 in Language Models

Tiiny-AI /PowerInfer

High-speed Large Language Model Serving for Local Deployment

$ git clone https://github.com/Tiiny-AI/PowerInfer.git
GitHub social preview for Tiiny-AI/PowerInfer
Stars
9.9K
9,890
This week
+16 ★
+56 this month
Forks
620
620
Language
C++
License
MIT
Created
Dec 15, 2023
2.9 years old
Last push
May 11, 2026

Stars, last 30 days

Oct 15, 2026 → Nov 16, 2026
★ 9.8K▲ +61★ 9.9K

Categories

GitHub topics

More in Language Models

Language Models

AI-Hypercomputer/JetStream

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

PythonApache-2.0updated Jan 5, 2026
GitHub ↗★ 474⑂ 70
Language Models

JayZhang42/SLED

SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Model https://arxiv.org/pdf/2411.02433

Pythonno licenseupdated Dec 5, 2024
GitHub ↗★ 123⑂ 20
Language Models

ray-project/ray🔥 active

▲ +63 ★ this week

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

PythonApache-2.0updated Nov 16, 2026
GitHub ↗★ 44.3K⑂ 8.3K
Language Models

ymcui/Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

🧠 Natural Language Processing
PythonApache-2.0updated Apr 19, 2026
GitHub ↗★ 18.9K⑂ 1.8K
Language Models

Lightning-AI/litgpt🔥 active

▲ +4 ★ this week

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

PythonApache-2.0updated Nov 10, 2026
GitHub ↗★ 13.8K⑂ 1.5K
Language Models

ymcui/Chinese-LLaMA-Alpaca-2

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

🧠 Natural Language Processing
PythonApache-2.0updated Apr 19, 2026
GitHub ↗★ 7.1K⑂ 557

Data from GitHub · snapshot Nov 16, 2026