🏆 #1,444 overall#320 of 1,378 in Language Models

Liuhong99 /Sophia

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”

$ git clone https://github.com/Liuhong99/Sophia.git
GitHub social preview for Liuhong99/Sophia
Stars
1K
1,002
This week
0 ★
Forks
56
56
Language
Python
License
MIT
Created
May 23, 2023
3.5 years old
Last push
Jan 30, 2024

Stars, last 17 days

Oct 28, 2026 → Nov 16, 2026
★ 1K▲ +2★ 1K

Categories

GitHub topics

More in Language Models

Language Models

zyushun/Adam-mini

Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793

Pythonno licenseupdated May 13, 2025
GitHub ↗★ 462⑂ 22
Language Models

yangluo7/CAME

[ACL 2023] The official implementation of "CAME: Confidence-guided Adaptive Memory Optimization"

PythonMITupdated Mar 22, 2025
GitHub ↗★ 100⑂ 9
Language Models

ray-project/ray🔥 active

▲ +63 ★ this week

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

PythonApache-2.0updated Nov 16, 2026
GitHub ↗★ 44.3K⑂ 8.3K
Language Models

camel-ai/camel🔥 active

▲ +21 ★ this week

🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org

🧠 Natural Language Processing
PythonApache-2.0updated Nov 16, 2026
GitHub ↗★ 18K⑂ 2.1K
Language Models

NVIDIA/DeepLearningExamples

▲ +9 ★ this week

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

🎙️ Speech Recognition🧠 Natural Language Processing
Jupyter Notebookno licenseupdated Aug 12, 2024
GitHub ↗★ 14.9K⑂ 3.4K

Data from GitHub · snapshot Nov 16, 2026