🏆 #2,334 overall#49 of 178 in Tokenization & Preprocessing

polm /fugashi

A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.

$ git clone https://github.com/polm/fugashi.git
GitHub social preview for polm/fugashi
Stars
538
538
This week
+1 ★
+2 this month
Forks
40
40
Language
C++
License
MIT
Created
Oct 14, 2019
7.1 years old
Last push
Oct 24, 2025

Stars, last 30 days

Oct 15, 2026 → Nov 16, 2026
★ 536▲ +2★ 538

Categories

GitHub topics

More in Tokenization & Preprocessing

Tokenization & Preprocessing

daac-tools/vibrato

▲ +2 ★ this week

🎤 vibrato: Viterbi-based accelerated tokenizer

🧠 Natural Language Processing
RustApache-2.0updated Oct 25, 2026
GitHub ↗★ 430⑂ 27
Tokenization & Preprocessing

taishi-i/nagisa🔥 active

▲ +1 ★ this week

A Japanese tokenizer based on recurrent neural networks

🧠 Natural Language Processing
PythonMITupdated Nov 14, 2026
GitHub ↗★ 424⑂ 24
Tokenization & Preprocessing

ku-nlp/jumanpp

Juman++ (a Morphological Analyzer Toolkit)

🧠 Natural Language Processing
C++Apache-2.0updated Oct 13, 2026
GitHub ↗★ 413⑂ 47
Tokenization & Preprocessing

daac-tools/vaporetto

▲ +2 ★ this week

🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer

🧠 Natural Language Processing
RustApache-2.0updated Nov 3, 2026
GitHub ↗★ 302⑂ 12
Tokenization & Preprocessing

marcelroed/gigatoken

▲ +2 ★ this week

Language model tokenization at GB/s

🧠 Natural Language Processing
RustMITupdated Nov 2, 2026
GitHub ↗★ 4.2K⑂ 225

Data from GitHub · snapshot Nov 16, 2026