Tokenization & Preprocessing
daac-tools/vibrato
▲ +2 ★ this week
🎤 vibrato: Viterbi-based accelerated tokenizer
🧠 Natural Language Processing
RustApache-2.0updated Oct 25, 2026
Japanese Morphological Analyzer written in Rust
$ git clone https://github.com/togatoga/kanpyo.git▲ +2 ★ this week
🎤 vibrato: Viterbi-based accelerated tokenizer
▲ +2 ★ this week
🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer
▲ +1 ★ this week
Self-contained Japanese Morphological Analyzer written in pure Go
▲ +3 ★ this week
A multilingual morphological analysis library.
▲ +1 ★ this week
A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.
▲ +1 ★ this week
A Japanese tokenizer based on recurrent neural networks
Data from GitHub · snapshot Nov 16, 2026