Tokenization & Preprocessing
polm/fugashi
▲ +1 ★ this week
A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.
🧠 Natural Language Processing
C++MITupdated Oct 24, 2025
aim to use JapaneseTokenizer as easy as possible
$ git clone https://github.com/Kensuke-Mitsuzawa/JapaneseTokenizers.git▲ +1 ★ this week
A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.
Juman++ (a Morphological Analyzer Toolkit)
▲ +2 ★ this week
Language model tokenization at GB/s
Solves basic Russian NLP tasks, API for lower level Natasha projects
한국어 자연어처리를 위한 파이썬 라이브러리입니다. 단어 추출/ 토크나이저 / 품사판별/ 전처리의 기능을 제공합니다.
▲ +1 ★ this week
Self-contained Japanese Morphological Analyzer written in pure Go
Data from GitHub · snapshot Nov 16, 2026