natasha/natasha
Solves basic Russian NLP tasks, API for lower level Natasha projects
Rule-based token, sentence segmentation for Russian language
$ git clone https://github.com/natasha/razdel.gitSolves basic Russian NLP tasks, API for lower level Natasha projects
▲ +2 ★ this week
Language model tokenization at GB/s
▲ +1 ★ this week
🧹 Python package for text cleaning
▲ +1 ★ this week
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
▲ +1 ★ this week
PHP Text Analysis is a library for performing Information Retrieval (IR) and Natural Language Processing (NLP) tasks using the PHP language
▲ +1 ★ this week
EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING
Data from GitHub · snapshot Nov 16, 2026