Tokenization & Preprocessing
lyeoni/prenlp
Preprocessing Library for Natural Language Processing
🧠 Natural Language Processing
PythonApache-2.0updated Dec 6, 2022
🏆 #1,011 overall#18 of 148 in Tokenization & Preprocessing
jfilter /clean-text
🧹 Python package for text cleaning
$ git clone https://github.com/jfilter/clean-text.gitStars
1K
1,031
Forks
82
82
Language
Python
License
Other
Created
Dec 6, 2018
7.8 years old
Last push
May 15, 2026
More in Tokenization & Preprocessing
Tokenization & Preprocessing
jbesomi/texthero
Text preprocessing, representation and visualization from zero to hero.
🧬 Embeddings🧠 Natural Language Processing💎 Text Mining
PythonMITupdated Aug 29, 2023
Tokenization & Preprocessing
Tokenization & Preprocessing
natasha/natasha
Solves basic Russian NLP tasks, API for lower level Natasha projects
🏷️ Named Entity Recognition🧠 Natural Language Processing
PythonMITupdated Apr 13, 2026
Tokenization & Preprocessing
explosion/spacy-streamlit
👑 spaCy building blocks and visualizers for Streamlit apps
🏷️ Named Entity Recognition🗂️ Text Classification🧠 Natural Language Processing
PythonMITupdated Jul 29, 2024
Tokenization & Preprocessing
NLPOptimize/flash-tokenizer
EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING
🧠 Natural Language Processing
C++no licenseupdated Feb 2, 2026
Data from GitHub · snapshot Sep 24, 2026