Tokenization & Preprocessing
jfilter/clean-text
🧹 Python package for text cleaning
🧠 Natural Language Processing
PythonOtherupdated May 15, 2026
🏆 #2,594 overall#104 of 148 in Tokenization & Preprocessing
lyeoni /prenlp
Preprocessing Library for Natural Language Processing
$ git clone https://github.com/lyeoni/prenlp.gitStars
164
164
Forks
12
12
Language
Python
License
Apache-2.0
Created
Nov 12, 2019
6.9 years old
Last push
Dec 6, 2022
More in Tokenization & Preprocessing
Tokenization & Preprocessing
jbesomi/texthero
Text preprocessing, representation and visualization from zero to hero.
🧬 Embeddings🧠 Natural Language Processing💎 Text Mining
PythonMITupdated Aug 29, 2023
Tokenization & Preprocessing
Tokenization & Preprocessing
explosion/spacy-streamlit
👑 spaCy building blocks and visualizers for Streamlit apps
🏷️ Named Entity Recognition🗂️ Text Classification🧠 Natural Language Processing
PythonMITupdated Jul 29, 2024
Tokenization & Preprocessing
cbaziotis/ekphrasis
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
🧠 Natural Language Processing
PythonMITupdated Jun 2, 2025
Tokenization & Preprocessing
open-korean-text/open-korean-text
Open Korean Text Processor - An Open-source Korean Text Processor
🧠 Natural Language Processing
ScalaApache-2.0updated Mar 12, 2024
Data from GitHub · snapshot Sep 24, 2026