Tokenization & Preprocessing
roshan-research/hazm
▲ +1 ★ this week
Persian NLP Toolkit
🧠 Natural Language Processing💎 Text Mining
PythonMITupdated Apr 1, 2026
🏆 #2,993 overall#66 of 178 in Tokenization & Preprocessing
michmech /lemmatization-lists
Machine-readable lists of lemma-token pairs in 23 languages.
$ git clone https://github.com/michmech/lemmatization-lists.gitStars
371
371
This week
—
Forks
101
101
Language
Other
License
ODbL-1.0
Created
May 11, 2018
8.5 years old
Last push
Jan 29, 2022
GitHub topics
More in Tokenization & Preprocessing
Tokenization & Preprocessing
bjascob/LemmInflect
A python module for English lemmatization and inflection.
🧠 Natural Language Processing
PythonMITupdated Sep 14, 2023
Tokenization & Preprocessing
anujvyas/Natural-Language-Processing-Projects
This repository consists of all my NLP Projects
🧠 Natural Language Processing💖 Sentiment Analysis🗂️ Text Classification
Jupyter Notebookno licenseupdated Nov 10, 2020
Tokenization & Preprocessing
janlukasschroeder/nlp-cheat-sheet-python
NLP Cheat Sheet, Python, spacy, LexNPL, NLTK, tokenization, stemming, sentence detection, named entity recognition
🧠 Natural Language Processing🏷️ Named Entity Recognition
Jupyter Notebookno licenseupdated Feb 11, 2023
Tokenization & Preprocessing
bnosac/udpipe
R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit
🧠 Natural Language Processing💎 Text Mining
C++MPL-2.0updated Mar 2, 2026
Tokenization & Preprocessing
adbar/simplemma🔥 active
Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
🧠 Natural Language Processing
PythonMITupdated Nov 6, 2026
Data from GitHub · snapshot Nov 13, 2026