Natural Language Processing
🏆 #1,785 overall#621 of 2,559 in Natural Language Processing
ChenghaoMou /text-dedup
All-in-one text de-duplication
$ git clone https://github.com/ChenghaoMou/text-dedup.gitStars
765
765
This week
—
Forks
82
82
Language
Python
License
Apache-2.0
Created
Mar 13, 2021
5.7 years old
Last push
Mar 9, 2026
GitHub topics
More in Natural Language Processing
Natural Language Processing
cbaziotis/ekphrasis
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
✂️ Tokenization & Preprocessing💎 Text Mining
PythonMITupdated Jun 2, 2025
Natural Language Processing
haven-jeon/PyKoSpacing
Automatic Korean word spacing with Python
💎 Text Mining
PythonGPL-3.0updated Nov 5, 2026
Natural Language Processing
RandyPen/TextCluster
短文本聚类预处理模块 Short text cluster
💎 Text Mining
PythonBSD-3-Clauseupdated Dec 28, 2019
Natural Language Processing
himkt/konoha🔥 active
🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.
💎 Text Mining
PythonMITupdated Nov 13, 2026
Natural Language Processing
pemagrg1/Natural-Language-Processing-NLP-Roadmap
A simple RoadMap to Natural Language Processing(NLP)
💎 Text Mining🗂️ Text Classification
Otherno licenseupdated Sep 10, 2025
Data from GitHub · snapshot Nov 13, 2026