Tokenization & Preprocessing
🏆 #6,366 overall#162 of 178 in Tokenization & Preprocessing
ropensci /hunspell
High-Performance Stemmer, Tokenizer, and Spell Checker for R
$ git clone https://github.com/ropensci/hunspell.gitStars
116
116
This week
0 ★
0 this month
Forks
47
47
Language
C++
License
Other
Created
Mar 9, 2016
10.7 years old
Last push
Feb 12, 2026
Stars, last 30 days
Oct 15, 2026 → Nov 16, 2026Categories
More in Tokenization & Preprocessing
Tokenization & Preprocessing
ropensci/tokenizers
Fast, Consistent Tokenization of Natural Language Text
🧠 Natural Language Processing💎 Text Mining
ROtherupdated Mar 27, 2024
Tokenization & Preprocessing
hunspell/hunspell🔥 active
▲ +1 ★ this week
The most popular spellchecking library.
🧠 Natural Language Processing
C++LGPL-2.1updated Nov 15, 2026
Tokenization & Preprocessing
bnosac/udpipe
R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit
🧠 Natural Language Processing💎 Text Mining
C++MPL-2.0updated Mar 2, 2026
Tokenization & Preprocessing
theseer/tokenizer
▲ +2 ★ this week
A small library for converting tokenized PHP source code into XML (and potentially other formats)
PHPOtherupdated Feb 3, 2026
Tokenization & Preprocessing
marcelroed/gigatoken
▲ +2 ★ this week
Language model tokenization at GB/s
🧠 Natural Language Processing
RustMITupdated Nov 2, 2026
Data from GitHub · snapshot Nov 16, 2026