๐Ÿ† #1,946 overall#38 of 178 in Tokenization & Preprocessing

mathewsanders /Mustard

๐ŸŒญ Mustard is a Swift library for tokenizing strings when splitting by whitespace doesn't cut it.

$ git clone https://github.com/mathewsanders/Mustard.git
GitHub social preview for mathewsanders/Mustard
Stars
683
683
This week
0 โ˜…
0 this month
Forks
18
18
Language
Swift
License
MIT
Created
Dec 30, 2016
9.9 years old
Last push
May 14, 2018

Stars, last 30 days

Oct 15, 2026 โ†’ Nov 16, 2026
โ˜… 683โ–ฒ +0โ˜… 683

Categories

GitHub topics

More in Tokenization & Preprocessing

Tokenization & Preprocessing

theseer/tokenizer

โ–ฒ +2 โ˜… this week

A small library for converting tokenized PHP source code into XML (and potentially other formats)

PHPOtherupdated Feb 3, 2026
GitHub โ†—โ˜… 5.2Kโ‘‚ 25
Tokenization & Preprocessing

marcelroed/gigatoken

โ–ฒ +2 โ˜… this week

Language model tokenization at GB/s

๐Ÿง  Natural Language Processing
RustMITupdated Nov 2, 2026
GitHub โ†—โ˜… 4.2Kโ‘‚ 225
Tokenization & Preprocessing

andialbrecht/sqlparse

A non-validating SQL parser module for Python

PythonBSD-3-Clauseupdated Nov 1, 2026
GitHub โ†—โ˜… 4Kโ‘‚ 781
Tokenization & Preprocessing

Chevrotain/chevrotain๐Ÿ”ฅ active

โ–ฒ +2 โ˜… this week

Parser Building Toolkit for JavaScript

TypeScriptApache-2.0updated Nov 14, 2026
GitHub โ†—โ˜… 2.8Kโ‘‚ 222
Tokenization & Preprocessing

dqbd/tiktokenizer

Online playground for OpenAPI tokenizers

TypeScriptMITupdated Apr 24, 2025
GitHub โ†—โ˜… 1.7Kโ‘‚ 188
Tokenization & Preprocessing

natasha/natasha

Solves basic Russian NLP tasks, API for lower level Natasha projects

๐Ÿท๏ธ Named Entity Recognition๐Ÿง  Natural Language Processing
PythonMITupdated Oct 14, 2026
GitHub โ†—โ˜… 1.4Kโ‘‚ 122

Data from GitHub ยท snapshot Nov 16, 2026