Tokenization & Preprocessing
🏆 #1,396 overall#33 of 148 in Tokenization & Preprocessing
jflex-de /jflex
The fast scanner generator for Java™ with full Unicode support
$ git clone https://github.com/jflex-de/jflex.gitStars
634
634
Forks
118
118
Language
Java
License
Other
Created
Feb 15, 2015
11.6 years old
Last push
Jan 1, 2025
Categories
More in Tokenization & Preprocessing
Tokenization & Preprocessing
Chevrotain/chevrotain🔥 active
Parser Building Toolkit for JavaScript
TypeScriptApache-2.0updated Sep 24, 2026
Tokenization & Preprocessing
no-context/moo
Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.
JavaScriptBSD-3-Clauseupdated Feb 27, 2026
Tokenization & Preprocessing
risesoft-y9/Data-Labeling
数据标注是一款专门对文本数据进行处理和标注的工具，通过简化快捷的文本标注流程和动态的算法反馈，支持用户快速标注关键词并能通过算法持续减少人工标注的成本和时间。数据标注的过程先由人工标注构建基础，再由自动标注反哺人工标注，最后由人工标注进行纠偏，从而大幅度提高标注的精准度和高效性。数据标注需要依赖开源的数字底座进行人员岗位管控。
JavaGPL-3.0updated Jun 23, 2025
Tokenization & Preprocessing
glayzzle/php-parser
:herb: NodeJS PHP Parser - extract AST or tokens
JavaScriptBSD-3-Clauseupdated Sep 10, 2026
Tokenization & Preprocessing
Data from GitHub · snapshot Sep 24, 2026