Tokenization & Preprocessing
Chevrotain/chevrotain🔥 active
▲ +2 ★ this week
Parser Building Toolkit for JavaScript
TypeScriptApache-2.0updated Nov 14, 2026
Contains source-code for viewers following along with my Beginners Guide To Building Interpreters series on my Youtube Channel.
$ git clone https://github.com/tlaceby/guide-to-interpreters-series.git▲ +2 ★ this week
Parser Building Toolkit for JavaScript
▲ +1 ★ this week
NodeJS PHP Parser - extract AST or tokens
▲ +2 ★ this week
Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.
Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.
VSCode extension to highlight nested code blocks
Data from GitHub · snapshot Nov 16, 2026