๐Ÿ† #1,738 overall#52 of 148 in Tokenization & Preprocessing

timtadh /lexmachine

Lex machinary for go.

$ git clone https://github.com/timtadh/lexmachine.git
GitHub social preview for timtadh/lexmachine
Stars
411
411
Forks
28
28
Language
Go
License
Other
Created
Apr 28, 2014
12.4 years old
Last push
Jul 15, 2022

Categories

GitHub topics

More in Tokenization & Preprocessing

Tokenization & Preprocessing

Chevrotain/chevrotain๐Ÿ”ฅ active

Parser Building Toolkit for JavaScript

TypeScriptApache-2.0updated Sep 24, 2026
GitHub โ†—โ˜… 2.8Kโ‘‚ 222
Tokenization & Preprocessing

no-context/moo

Optimised tokenizer/lexer generator! ๐Ÿ„ Uses /y for performance. Moo.

JavaScriptBSD-3-Clauseupdated Feb 27, 2026
GitHub โ†—โ˜… 885โ‘‚ 75
Tokenization & Preprocessing

jflex-de/jflex

The fast scanner generator for Javaโ„ข with full Unicode support

JavaOtherupdated Jan 1, 2025
GitHub โ†—โ˜… 634โ‘‚ 118
Tokenization & Preprocessing

glayzzle/php-parser

:herb: NodeJS PHP Parser - extract AST or tokens

JavaScriptBSD-3-Clauseupdated Sep 10, 2026
GitHub โ†—โ˜… 565โ‘‚ 75
Tokenization & Preprocessing

tlaceby/guide-to-interpreters-series

Contains source-code for viewers following along with my Beginners Guide To Building Interpreters series on my Youtube Channel.

TypeScriptno licenseupdated May 2, 2024
GitHub โ†—โ˜… 286โ‘‚ 37
Tokenization & Preprocessing

daulet/tokenizers

Go, Wasm bindings for HF Tokenizers and Tiktoken

RustMITupdated Mar 30, 2026
GitHub โ†—โ˜… 227โ‘‚ 44

Data from GitHub ยท snapshot Sep 24, 2026