Tokenization & Preprocessing
🏆 #929 overall#16 of 148 in Tokenization & Preprocessing🔥 active this week
uhop /stream-json
A micro-library of stream components for building custom JSON and JSONC processing pipelines with a minimal memory footprint — parse, filter, and transform JSON far larger than available memory with a SAX-inspired token API, on Node.js or Web Streams.
$ git clone https://github.com/uhop/stream-json.gitStars
1.2K
1,197
Forks
54
54
Language
JavaScript
License
BSD-3-Clause
Created
Aug 11, 2013
13.1 years old
Last push
Sep 20, 2026
🔥 this week
Categories
More in Tokenization & Preprocessing
Tokenization & Preprocessing
glayzzle/php-parser
:herb: NodeJS PHP Parser - extract AST or tokens
JavaScriptBSD-3-Clauseupdated Sep 10, 2026
Tokenization & Preprocessing
leodevbro/vscode-blockman
VSCode extension to highlight nested code blocks
TypeScriptMITupdated Jul 17, 2026
Tokenization & Preprocessing
netgen/query-translator
Query Translator is a search query translator with AST representation
PHPMITupdated Feb 13, 2026
Tokenization & Preprocessing
snchimata/tokenfold🔥 active
Private, provider-neutral context compression for LLMs. Reversibly compress schemas and logs inside your perimeter with zero third-party calls.
RustApache-2.0updated Sep 21, 2026
Tokenization & Preprocessing
mck89/peast
JavaScript parser written in PHP that generates AST from your code according to ECMAScript specification
PHPBSD-3-Clauseupdated Sep 15, 2026
Data from GitHub · snapshot Sep 24, 2026