🏆 #3,656 overall#89 of 178 in Tokenization & Preprocessing

amazon-far /deltatok

[CVPR 2026 Highlight] A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens

$ git clone https://github.com/amazon-far/deltatok.git
GitHub social preview for amazon-far/deltatok
Stars
279
279
This week
+2 ★
+6 this month
Forks
7
7
Language
Python
License
Apache-2.0
Created
Mar 27, 2026
0.6 years old
Last push
Jul 17, 2026

Stars, last 30 days

Oct 15, 2026 → Nov 16, 2026
★ 273▲ +6★ 279

Categories

GitHub topics

More in Tokenization & Preprocessing

Tokenization & Preprocessing

FoundationVision/UniTok

▲ +2 ★ this week

[NeurIPS 2025 Spotlight] A Unified Tokenizer for Visual Generation and Understanding

🤖 Language Models
PythonMITupdated Nov 14, 2025
GitHub ↗★ 538⑂ 16
Tokenization & Preprocessing

guillaume-be/rust-tokenizers

Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram (SentencePiece) models

RustApache-2.0updated Jan 22, 2026
GitHub ↗★ 345⑂ 33
Tokenization & Preprocessing

sugarme/tokenizer

NLP tokenizers written in Go language

🧠 Natural Language Processing
GoApache-2.0updated May 26, 2026
GitHub ↗★ 342⑂ 69
Tokenization & Preprocessing

AayushSameerShah/Neural-Net-Zero-to-Hero-with-Andrej

▲ +1 ★ this week

This repository contains the collection of explorative notebooks pure in python and in the language that we, humans can read. Have tried to compile all lectures from the Andrej Karpathy's 💎 playlist on Neural Networks - which we will end up with building GPT.

Jupyter NotebookMITupdated Apr 30, 2024
GitHub ↗★ 148⑂ 18
Tokenization & Preprocessing

frothywater/kanade-tokenizer

Kanade is a single-layer disentangled speech tokenizer that extracts compact tokens suitable for both generative and discriminative modeling.

Pythonno licenseupdated Jul 18, 2026
GitHub ↗★ 119⑂ 16
Tokenization & Preprocessing

rasbt/LLMs-from-scratch

▲ +499 ★ this week

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

🤖 Language Models🧠 Natural Language Processing
Jupyter NotebookOtherupdated Nov 9, 2026
GitHub ↗★ 108.5K⑂ 16.8K

Data from GitHub · snapshot Nov 16, 2026