Syllable-aware BPE tokenizer for the Amharic language (አማርኛ) – fast, accurate, trainable.
$ git clone https://github.com/sefineh-ai/Amharic-Tokenizer.gitFull-stack open-source AI engine for building language models — tokenizer training, transformer architecture, cognitive reasoning and chat pipeline.
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
💫 Industrial-strength Natural Language Processing (NLP) in Python
EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING
Fast and customizable text tokenization library with BPE and SentencePiece support
NLP tokenizers written in Go language
Data from GitHub · snapshot Sep 24, 2026