hplt-project/sacremoses
▲ +1 ★ this week
Python port of Moses tokenizer, truecaser and normalizer
The code and models for "An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks" (AACL-IJCNLP 2020)
$ git clone https://github.com/kakaobrain/kortok.git▲ +1 ★ this week
Python port of Moses tokenizer, truecaser and normalizer
▲ +1 ★ this week
Fast and customizable text tokenization library with BPE and SentencePiece support
Implementations for a family of attention mechanisms, suitable for all kinds of natural language processing tasks and compatible with TensorFlow 2.0 and Keras.
▲ +4 ★ this week
State of the Art Natural Language Processing
▲ +1 ★ this week
Natural Language Processing Pipeline - Sentence Splitting, Tokenization, Lemmatization, Part-of-speech Tagging and Dependency Parsing
Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.
Data from GitHub · snapshot Nov 16, 2026