🏆 #6,256 overall#123 of 133 in Machine Translation

kakaobrain /kortok

The code and models for "An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks" (AACL-IJCNLP 2020)

$ git clone https://github.com/kakaobrain/kortok.git
GitHub social preview for kakaobrain/kortok
Stars
119
119
This week
0 ★
0 this month
Forks
10
10
Language
Python
License
Apache-2.0
Created
Oct 5, 2020
6.1 years old
Last push
Oct 8, 2020

Stars, last 30 days

Oct 15, 2026 → Nov 16, 2026
★ 119▲ +0★ 119

Categories

GitHub topics

More in Machine Translation

Machine Translation

hplt-project/sacremoses

▲ +1 ★ this week

Python port of Moses tokenizer, truecaser and normalizer

✂️ Tokenization & Preprocessing🧠 Natural Language Processing
PythonMITupdated Sep 7, 2026
GitHub ↗★ 498⑂ 60
Machine Translation

OpenNMT/Tokenizer

▲ +1 ★ this week

Fast and customizable text tokenization library with BPE and SentencePiece support

✂️ Tokenization & Preprocessing🧠 Natural Language Processing
C++MITupdated Jan 10, 2026
GitHub ↗★ 344⑂ 83
Machine Translation

uzaymacar/attention-mechanisms

Implementations for a family of attention mechanisms, suitable for all kinds of natural language processing tasks and compatible with TensorFlow 2.0 and Keras.

🤖 Language Models💖 Sentiment Analysis🧠 Natural Language Processing
PythonMITupdated Feb 6, 2024
GitHub ↗★ 384⑂ 87
Machine Translation

JohnSnowLabs/spark-nlp🔥 active

▲ +4 ★ this week

State of the Art Natural Language Processing

🧠 Natural Language Processing🏷️ Named Entity Recognition💖 Sentiment Analysis🗂️ Text Classification❓ Question Answering✂️ Tokenization & Preprocessing
ScalaApache-2.0updated Nov 14, 2026
GitHub ↗★ 4.2K⑂ 743
Machine Translation

adobe/NLP-Cube

▲ +1 ★ this week

Natural Language Processing Pipeline - Sentence Splitting, Tokenization, Lemmatization, Part-of-speech Tagging and Dependency Parsing

✂️ Tokenization & Preprocessing⛏️ Information Extraction
HTMLApache-2.0updated Nov 3, 2024
GitHub ↗★ 565⑂ 92
Machine Translation

sebastianruder/NLP-progress

Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.

🏷️ Named Entity Recognition🧠 Natural Language Processing
PythonMITupdated Jul 28, 2024
GitHub ↗★ 23K⑂ 3.6K

Data from GitHub · snapshot Nov 16, 2026