Search & Retrieval
🏆 #3,099 overall#388 of 388 in Search & Retrieval
JuliaText /WordTokenizers.jl
High performance tokenizers for natural language processing and other related tasks
$ git clone https://github.com/JuliaText/WordTokenizers.jl.gitStars
100
100
Forks
25
25
Language
Julia
License
Other
Created
Apr 10, 2018
8.5 years old
Last push
Dec 30, 2021
More in Search & Retrieval
Search & Retrieval
piskvorky/gensim
Topic Modelling for Humans
🫧 Topic Modeling🧬 Embeddings🧠 Natural Language Processing
PythonLGPL-2.1updated Nov 1, 2025
Search & Retrieval
LSYS/LexicalRichness
:smile_cat: :speech_balloon: A module to compute textual lexical richness (aka lexical diversity).
🧠 Natural Language Processing
PythonMITupdated May 2, 2026
Search & Retrieval
artitw/text2text
Text2Text Language Modeling Toolkit
✂️ Tokenization & Preprocessing🧠 Natural Language Processing
PythonOtherupdated Jan 14, 2025
Search & Retrieval
neuml/txtai🔥 active
💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
🧬 Embeddings🤖 Language Models🧠 Natural Language Processing
PythonApache-2.0updated Sep 23, 2026
Search & Retrieval
beir-cellar/beir
A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
🧠 Natural Language Processing
PythonApache-2.0updated Oct 16, 2025
Data from GitHub · snapshot Sep 24, 2026