✦ 16 veins to mine ✦
Every open source text mining and NLP project in the directory, grouped by what it does.
General-purpose natural language processing toolkits: tokenizing, tagging, parsing and modeling text in dozens of languages. Start here if you need a broad NLP library rather than a single-task tool.
▲ +5.6K stars this week
Information retrieval, semantic search, text similarity and retrieval-augmented generation (RAG): everything for finding the right passage in a large pile of text.
▲ +7.5K stars this week
Projects for training, fine-tuning, running and evaluating language models, from compact transformers to large language models, plus the tooling that makes them practical for text tasks.
▲ +6.6K stars this week
Automatic speech recognition (ASR) and speech-to-text projects that turn audio into transcripts you can mine like any other text.
▲ +4.4K stars this week
Turn scans, PDFs and images into usable text. Optical character recognition, layout analysis, table extraction and document understanding.
▲ +6.4K stars this week
Find people, organizations, places, products and other entities in text, then link them to knowledge bases. Includes NER models, entity linking and extraction tools.
▲ +434 stars this week
Label text automatically: topic and intent classification, spam and toxicity detection, hate speech filtering and general-purpose classifiers.
▲ +286 stars this week
Measure opinion and emotion in reviews, social posts and support tickets, from lexicon-based scorers to fine-tuned transformer models.
▲ +295 stars this week
End-to-end text mining and text analytics: frameworks and apps for exploring, processing and finding patterns across large text collections.
▲ +220 stars this week
Turn words, sentences and documents into vectors for search, clustering and similarity: word2vec-style embeddings, sentence transformers and modern text embedding models.
▲ +534 stars this week
Pull structured facts out of unstructured text: relations, events, keywords and keyphrases, ready to feed into databases and knowledge graphs.
▲ +208 stars this week
Systems that read documents and answer questions about them: extractive QA, reading comprehension and open-domain question answering.
▲ +859 stars this week
The unglamorous first step of every text pipeline: tokenizers, stemmers, lemmatizers, spell checkers and text cleaning utilities.
▲ +659 stars this week
Neural machine translation models, toolkits and services for translating text between languages, including low-resource language work.
▲ +278 stars this week
Condense long documents into short summaries, with extractive approaches that pick key sentences and abstractive models that rewrite.
▲ +101 stars this week
Discover the themes running through a collection of documents, from classic LDA to embedding-based approaches like BERTopic.
▲ +59 stars this week
