lightfeed/extractor
Use LLMs to robustly extract web data
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
$ git clone https://github.com/adbar/trafilatura.gitUse LLMs to robustly extract web data
MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.
▲ +1 ★ this week
Text2Text Language Modeling Toolkit
▲ +1 ★ this week
Language AI Engineering Lab, a place where you can deeply understand and build modern Language AI systems, from fundamentals to production.
▲ +300 ★ this week
💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
▲ +2 ★ this week
Dealing with all unstructured data, such as reverse image search, audio search, molecular search, video analysis, question and answer systems, NLP, etc.
Data from GitHub · snapshot Nov 16, 2026