Use LLMs to robustly extract web data
$ git clone https://github.com/lightfeed/extractor.git▲ +43 ★ this week
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
▲ +63 ★ this week
Incremental engine for long horizon agents 🌟 Star if you like it!
mloda.ai - Open Data Access for AI and ML. Plugin-based. Traceable. Framework-agnostic.
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
▲ +3 ★ this week
Any file → clean Markdown for AI agents: PDF, Word, PowerPoint, Excel, EPUB, HTML and web pages, images (OCR), audio and video (metadata, subtitles, transcripts). A fast Rust MCP server and CLI that runs on your machine. No API key.
Data from GitHub · snapshot Nov 13, 2026