crawl4ai

github.com ↗

An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents.