🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Lightweight library for scraping web-sites with LLMs

➖ Stripped down, stable version of firecrawl optimized for self-hosting and ease of contribution. Billing logic and AI features are completely removed. Crawl and convert any website into LLM-ready markdown.

🔥 This repository contains complete application examples, including websites and other projects, developed using Firecrawl.

⬇️ A simple all-in-one CLI tool to download EVERYTHING from a URL (like youtube-dl/yt-dlp, forum-dl, gallery-dl, simpler ArchiveBox). 🎭 Uses headless Chrome to get HTML, JS, CSS, images/video/audio/subtitles, PDFs, screenshots, article text, git repos, and more...

AI web scraper built with Crawl4AI for extracting structured leads data from websites.

How to guides on web-crawling or scraping

Python, Javascript, and Rust libraries for the Spider Cloud API.

[Mirror] Self-hosted abuse detection and rule enforcement against low-effort mass AI scraping and bots.

A CLI tool and REST API that converts web content to clean Markdown, bypassing anti-scraping measures using headless browsers. Perfect for AI/LLM applications

Use LLaMA 3 and Python to extract structured data from websites like Amazon, leveraging LLM-powered parsing for resilient, AI-driven web scraping.

This repository contains complete application examples, developed using Skrape.ai

Integrating OpenAI Agents SDK with Bright Data Web Unlocker, enabling AI agents to access, extract, and process structured data from protected web pages

TypeScript/Node.js SDK to easily interact with the skrape.ai API

AI tools to enhance productivity and automate web-scraping