Added AU, FR, DE to Enhanced Mode proxies, crawl subdomains with allowSubdomains parameter, Google Slides scraping, PDF generation of current page, higher resolution screenshots with quality parameter, weekly usage view on dashboard, open-source Fireplexity (Perplexity clone) example, and additional improvements.
New concurrency system with per-request max concurrency control for crawl and batch scrape, crawl entire domain parameter to follow internal links to siblings/parent URLs, Google Docs scraping support, improved activity logs with FIRE-1 request support, enhanced /search playground with location parameters, open-source Firestarter chatbot building platform example, plus performance improvements and bug fixes.
Firecrawl Index for 500% faster scraping speeds, enhanced activity logs with webhook event viewing and active crawl management, fire-enrich open-source Clay integration example, community Java SDK support, and many additional improvements.
Launch of new Search API endpoint combining web search with Firecrawl's scraping capabilities. Features include searching the web and getting full content from results in one API call, customizable output formats (markdown, HTML, links, screenshots), search parameter customization (language, country, time range, number of results), and full SDK support for Python and Node.js.
Self-host improvements including Supabase client error fixes, LLM provider support fixes, faster crawling, global adoption of cacheable lookup system, and easier setup. MCP improvements with better prompts and examples. SDK enhancements: change tracking in SDK 2.0, crawl delay support with per-crawl concurrency limiting, Qwen3 crawler example, cancel batch scrape endpoint. Performance improvements: map endpoint limit increased from 5,000 to 30,000, search schema limit from 50 to 100. Better error handling for SSL failures, bug fixes for optional chaining and WaitAction field validation, concurrency queue prioritization by time. New activity logs in dashboard.
Enhanced Mode pricing change effective May 8th, 2025: proxy requests now cost 5 credits per request (previously included in standard pricing). Enhanced Mode is a specialized proxy feature for sites with advanced anti-bot protection, providing better success rates. Quality improvements reflected in pricing adjustment. Recommendation: use Enhanced Mode as retry mechanism for error codes 401, 403, or 500 to optimize credit usage.
Week-long launch event featuring: Day 7 - 20+ new/updated integrations (Discord Bot, Make, n8n, Langflow, LlamaIndex, Dify); Day 6 - Firecrawl MCP with FIRE-1 agent and SSE streaming; Day 5 - Python SDK v2.0 (async, named params, typed responses), Rust SDK, up to 20 seats per plan, Firecrawl VS Code Dark Theme; Day 4 - LLMstxt.new for clean .txt output with llms.txt and llms-full.txt; Day 3 - /extract v2 with FIRE-1 powered extraction, pagination, dynamic flows, search without URL; Day 2 - FIRE-1 Agent for navigation and interaction; Day 1 - Change tracking for detecting webpage changes with structured format; Day 0 - Firecrawl Editor Theme for VS Code, Cursor, Windsurf.
Deep Research open alpha with structured outputs and customizability, concurrent browsers with improved rate limits, Compare beta feature to detect website changes in /scrape and /crawl (closed beta), optional URLs in /extract endpoint, scrape endpoint concurrency-limit warnings, new examples featuring Claude 3.7, Gemini 2.5, Deepseek V3, Mistral 3.1, maxDiscoveryDepth option for crawl. Fixes: circular JSON error in search, reworked tally system, sitemaps poisoning fixed, crawler status retries (up to 3), credit check snaps to remaining credits, path filtering bug fix in Map, unsupported schema removed from llmExtract.
Introduced concurrent browsers in pricing and 5x'd the rate limits, now displayed as concurrent browsers. This change provides more visibility and allows users to upgrade for faster speeds across all plans. See the new rate limits in the docs.
LLMs.txt API: The /llmstxt endpoint transforms websites into clean, LLM-ready text files (llms.txt and llms-full.txt) for training or analysis with any LLM.
Deep Research API (Alpha): The /deep-research endpoint enables AI-powered deep research and analysis. Provide a research query and Firecrawl autonomously explores the web to synthesize comprehensive insights.
Official Firecrawl MCP Server: Give Cursor, Windsurf, and Claude enhanced web extraction capabilities.
Fixes & Enhancements: Improved charset detection, fixed extract token limit issues, addressed includes/excludes handling, and fixed AI SDK JSON response handling.
New Features: AI-SDK migration, auto-recharge emails, Fire-Index indexing system, self-hosting enhancements (OpenAI-compatible API & Ollama support), batch billing, Supabase read replica routing, Claude 3.7 and GPT-4.5 web crawlers, Groq web crawler example, improved crawl-status error handling, and better cross-origin redirect handling.
Self-Host Fixes: Reworked SELF_HOST.md and docker-compose.yaml for clarity, updated Kubernetes deployment examples, improved self-hosted performance and stability, added proxy support for self-hosted environments, fixed Playwright microservice integration, and added SearXNG support for /search endpoint.
Core Fixes & Enhancements: Fixed crawl status race conditions, added timeout enforcement for scrapeURL engines, preserved query parameters in map results, ensured screenshots execute after actions, improved PDF scraping on complex websites, integrated AbortController for timeout handling, and enforced SDK request timeouts.
New Features: Proxy configuration options, deep research alpha implementation, and llms.txt generation endpoint.
Docker & Containerization: Production-ready Docker image available for simplified self-hosted deployments.
Features & Enhancements: Added action and wait time validation for scrape API, improved PDF/image sub-link detection and text extraction via Gemini, enhanced multi-entity extraction prompts, moved source tracking out of experimental, added Serper and Search API env vars to docker-compose, updated credit system terminology from "credits" to "tokens".
Examples: Implemented Gemini 2.0 crawler example, Gemini TrendFinder, and Open Deep Research.
Fixes: Updated HTML transformer free_string parameter type, improved Gemini crawler library and PDF link extraction, fixed crawl queue worker page counting, fixed relative URL conversion, and enforced scrape rate limits in batch scraping.
Open source version of OpenAI Deep Research released. New extraction capability for R1 web extractor, lightweight O3-mini web crawler introduced, updated model parameters for o3-mini company researcher. Improvements include URL deduplication for trailing slashes and alternate forms, improved URL blocking using tldts parsing, valid JSON extraction via rawHtml in scrape, product reviews summarizer using o3-mini, scrape options for extract, O3-mini job resource extractor, and cached scrapes for extract evaluations.
New Features: Web search capabilities added to extract endpoint via enableWebSearch parameter, source tracking with __experimental_showSources parameter, configurable webhook events for crawl and batch operations, new timeout parameter for map endpoint, optional ad blocking with blockAds parameter (enabled by default).
Infrastructure & UI: Enhanced proxy selection and infrastructure reliability, added domain checker tool to cloud platform, redesigned LLMs.txt generator interface for better usability.
Extract endpoint now returns significantly more data. Improved infrastructure reliability, migrated from Cheerio to high-performance Rust-based parser for faster and more memory-efficient parsing, enhanced crawl cancellation functionality for better control over running jobs.
The /extract endpoint is now asynchronous. Requests return an ID for checking job status. SDK users require no code changes (update SDKs to latest versions). For direct API users, the change is backwards compatible with a 10-day migration window to the new asynchronous model.
New search endpoint combines web search with Firecrawl's scraping to return full page content for any query. Include scrapeOptions with formats: ["markdown"] to get complete markdown content for each result, otherwise defaults to SERP results.
Fixes and improvements: Fixed LLM schema following in Python SDK for /extract, fixed schema JSON sending in Node SDK for /extract, made prompt optional for /extract endpoint, set fork of MinerU as default for PDF parsing.
Feature Enhancements: Geolocation and mobile scraping support, 4x faster parsing, improved webhooks, credit packs, auto-recharges, batch scraping support, iframe support, query parameter differentiation for URLs, similar URL deduplication, enhanced map ranking, and improved sitemap fetching.
Performance Improvements: Faster crawl status filtering, improved map ranking algorithm, optimized Kubernetes setup, simplified build processes, enhanced sitemap discoverability.
Bug Fixes: Resolved JSON formatting, scrolling actions, encoding errors, crawl limits, relative URLs, error handling, self-hosted crawling inconsistencies, and schema errors.
SDK Updates: Added dynamic WebSocket imports with fallback, optional API keys for self-hosted instances, improved error handling.
Documentation Updates: Improved API docs and examples, updated self-hosting URLs, added Kubernetes optimizations, new articles on /scrape and /crawl.
New Batch Scrape endpoint allows scraping multiple URLs simultaneously. Python SDK (1.4.x) and Node SDK (1.7.x) updated with batch scrape support.
Added crawl cancellation support for Python SDK (1.3.x) and Node SDK (1.6.x). New examples: OpenAI Voice + Firecrawl and CRM lead enrichment. Improved Docker images, fixed limits and timeout issues for self-hosted Playwright scraper, and improved scrape speed overall.