v1.4.2

We're excited to announce several new features and improvements:

New Features

• Added web search capabilities to the extract endpoint via the enableWebSearch parameter
• Introduced source tracking with __experimental_showSources parameter
• Added configurable webhook events for crawl and batch operations
• New timeout parameter for map endpoint
• Optional ad blocking with blockAds parameter (enabled by default)

Infrastructure & UI

• Enhanced proxy selection and infrastructure reliability
• Added domain checker tool to cloud platform
• Redesigned LLMs.txt generator interface for better usability

What's Changed

• (feat/extract) Refactor and Reranker improvements by @nickscamara in https://github.com/mendableai/firecrawl/pull/1100
• Fix bad WebSocket URL in CrawlWatcher by @ProfHercules in https://github.com/mendableai/firecrawl/pull/1053
• (feat/extract) Add sources to the extraction by @nickscamara in https://github.com/mendableai/firecrawl/pull/1101
• feat(v1/map): Timeout parameter (FIR-393) by @mogery in https://github.com/mendableai/firecrawl/pull/1105
• fix(scrapeURL/fire-engine): default to separate US-generic proxy list if no location is specified (FIR-728) by @mogery in https://github.com/mendableai/firecrawl/pull/1104
• feat(scrapeUrl/fire-engine): add blockAds flag (FIR-692) by @mogery in https://github.com/mendableai/firecrawl/pull/1106
• (feat/extract) Logs analyzeSchemaAndPrompt output did not match the schema by @nickscamara in https://github.com/mendableai/firecrawl/pull/1108
• (feat/extract) Improved completions to use model's limits by @nickscamara in https://github.com/mendableai/firecrawl/pull/1109
• feat(v0): store v0 users (team ID) in Redis for collection (FIR-698) by @mogery in https://github.com/mendableai/firecrawl/pull/1111
• feat(github/ci): connect to tailscale (FIR-748) by @mogery in https://github.com/mendableai/firecrawl/pull/1112
• (feat/conc) Move fully to a concurrency limit system by @nickscamara in https://github.com/mendableai/firecrawl/pull/1045
• Added instructions for empty string to extract prompts by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/1114

New Contributors

• @ProfHercules made their first contribution in https://github.com/mendableai/firecrawl/pull/1053

Full Changelog: https://github.com/mendableai/firecrawl/compare/1.4.1...v1.4.2 Firecrawl website changelog: https://firecrawl.dev/changelog