Headless browsers, crawlers, and scraping APIs — the data layer for AI agents.
This week: Hardening the foundations: stability fixes across browser automation →
Stagehand patched cross-frame XPath resolution, fixing element targeting inside iframes, alongside dependency refreshes across the SDK and server.
Stagehand's server-v3 release corrects coordinate normalization for Computer Use agents and ensures structured outputs retain the caller's provider for v3 users.
The Firecrawl Developer Index is now available, covering 70M+ artifacts across READMEs, docs, issues, PRs, and OpenAPI specs with semantic retrieval and metadata filters. It scores 0.63 recall@10 on the open DevDex benchmark, and each result includes stable IDs, URLs, and matched passages in markdown. Available via API at /search/developer, plus CLI, MCP, SDKs, and a companion skill.
Crawlee's patch release stops purging storages that are actively in use and adapts its Playwright Cloudflare challenge handling to the updated challenge markup.
Firecrawl Research Index now covers 41M+ life sciences papers across drug discovery, clinical trial, and biology literature, with 90% recall@10 on paper-retrieval evals. The index is now free to use and accessible via API, CLI, MCP, and SDKs.
The Python SDK fixes a websocket iterator leak on shutdown and stops terminal status failures from blocking Actor exit.
Crawlee v3.18.0 relaxes click options for link enqueuing and tightens crawler persistence to avoid duplicates.
Stagehand replaced raw daemon socket errors with a human-readable message that tells users the exact `browse open` command to start the session.
Firecrawl /search now returns the excerpts that best answer each query from each result, scoring paragraphs, lists, and tables with a custom relevance model. Agents achieve 94.7% on SimpleQA while using 10x fewer tokens than processing full pages — no API changes required.
Defines search queries and a goal, and Firecrawl searches the entire web on a schedule, alerting via webhook or email when something new matches. The goal field filters results to decide which trigger alerts, with a plain-English explanation of relevance.
Browserbase Agents is a managed solution that enables you to deploy agents on the web with a simple API call. No code to maintain or deploy—describe t...
Firecrawl Research Index searches 3M+ arXiv papers and GitHub code, core endpoints can be accessed without an API key from official clients, and a new redactPII option strips personal data from scraped content. Also ships deterministicJson format and video discovery on any page.
