Apify

Crawlee v3.17.0 shipped June 4 with a run of reliability fixes across browser-based crawling.

Sitemap hangs eliminated — network timeouts now guard discoverValidSitemaps, preventing indefinitely stalled crawl runs when a sitemap endpoint stops responding. Alongside that, Browser.pages() was rebound correctly in PuppeteerPlugin so multi-page Puppeteer sessions no longer lose their handle.

Browser-pool robustness improved on two fronts: proxy URLs on a launch context can now be unset (previously stuck once set), and the caller's AbortContext is preserved across the p-limit internal queue so cancellation signals propagate correctly under concurrency.

Actor.start, Actor.call, and Actor.call_task now correctly forward run kwargs that were previously dropped, shipped in Python SDK 3.4.1 on May 29.

Persistent actor state landed with Actor.use_state in 3.2.0, giving actors a first-class API for reading and writing durable state without manual storage wiring. The same release stopped silently swallowing exceptions in the request queue and fixed a TimeoutError in Actor.__aexit__ that had been leaking resources.

Named storage aliases arrived via the Actor schema Alias mechanism, letting operators reference storages by logical name — default resolves to the default storage. The Scrapy integration also migrated to Scrapy's native AsyncCrawlerRunner, dropping a custom async-event-loop shim.

Async and Scrapy paths hardened across patch releases: negative timedelta clamping, a ReentrantLock guarded against a None task context, cascading cleanup steps in Actor.__aexit__ isolated per step, mid-stream platform WebSocket disconnects handled gracefully, non-UTF-8 bodies tolerated in ApifyRequestList remote sources, and max_total_charge_usd=0 correctly preserved as a zero-dollar cap.

Finer proxy geography shipped with subdivision code support added to proxy configuration, enabling actors to target sub-country regions. Annotation correctness improved in 3.4.1 with push_data annotated as JsonSerializable instead of Any.