Apify SDK

2.3.0 (2025-02-19)

🚀 Features

• Add rate_limit_errors property for ApifyStorageClient (#387) (89c230a) by @Mantisus
• Unify Apify and Scrapy to use single event loop & remove nest-asyncio (#390) (96949be) by @vdusek
• Support pay-per-event via Actor.charge (#393) (78888c4) by @janbuchar

🐛 Bug Fixes

• Fix RQ usage in Scrapy scheduler (#385) (3363478) by @vdusek
• Make sure that Actor instances with non-default configurations are also accessible through the global Actor proxy after initialization (#402) (b956a02) by @janbuchar

2.2.1 (2025-01-17)

🐛 Bug Fixes

• Better event listener type definitions (#354) (52a6dee) by @janbuchar

2.2.0 (2025-01-10)

🚀 Features

• Add new config variables to Actor.config (#351) (7b6478c) by @fnesveda
• Upgrade to Crawlee v0.5 (#355) (826f4db) by @vdusek

🐛 Bug Fixes

• Better error message when attempting to use force_cloud without an Apify token (#356) (33245ce) by @janbuchar
• Allow calling Actor.reboot() from migrating handler, align reboot behavior with JS SDK (#361) (7ba0221) by @fnesveda

2.1.0 (2024-12-03)

🚀 Features

• Handle request list user input (#326) (c14fb9a) by @Pijukatel

🐛 Bug Fixes

• Add upper bound of HTTPX version (#347) (e86dbce) by @vdusek

2.0.2 (2024-11-12)

🐛 Bug Fixes

• Fix CPU usage calculation (#315) (0521d91) by @janbuchar
• Set version constraint of the websockets dependency to <14.0.0 (#322) (15ad055) by @Pijukatel
• Fix Dataset.iter_items for apify_storage (#321) (2db1beb) by @Pijukatel

🚀 Features

• Add standby URL and change default standby port
• Add crawlee version to system info output

🐛 Bug Fixes

• Allow empty timeout_at environment variable
• Use HttpHeaders type in Scrapy integration
• Adjust tests for Scrapy user data

2.0.0 (2024-09-10)

• Check the Upgrading to v2 guide.

🚀 Features

• Using Crawlee for Python under the hood (#210) by @janbuchar
• Better Actor API typing (#256) by @janbuchar
• Expose Request from Crawlee (#266) by @vdusek
• Improve examples in README (#269) by @vdusek
• Automatically configure logging (#271) by @janbuchar

🐛 Bug Fixes

• Make apify.log public (#249) by @janbuchar
• Dataset list response handling (#257) by @janbuchar
• Ignore deprecated platform events (#258) by @janbuchar
• Possible infinity loop in Apify-Scrapy proxy middleware (#259) by @vdusek
• Hotfix for batch_add_requests batch size limit (#261) by @janbuchar

Added

• Add actor standby port

Fixed

• Set a timeout for Actor cleanup

Added

• Add a new way of generating the uniqueKey field of the request, aligning it with the Crawlee.

Fixed

• Improve error handling for to_apify_request serialization failures
• Scrapy's Request.dont_filter works.

Fixed

• Update of Scrapy integration, fixes in ApifyScheduler, to_apify_request and apply_apify_settings

Removed

• Removed ApifyRetryMiddleware and stay with the Scrapy's default one

Fixed

• Fix conversion of headers fields in Apify <--> Scrapy request translation

Fixed

• Fix conversion of userData and headers fields in Apify <--> Scrapy request translation

Added

• Add apply_apify_settings function to Scrapy subpackage

Internal changes

• Create a new subpackage for Scrapy pipelines
• Remove some noqas thanks to the new Ruff release
• Replace relative imports with absolute imports
• Replace asserts with custom checks in Scrapy subpackage

Fixed

• Add missing import check to ApifyHttpProxyMiddleware

Internal changes

• Allowed running integration tests from PRs from forks, after maintainer approval
• Do not close nested_event_loop in the Scheduler.__del__()

Added

• Added ApifyHttpProxyMiddleware

Fixed

• Resolved issue in ApifyRetryMiddleware.process_exception(), where requests were getting stuck in the request queue

Internal changes

• Fixed type hint problems for resource clients

Internal changes

• Migrate from Autopep8 and Flake8 to Ruff

Added

• Added scrapy extra