Firecrawl V1 is here! With that we introduce a more reliable and developer friendly API.
August 29th, 2024
Here is what’s new:
- Output Formats for /scrape. Choose what formats you want your output in.
- New /map endpoint for getting most of the URLs of a webpage.
- Developer friendly API for /crawl/{id} status.
- 2x Rate Limits for all plans.
- Go SDK and Rust SDK
- Teams support
- API Key Management in the dashboard.
- onlyMainContent is now default to true.
- /crawl webhooks and websocket support.
Learn more about it here
Start using v1 right away at https://firecrawl.dev
What's Changed (including v0 + v1)
- Delete .DS_Store by @szepeviktor in https://github.com/mendableai/firecrawl/pull/8
- [Bugfix] added normalized apikey to craw/status route by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/12
- [Feat] improving reative paths by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/4
- Fix typos by @szepeviktor in https://github.com/mendableai/firecrawl/pull/9
- [Feat] Added html to markdown table parser by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/11
- Option to extract only the main content, excluding headers, navs, footers etc. by @nickscamara in https://github.com/mendableai/firecrawl/pull/14
- [Feat] Adding pdf parser by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/17
- adding ci-cd workflow by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/20
- adding workflow by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/21
- adding env secrets by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/22
- [Feat] Added TSDocs and types for js-sdk by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/28
- Added option to replace all relative paths with absolute paths by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/25
- [Bugfix] Fixed scrape preview test by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/30
- Caleb: fixing some documentation and rebuilding the server by @calebpeffer in https://github.com/mendableai/firecrawl/pull/32
- Rate limit fixes for crawl status by @nickscamara in https://github.com/mendableai/firecrawl/pull/36
- Better logging by @nickscamara in https://github.com/mendableai/firecrawl/pull/35
- [Feat] Added type declarations by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/31
- Refactor api routes by @nickscamara in https://github.com/mendableai/firecrawl/pull/37
- Logging by @nickscamara in https://github.com/mendableai/firecrawl/pull/38
- Cjp/making db auth optional <> Running project locally by @calebpeffer in https://github.com/mendableai/firecrawl/pull/40
- chore: add context.close by @mattzcarey in https://github.com/mendableai/firecrawl/pull/46
- Fixes table parsing for websites such as news.ycombinator.com (HN) by @nickscamara in https://github.com/mendableai/firecrawl/pull/52
- [Feat] Server health check + slack message by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/53
- [Feat] Added blocklist for social media urls by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/55
- [Feat:mvp] Search Endpoint => serp api + firecrawl => 🔥 🔍 by @nickscamara in https://github.com/mendableai/firecrawl/pull/56
- [Feat] Added anthropic vision api by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/5
- [Bugfix] Trim and Lowercase all urls by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/13
- Implements the ability for the crawler to output all the links it found, without scraping by @nickscamara in https://github.com/mendableai/firecrawl/pull/34
- Serper params by @nickscamara in https://github.com/mendableai/firecrawl/pull/62
- Support for tbs, filter, lang, country and location with Serper search. by @rogerserper in https://github.com/mendableai/firecrawl/pull/61
- [Feat] Added allowed urls by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/64
- /search support in node sdk by @nickscamara in https://github.com/mendableai/firecrawl/pull/72
- Free credits increase by @nickscamara in https://github.com/mendableai/firecrawl/pull/75
- [Bugfix] JS-SDK: Remove dotenv and add tests by @mdp in https://github.com/mendableai/firecrawl/pull/68
- [Feat] Coupon system by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/66
- Specific website params support by @nickscamara in https://github.com/mendableai/firecrawl/pull/83
- Greenpay fixes by @nickscamara in https://github.com/mendableai/firecrawl/pull/84
- [Feat] Implemented retry attempts to handle 502 errors by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/67
- feat: LLM Extraction (mvp) by @nickscamara in https://github.com/mendableai/firecrawl/pull/90
- Update README.md by @bllchmbrs in https://github.com/mendableai/firecrawl/pull/110
- Add Posthog Logging by @ericciarla in https://github.com/mendableai/firecrawl/pull/109
- Refactor of main web scraper + Partial data streaming by @nickscamara in https://github.com/mendableai/firecrawl/pull/120
- [Feat] Added includeHTML option by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/126
- Cancel Job Route by @nickscamara in https://github.com/mendableai/firecrawl/pull/129
- [Feat] Added max depth option by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/130
- Add keyAuth endpoint by @ericciarla in https://github.com/mendableai/firecrawl/pull/131
- [Test] Added integration tests suite by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/118
- Adds Zod Integration for LLM Extraction in the Firecrawl JS SDK by @nickscamara in https://github.com/mendableai/firecrawl/pull/135
- [Docs] Updated examples by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/137
- Switching to AGPL - We Need Your Consent! by @calebpeffer in https://github.com/mendableai/firecrawl/pull/134
- Nsc/refactor scraping order by @nickscamara in https://github.com/mendableai/firecrawl/pull/139
- Update models.ts by @ericciarla in https://github.com/mendableai/firecrawl/pull/144
- Timeout on /scrape by @nickscamara in https://github.com/mendableai/firecrawl/pull/145
- [Doc] Added default value for crawlOptions.limit by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/142
- feat: 4x-5x faster crawler (fast mode) by @nickscamara in https://github.com/mendableai/firecrawl/pull/149
- Add Docker Compose for easy self hosting by @chand1012 in https://github.com/mendableai/firecrawl/pull/119
- refactor: fix typo in WebScraper/index.ts by @eltociear in https://github.com/mendableai/firecrawl/pull/27
- [Tests] Added crawl test suite -> crawl improvements by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/153
- feat: Docx Support by @nickscamara in https://github.com/mendableai/firecrawl/pull/158
- Fixes pdfs not found if .pdf is not present by @nickscamara in https://github.com/mendableai/firecrawl/pull/29
- Update README.md: Typo fix by @elimisteve in https://github.com/mendableai/firecrawl/pull/160
- [Feat] Added rate limits by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/151
- Allow override of API URL by @mattjoyce in https://github.com/mendableai/firecrawl/pull/166
- feat: HyperDX Integration by @nickscamara in https://github.com/mendableai/firecrawl/pull/167
- beta: Fire-Engine fallback by @nickscamara in https://github.com/mendableai/firecrawl/pull/174
- Add additional file extensions to crawler.ts by @tractorjuice in https://github.com/mendableai/firecrawl/pull/77
- [Bug] Fixing /crawl limit by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/143
- Update issue templates by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/180
- [Feat] Added proxy and media blocking support for Playwright by @JakobStadlhuber in https://github.com/mendableai/firecrawl/pull/181
- update: wait until body attached in playwright-service by @qyou in https://github.com/mendableai/firecrawl/pull/170
- feat: Allow privacy/legal/ other pages in social media websites by @nickscamara in https://github.com/mendableai/firecrawl/pull/168
- [Bug] Added data check for python SDK by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/176
- Fix FIRECRAWL_API_URL bug, also various PyLint fixes by @mattjoyce in https://github.com/mendableai/firecrawl/pull/178
- [Feat] Added idempotency key to crawl route by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/132
- Feat: Provide more details for 429 error msg by @simonha9 in https://github.com/mendableai/firecrawl/pull/190
- Limit on /search is not deterministic by @Keredu in https://github.com/mendableai/firecrawl/pull/186
- Various PyPi Metadata by @mattjoyce in https://github.com/mendableai/firecrawl/pull/191
- [Test] Added sdk e2e tests by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/183
- Allow users to manually set the waitFor param on /scrape by @nickscamara in https://github.com/mendableai/firecrawl/pull/200
- [Feat] Added custom scraping conditions for readme docs by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/204
- Feat/screenshot support by @ericciarla in https://github.com/mendableai/firecrawl/pull/207
- feat: New pricing/limits changes by @nickscamara in https://github.com/mendableai/firecrawl/pull/216
- [sdk] Fixes waiting status not being present on check status by @nickscamara in https://github.com/mendableai/firecrawl/pull/218
- Fixed fire-engine content bug by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/228
- Use @ instead of # for default BULL_AUTH_KEY. Hash mark is reserved for URI fragments. by @rombru in https://github.com/mendableai/firecrawl/pull/229
- feat: Ability to forward headers to reliable providers for auth etc... by @nickscamara in https://github.com/mendableai/firecrawl/pull/221
- Improvements to the blocklist regex by @nickscamara in https://github.com/mendableai/firecrawl/pull/185
- Playwright service bugs #222 #179 #197 by @mattjoyce in https://github.com/mendableai/firecrawl/pull/224
- [Fix] Changed timeout parameter name on js sdk by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/211
- [Bug] Improved js response and test for getting partial_data by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/212
- Better fallbacks for initial crawl start by @nickscamara in https://github.com/mendableai/firecrawl/pull/148
- [Feat] Added custom scraping for google-drive pdf usecase by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/235
- Script to check local vs published versions of sdk packages by @mattjoyce in https://github.com/mendableai/firecrawl/pull/213
- Update README.md by @calebpeffer in https://github.com/mendableai/firecrawl/pull/140
- Refactor custom scraping and added security center vanta by @nickscamara in https://github.com/mendableai/firecrawl/pull/237
- [Feat] Added scroll xpaths on fire-engine for handling readme docs by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/239
- Add Kubernetes configuration for Firecrawl deployment by @JakobStadlhuber in https://github.com/mendableai/firecrawl/pull/236
- [Feat] Improved the scrape for gdrive pdfs by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/238
- Add support for Self-Hosted Webhook URL Usage and added project_id into the webhook payload by @JakobStadlhuber in https://github.com/mendableai/firecrawl/pull/234
- Partial Data Sliding window of 50 by @nickscamara in https://github.com/mendableai/firecrawl/pull/242
- feat: Transactional emails for rate limits and credit limits by @nickscamara in https://github.com/mendableai/firecrawl/pull/244
- Python-SDK transitional build setup for pyproject.toml by @mattjoyce in https://github.com/mendableai/firecrawl/pull/196
- [Feat] CI/CD for publishing js and python SDKs by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/246
- 194 sdk ci pipeline for publishing pythonnode sdk by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/248
- fixing minor problems on workflow by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/250
- Fix 208 py sdk interval poll name by @mattjoyce in https://github.com/mendableai/firecrawl/pull/252
- Moving from fetch to axios and preventing deadlocks by setting timeouts on fallbacks by @nickscamara in https://github.com/mendableai/firecrawl/pull/260
- ignoreSitemap feature, pageOptions now respected in the initial crawl as well by @nickscamara in https://github.com/mendableai/firecrawl/pull/263
- Fixed bugs associated with absolute path replacements by @nickscamara in https://github.com/mendableai/firecrawl/pull/268
- Only fetch webhook from db if self host webhook not set and using db auth by @nickscamara in https://github.com/mendableai/firecrawl/pull/262
- [Feat] Added route to clean completed jobs and a github action cron that triggers every 24h by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/265
- [Feat] Added allowBackwardCrawling option by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/269
- [Feat] Added jobId to webhook data by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/272
- Clusters support to scale our API to # of CPUs running by @nickscamara in https://github.com/mendableai/firecrawl/pull/274
- Added pageOptions.removeTags by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/275
- Added logging to python sdk FIRECRAWL_LOGGING_LEVEL by @mattjoyce in https://github.com/mendableai/firecrawl/pull/256
- Python sdk improve response handling by @mattjoyce in https://github.com/mendableai/firecrawl/pull/253
- [Feat] Added parsePDF option to pageOptions by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/271
- [Feat] Added metadata.pageStatusCode and metadata.pageError properties by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/276
- Make maxDepth relative to the entered url by @ericciarla in https://github.com/mendableai/firecrawl/pull/277
- Test/load testing by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/175
- Fixes crawler getting confused with base paths that contain www. by @nickscamara in https://github.com/mendableai/firecrawl/pull/283
- Feat/max depth relative by @ericciarla in https://github.com/mendableai/firecrawl/pull/285
- [Feat] Improvements on response document types by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/298
- test: Rate Limit Unit Tests by @nickscamara in https://github.com/mendableai/firecrawl/pull/282
- Added local host support for the javascript SDK by @NeevJewalkar in https://github.com/mendableai/firecrawl/pull/296
- [Test] Transcribed from e2e to unit tests for many cases by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/294
- [Feat] Added support for RegEx in removeTags by @AndyMik90 in https://github.com/mendableai/firecrawl/pull/297
- [Bug] Fixed includeHTML to use cleanedHtml as response by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/301
- Cjp/email to posthog logging by @calebpeffer in https://github.com/mendableai/firecrawl/pull/302
- Fix Broken Link by @Lakr233 in https://github.com/mendableai/firecrawl/pull/303
- docs: Fix pydanti to pydantic by @100gle in https://github.com/mendableai/firecrawl/pull/308
- [Bug] Fixed axios bug that were making jobs stuck on active queue by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/312
- fix multi-word search term issue: /search (w/o Serp) by @snippet in https://github.com/mendableai/firecrawl/pull/313
- [Bug] Fixed clean jobs by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/317
- [Bug] Fixed the regex test for google drive pdf files by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/318
- add some types by @snippet in https://github.com/mendableai/firecrawl/pull/316
- [Bug] Added default values and fixed pdf bug by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/321
- [Test] Added E2E tests for checking metadata values by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/322
- pageOptions.onlyIncludeTags param by @nickscamara in https://github.com/mendableai/firecrawl/pull/328
- Update CONTRIBUTING.md by @george-zakharov in https://github.com/mendableai/firecrawl/pull/329
- Fix headers | pageOptions not being passed correctly to fire engine by @nickscamara in https://github.com/mendableai/firecrawl/pull/331
- [Tests] Added crawl limit unit test by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/323
- Adds pageOptions.includeRawHtml and new extraction mode "llm-extraction-from-raw-html" by @ericciarla in https://github.com/mendableai/firecrawl/pull/332
- [Proposal] new feature allowExternalContentLinks by @snippet in https://github.com/mendableai/firecrawl/pull/336
- [PROPOSAL] (deps): making sure all deps are always up to date by @Sanix-Darker in https://github.com/mendableai/firecrawl/pull/338
- [PROPOSAL] (docker-compose) regroup envs vars between services by @Sanix-Darker in https://github.com/mendableai/firecrawl/pull/337
- new playwright service by @snippet in https://github.com/mendableai/firecrawl/pull/325
- apps/playwright-service(deps): bump the prod-deps group in /apps/playwright-service with 3 updates by @dependabot in https://github.com/mendableai/firecrawl/pull/346
- Logging for all scraper methods by @nickscamara in https://github.com/mendableai/firecrawl/pull/363
- setting up docker to ts playwright service by @snippet in https://github.com/mendableai/firecrawl/pull/361
- (Docs) Self Host added new ts playwright service instructions by @snippet in https://github.com/mendableai/firecrawl/pull/362
- apps/test-suite(deps): bump the prod-deps group in /apps/test-suite with 6 updates by @dependabot in https://github.com/mendableai/firecrawl/pull/347
- apps/test-suite(deps-dev): bump typescript from 5.4.5 to 5.5.3 in /apps/test-suite in the dev-deps group by @dependabot in https://github.com/mendableai/firecrawl/pull/348
- apps/api(deps): bump the prod-deps group in /apps/api with 28 updates by @dependabot in https://github.com/mendableai/firecrawl/pull/349
- [Feat] Added veeva to special case params by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/371
- Only check Supabase if configured to when cancelling job by @StefanTerdell in https://github.com/mendableai/firecrawl/pull/374
- dependabot for security checks, fixed crawl test by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/370
- [Feat] Added implementation for saving docs on supabase by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/326
- Self-hosting quality of life fixes by @StefanTerdell in https://github.com/mendableai/firecrawl/pull/375
- [Feat] Added fire-engine fallback for getting sitemaps by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/386
- fix(bull): requeue jobs after restart by @mogery in https://github.com/mendableai/firecrawl/pull/393
- Fix USE_DB_AUTHENTICATION checks for self-host by @kun432 in https://github.com/mendableai/firecrawl/pull/395
- [BUG] Fixed docker self hosting issue by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/396
- Slack Alerts for when queue is over capacity by @nickscamara in https://github.com/mendableai/firecrawl/pull/397
- CI for alerts instead by @nickscamara in https://github.com/mendableai/firecrawl/pull/398
- Log Fire-engine page errors by @nickscamara in https://github.com/mendableai/firecrawl/pull/394
- Sitemap fallback fixes w/ fire-engine by @nickscamara in https://github.com/mendableai/firecrawl/pull/399
- Separate Rate Limiter from Main Redis by @nickscamara in https://github.com/mendableai/firecrawl/pull/417
- [Feat] Pass along current, total, current_step, and current_url in js sdk by @jhoseph88 in https://github.com/mendableai/firecrawl/pull/391
- Redis Health Checks by @nickscamara in https://github.com/mendableai/firecrawl/pull/424
- Small fix for empty pageOptions by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/420
- Caleb: Return a list of links on a page by default by @calebpeffer in https://github.com/mendableai/firecrawl/pull/423
- [Docs] Updating docs by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/427
- Fix queue stuck bug via lock settings changes by @mogery in https://github.com/mendableai/firecrawl/pull/429
- fix(js-sdk): transform tests with ts-jest and configure node by @mogery in https://github.com/mendableai/firecrawl/pull/433
- fix(WebScraper): infinite regex leading to fly.io instance hangs by @mogery in https://github.com/mendableai/firecrawl/pull/436
- Support chrome-cdp and restructure sitemap fire-engine support. by @tomkosm in https://github.com/mendableai/firecrawl/pull/410
- /scrape should now be 600ms-900ms faster by @nickscamara in https://github.com/mendableai/firecrawl/pull/447
- Notion Website Fixes by @nickscamara in https://github.com/mendableai/firecrawl/pull/453
- Firecrawl UI template by @ericciarla in https://github.com/mendableai/firecrawl/pull/458
- Added regex for links in sitemap by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/449
- Added logger by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/450
- Add scrape monitoring by @mogery in https://github.com/mendableai/firecrawl/pull/455
- Readiness liveness probes by @JakobStadlhuber in https://github.com/mendableai/firecrawl/pull/457
- Client side error handling by @nickscamara in https://github.com/mendableai/firecrawl/pull/461
- Admin router + Improve redis notifications by @nickscamara in https://github.com/mendableai/firecrawl/pull/460
- support custom models by @NiuBlibing in https://github.com/mendableai/firecrawl/pull/414
- DNS Cacheable Lookup to avoid DNS blocking ops by @nickscamara in https://github.com/mendableai/firecrawl/pull/478
- Fire-engine to be the default for our cloud-service by @nickscamara in https://github.com/mendableai/firecrawl/pull/479
- [Bug] Rate limiter bug on switch for some cases by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/482
- [Feat] Better self hosting guide by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/483
- Fix parallelization issues with caching by @nickscamara in https://github.com/mendableai/firecrawl/pull/484
- [Bug] Issue with crawl going beyond Limit by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/485
- [Bug] pdfs and logging pdf events, also added trycatchs for docx by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/475
- [Bug] Nested sitemaps by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/491
- Fix AMZN | Removal of redis alerts by @nickscamara in https://github.com/mendableai/firecrawl/pull/492
- Custom engine params fix by @nickscamara in https://github.com/mendableai/firecrawl/pull/494
- [Bug] Fixed the empty excludes.filter is undefined bug by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/503
- [Feat] Added fullpagescreenshot capabilities by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/504
- fix(js-sdk): build both CommonJS and ESM versions by @mogery in https://github.com/mendableai/firecrawl/pull/432
- fix(js-sdk): build both CJS and EJS versions by @mogery in https://github.com/mendableai/firecrawl/pull/508
- Improve logs by @tak-s in https://github.com/mendableai/firecrawl/pull/496
- Digikey support by @nickscamara in https://github.com/mendableai/firecrawl/pull/512
- [Feat] Add Go SDK implementation by @KentHsu in https://github.com/mendableai/firecrawl/pull/497
- Redlock cache in auth by @nickscamara in https://github.com/mendableai/firecrawl/pull/529
- Update redis urls in example .env by @wahpiangle in https://github.com/mendableai/firecrawl/pull/515
- Self-host fix: Moving comments of .env.example values from end-of-line to above-line. by @kevinswiber in https://github.com/mendableai/firecrawl/pull/531
- fix: go-sdk module name by @KentHsu in https://github.com/mendableai/firecrawl/pull/534
- Fixed e2e tests by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/536
- Removed obsoleted declaration by @matsubo in https://github.com/mendableai/firecrawl/pull/532
- [Feat] Added go-sdk as submodule by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/537
- feat: Move scraper to queue by @mogery in https://github.com/mendableai/firecrawl/pull/459
- Reduce metrics ingestion w/ HyperDX v0.8.1 by @nickscamara in https://github.com/mendableai/firecrawl/pull/480
- Check team credits based on the crawl limit by @nickscamara in https://github.com/mendableai/firecrawl/pull/554
- added variables to beta customers by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/555
- Set the crawl limit to the remaining credits by @nickscamara in https://github.com/mendableai/firecrawl/pull/559
- [Feat] Added check job and cancel to fire-engine requests by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/560
- [Bug] Added a way for dealing with DNS without IP resolution by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/561
- Added Sentry Monitoring by @nickscamara in https://github.com/mendableai/firecrawl/pull/562
- Internal Concurrency Limits <> Job Priorities by @nickscamara in https://github.com/mendableai/firecrawl/pull/566
- [V1] Release by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/527
- [Feat]: Add RUST SDK client for firecrawl API by @Sanix-Darker in https://github.com/mendableai/firecrawl/pull/373
- Update README.md by @ericciarla in https://github.com/mendableai/firecrawl/pull/590
- [Bug] Moved delete rawHtml to end of controller by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/582
- Fixed RPC cloudflare issues with job ids by @nickscamara in https://github.com/mendableai/firecrawl/pull/593
- [v1] Websockets SDKs by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/591
- [v1] LLM Extract by @nickscamara in https://github.com/mendableai/firecrawl/pull/586
- [v1] Webhooks by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/594
- fix(v1): maxDepth by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/607
- Ensuring USE_DB_AUTHENTICATION is true in single URL scraper. by @kevinswiber in https://github.com/mendableai/firecrawl/pull/516
- Bill team async by @nickscamara in https://github.com/mendableai/firecrawl/pull/619
- Optimize check credits func w/ caching by @nickscamara in https://github.com/mendableai/firecrawl/pull/624
- Feat: Added go html to md parser by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/608
- Feat: parser singleton by @rafaelsideguide in https://github.com/mendableai/firecrawl/pull/629
New Contributors
Full Changelog: https://github.com/mendableai/firecrawl/commits/v1.0.0