Crawlee released v4.0.0-rc.0
A release candidate for version 4.0.0, tagged by Martin Adámek on 12 Aug, including a fix for 403 responses in camoufox-cloudflare e2e tests.
See the latest news and media coverage for Crawlee. We track all announcements, press releases, and industry mentions in real time, all in one place.
Last updated
In short: Crawlee released the version 4.0.0 candidate and transitioned its Python library to stable version 1, enhancing its scraping toolset.
A release candidate for version 4.0.0, tagged by Martin Adámek on 12 Aug, including a fix for 403 responses in camoufox-cloudflare e2e tests.
It includes network timeout fixes, dynamic memory snapshots, and sitemap filtering enhancements.
Includes bug fixes, new features like @crawlee/stagehand package, and performance improvements across multiple packages.
Introduces AI-powered StagehandCrawler for natural language web interactions, async iterators for Dataset and KeyValueStore, discoverValidSitemaps utility, and enhanced Cloudflare handling.
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG...
I tried MemoryRequestQueueClient in python and it fails because of "'MemoryRequestQueueClient' object has no attribute 'add_requests'". Same with SqlRequestQueueClient and FileSystemRequestQueueClient. It seems like they...
I'm crawling thousands of domains using 10+ workers processing requests from a shared PostgreSQL request queue. I need per-domain rate limiting (e.g., "max 10 requests/minute...
Crawlee covers your crawling and scraping end-to-end and helps you build reliable scrapers. Fast. Your crawlers will appear human-like and fly under the radar of...
Track Crawlee and your other target companies to get real-time alerts and weekly summaries delivered straight to your inbox.
Browse news for competitors to Crawlee and other trending companies.