Skip to main content
Scrape.do logo

Overview

Add Scrape.do to any ModuleX agent or workflow. Enterprise web-scraping API integration: basic HTTP, JS-rendered browser, screenshots, markdown conversion, and credit-usage stats. All five actions hit api.scrape.do (or /info for usage).
Categories: Web Search & Scraping · Data Extraction · Auth: API Key · Actions: 5

Authentication

API Key Authentication

Authenticate using your Scrape.do API key

Required Credentials

Available Actions

Parameters

string
required
URL to scrape
string
HTTP method (GET, POST, PUT, DELETE, HEAD) (Default: GET)
string
Request body for POST/PUT requests
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
boolean
Let Scrape.do add default headers
boolean
Forward extra upstream headers
boolean
Forward client headers to target
string
Cookies to send (JSON string or header)
boolean
Block images, CSS, fonts to speed up loading
boolean
Block advertisements
string
Output format (‘raw’ or ‘markdown’)
boolean
Return the origin response body with no parsing

Response

Parameters

string
required
URL to scrape
string
HTTP method (GET, POST, PUT, DELETE, HEAD) (Default: GET)
string
Request body for POST/PUT requests
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
boolean
Let Scrape.do add default headers
boolean
Forward extra upstream headers
boolean
Forward client headers to target
string
Cookies to send (JSON string or header)
boolean
Block images, CSS, fonts to speed up loading
boolean
Block advertisements
string
Output format (‘raw’ or ‘markdown’)
string
Wait condition: ‘domcontentloaded’, ‘networkidle0’, ‘networkidle2’, ‘load’
string
CSS selector to wait for before capturing
integer
Additional wait time in ms
integer
Browser viewport width (Default: 1920)
integer
Browser viewport height (Default: 1080)
string
JSON-encoded Play-with-Browser action list

Response

Parameters

string
required
URL to capture
boolean
Capture full page instead of viewport (Default: false)
string
CSS selector for element-specific screenshot
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
integer
Viewport width (Default: 1920)
integer
Viewport height (Default: 1080)
string
Wait condition for render completion
string
CSS selector to wait for before capturing
integer
Additional wait time in ms
boolean
Block advertisements
boolean
Let Scrape.do add default headers
string
Cookies to send

Response

Parameters

string
required
URL to scrape
boolean
Enable JavaScript rendering (Default: false)
string
HTTP method (Default: GET)
string
Request body for POST/PUT
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
boolean
Block images, CSS, fonts to speed up loading
boolean
Block advertisements
boolean
Let Scrape.do inject default headers
string
Cookies to send
string
JSON-encoded Play-with-Browser script

Response

Response

Limits & Quotas

  • Each scrape action exposes 20+ optional knobs (proxy routing, geo-targeting, device emulation, cookies, headers, wait conditions, viewport). All map to Scrape.do’s camelCase query string keys via a single _PARAM_MAP translation table.
  • take_screenshot is mutually-exclusive between viewport / full-page / element modes — the tool validates that full_page and selector aren’t both set.
  • Output shape varies per upstream response:
    • JSON → payload: dict
    • text/html/markdown → data: str with is_binary=False
    • image/* → data: <base64> with is_binary=True
  • 180s timeout for scrape operations (matches legacy); 30s for the usage-stats endpoint.

Ahrefs

Airweave

Apify