Overview
Add Scrape.do to any ModuleX agent or workflow. Enterprise web-scraping API integration: basic HTTP, JS-rendered browser, screenshots, markdown conversion, and credit-usage stats. All five actions hitapi.scrape.do (or /info for usage).
Categories: Web Search & Scraping · Data Extraction · Auth: API Key · Actions: 5
Authentication
API Key Authentication
Authenticate using your Scrape.do API keyRequired Credentials
Available Actions
scrape — Perform basic web scraping without JavaScript rendering. Ideal for static websites and APIs.
scrape — Perform basic web scraping without JavaScript rendering. Ideal for static websites and APIs.
Parameters
string
required
URL to scrape
string
HTTP method (GET, POST, PUT, DELETE, HEAD) (Default:
GET)string
Request body for POST/PUT requests
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
boolean
Let Scrape.do add default headers
boolean
Forward extra upstream headers
boolean
Forward client headers to target
string
Cookies to send (JSON string or header)
boolean
Block images, CSS, fonts to speed up loading
boolean
Block advertisements
string
Output format (‘raw’ or ‘markdown’)
boolean
Return the origin response body with no parsing
Response
scrape_with_js — Scrape JavaScript-rendered pages using headless browser. Essential for SPAs and dynamic content.
scrape_with_js — Scrape JavaScript-rendered pages using headless browser. Essential for SPAs and dynamic content.
Parameters
string
required
URL to scrape
string
HTTP method (GET, POST, PUT, DELETE, HEAD) (Default:
GET)string
Request body for POST/PUT requests
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
boolean
Let Scrape.do add default headers
boolean
Forward extra upstream headers
boolean
Forward client headers to target
string
Cookies to send (JSON string or header)
boolean
Block images, CSS, fonts to speed up loading
boolean
Block advertisements
string
Output format (‘raw’ or ‘markdown’)
string
Wait condition: ‘domcontentloaded’, ‘networkidle0’, ‘networkidle2’, ‘load’
string
CSS selector to wait for before capturing
integer
Additional wait time in ms
integer
Browser viewport width (Default:
1920)integer
Browser viewport height (Default:
1080)string
JSON-encoded Play-with-Browser action list
Response
take_screenshot — Capture webpage screenshots: viewport, full-page, or element-specific.
take_screenshot — Capture webpage screenshots: viewport, full-page, or element-specific.
Parameters
string
required
URL to capture
boolean
Capture full page instead of viewport (Default:
false)string
CSS selector for element-specific screenshot
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
integer
Viewport width (Default:
1920)integer
Viewport height (Default:
1080)string
Wait condition for render completion
string
CSS selector to wait for before capturing
integer
Additional wait time in ms
boolean
Block advertisements
boolean
Let Scrape.do add default headers
string
Cookies to send
Response
scrape_to_markdown — Scrape web pages and convert content to clean, readable markdown format.
scrape_to_markdown — Scrape web pages and convert content to clean, readable markdown format.
Parameters
string
required
URL to scrape
boolean
Enable JavaScript rendering (Default:
false)string
HTTP method (Default:
GET)string
Request body for POST/PUT
boolean
Use residential & mobile proxy network
string
Country code for proxy location (e.g. ‘us’, ‘uk’, ‘de’)
string
Regional proxy location: ‘europe’, ‘asia’, ‘africa’, ‘oceania’, ‘northamerica’, ‘southamerica’
integer
Sticky session ID (0-1000000) for IP persistence
string
Device emulation (‘desktop’, ‘mobile’, ‘tablet’)
integer
Request timeout in ms (5000-120000)
integer
Retry timeout in ms (5000-55000)
boolean
Disable automatic retry on failure
boolean
Disable following redirects
boolean
Block images, CSS, fonts to speed up loading
boolean
Block advertisements
boolean
Let Scrape.do inject default headers
string
Cookies to send
string
JSON-encoded Play-with-Browser script
Response
get_usage_stats — Get API usage statistics and remaining credits for your Scrape.do account.
get_usage_stats — Get API usage statistics and remaining credits for your Scrape.do account.
Response
Limits & Quotas
- Each scrape action exposes 20+ optional knobs (proxy routing,
geo-targeting, device emulation, cookies, headers, wait
conditions, viewport). All map to Scrape.do’s camelCase query
string keys via a single
_PARAM_MAPtranslation table. take_screenshotis mutually-exclusive between viewport / full-page / element modes — the tool validates thatfull_pageandselectoraren’t both set.- Output shape varies per upstream response:
- JSON →
payload: dict - text/html/markdown →
data: strwithis_binary=False - image/* →
data: <base64>withis_binary=True
- JSON →
- 180s timeout for scrape operations (matches legacy); 30s for the usage-stats endpoint.