> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulex.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape.do Integration for AI Agents & Workflows

> Enterprise-grade web scraping API with JavaScript rendering, proxy rotation, geo-targeting, and screenshot capture.

{/* Self-hosted logo — build-time vendored from modulex.json / Iconify (see scripts/lib/logo.js). */}

<img className="block dark:hidden" src="https://mintcdn.com/modulexaillc/df8MOr-hXotYLTh5/logos/scrape-do-light.svg?fit=max&auto=format&n=df8MOr-hXotYLTh5&q=85&s=1e023675893db93f233872916070b074" alt="Scrape.do logo" width="72" height="72" data-path="logos/scrape-do-light.svg" />

<img className="hidden dark:block" src="https://mintcdn.com/modulexaillc/df8MOr-hXotYLTh5/logos/scrape-do-dark.svg?fit=max&auto=format&n=df8MOr-hXotYLTh5&q=85&s=993c79b3f5635db83fd868c5549fd418" alt="Scrape.do logo" width="72" height="72" data-path="logos/scrape-do-dark.svg" />

## Overview

Add **Scrape.do** to any ModuleX agent or workflow. Enterprise web-scraping API integration: basic HTTP, JS-rendered browser, screenshots, markdown conversion, and credit-usage stats. All five actions hit `api.scrape.do` (or `/info` for usage).

<Info>
  **Categories**: Web Search & Scraping · Data Extraction · **Auth**: API Key · **Actions**: 5
</Info>

## Authentication

### API Key Authentication

Authenticate using your Scrape.do API key

#### Required Credentials

| Field             | Description                               | Required | Format                             |
| ----------------- | ----------------------------------------- | -------- | ---------------------------------- |
| Scrape.do API Key | Your Scrape.do API key for authentication | Yes      | `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` |

## Available Actions

<AccordionGroup>
  <Accordion title="scrape — Perform basic web scraping without JavaScript rendering. Ideal for static websites and APIs.">
    ### Parameters

    <ResponseField name="url" type="string" required>
      URL to scrape
    </ResponseField>

    <ResponseField name="method" type="string">
      HTTP method (GET, POST, PUT, DELETE, HEAD) (Default: `GET`)
    </ResponseField>

    <ResponseField name="body" type="string">
      Request body for POST/PUT requests
    </ResponseField>

    <ResponseField name="super_proxy" type="boolean">
      Use residential & mobile proxy network
    </ResponseField>

    <ResponseField name="geo_code" type="string">
      Country code for proxy location (e.g. 'us', 'uk', 'de')
    </ResponseField>

    <ResponseField name="regional_geo_code" type="string">
      Regional proxy location: 'europe', 'asia', 'africa', 'oceania', 'northamerica', 'southamerica'
    </ResponseField>

    <ResponseField name="session_id" type="integer">
      Sticky session ID (0-1000000) for IP persistence
    </ResponseField>

    <ResponseField name="device" type="string">
      Device emulation ('desktop', 'mobile', 'tablet')
    </ResponseField>

    <ResponseField name="timeout" type="integer">
      Request timeout in ms (5000-120000)
    </ResponseField>

    <ResponseField name="retry_timeout" type="integer">
      Retry timeout in ms (5000-55000)
    </ResponseField>

    <ResponseField name="disable_retry" type="boolean">
      Disable automatic retry on failure
    </ResponseField>

    <ResponseField name="disable_redirection" type="boolean">
      Disable following redirects
    </ResponseField>

    <ResponseField name="custom_headers" type="boolean">
      Let Scrape.do add default headers
    </ResponseField>

    <ResponseField name="extra_headers" type="boolean">
      Forward extra upstream headers
    </ResponseField>

    <ResponseField name="forward_headers" type="boolean">
      Forward client headers to target
    </ResponseField>

    <ResponseField name="set_cookies" type="string">
      Cookies to send (JSON string or header)
    </ResponseField>

    <ResponseField name="block_resources" type="boolean">
      Block images, CSS, fonts to speed up loading
    </ResponseField>

    <ResponseField name="block_ads" type="boolean">
      Block advertisements
    </ResponseField>

    <ResponseField name="output" type="string">
      Output format ('raw' or 'markdown')
    </ResponseField>

    <ResponseField name="transparent_response" type="boolean">
      Return the origin response body with no parsing
    </ResponseField>

    ### Response

    ```json theme={null}
    {
      "additionalProperties": false,
      "properties": {
        "success": {
          "title": "Success",
          "type": "boolean"
        },
        "error": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Error"
        },
        "status_code": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Status Code"
        },
        "content_type": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Content Type"
        },
        "data": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Data"
        },
        "is_binary": {
          "default": false,
          "title": "Is Binary",
          "type": "boolean"
        },
        "payload": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Payload"
        }
      },
      "required": [
        "success"
      ],
      "title": "ScrapeOutput",
      "type": "object"
    }
    ```
  </Accordion>

  <Accordion title="scrape_with_js — Scrape JavaScript-rendered pages using headless browser. Essential for SPAs and dynamic content.">
    ### Parameters

    <ResponseField name="url" type="string" required>
      URL to scrape
    </ResponseField>

    <ResponseField name="method" type="string">
      HTTP method (GET, POST, PUT, DELETE, HEAD) (Default: `GET`)
    </ResponseField>

    <ResponseField name="body" type="string">
      Request body for POST/PUT requests
    </ResponseField>

    <ResponseField name="super_proxy" type="boolean">
      Use residential & mobile proxy network
    </ResponseField>

    <ResponseField name="geo_code" type="string">
      Country code for proxy location (e.g. 'us', 'uk', 'de')
    </ResponseField>

    <ResponseField name="regional_geo_code" type="string">
      Regional proxy location: 'europe', 'asia', 'africa', 'oceania', 'northamerica', 'southamerica'
    </ResponseField>

    <ResponseField name="session_id" type="integer">
      Sticky session ID (0-1000000) for IP persistence
    </ResponseField>

    <ResponseField name="device" type="string">
      Device emulation ('desktop', 'mobile', 'tablet')
    </ResponseField>

    <ResponseField name="timeout" type="integer">
      Request timeout in ms (5000-120000)
    </ResponseField>

    <ResponseField name="retry_timeout" type="integer">
      Retry timeout in ms (5000-55000)
    </ResponseField>

    <ResponseField name="disable_retry" type="boolean">
      Disable automatic retry on failure
    </ResponseField>

    <ResponseField name="disable_redirection" type="boolean">
      Disable following redirects
    </ResponseField>

    <ResponseField name="custom_headers" type="boolean">
      Let Scrape.do add default headers
    </ResponseField>

    <ResponseField name="extra_headers" type="boolean">
      Forward extra upstream headers
    </ResponseField>

    <ResponseField name="forward_headers" type="boolean">
      Forward client headers to target
    </ResponseField>

    <ResponseField name="set_cookies" type="string">
      Cookies to send (JSON string or header)
    </ResponseField>

    <ResponseField name="block_resources" type="boolean">
      Block images, CSS, fonts to speed up loading
    </ResponseField>

    <ResponseField name="block_ads" type="boolean">
      Block advertisements
    </ResponseField>

    <ResponseField name="output" type="string">
      Output format ('raw' or 'markdown')
    </ResponseField>

    <ResponseField name="wait_until" type="string">
      Wait condition: 'domcontentloaded', 'networkidle0', 'networkidle2', 'load'
    </ResponseField>

    <ResponseField name="wait_selector" type="string">
      CSS selector to wait for before capturing
    </ResponseField>

    <ResponseField name="custom_wait" type="integer">
      Additional wait time in ms
    </ResponseField>

    <ResponseField name="width" type="integer">
      Browser viewport width (Default: `1920`)
    </ResponseField>

    <ResponseField name="height" type="integer">
      Browser viewport height (Default: `1080`)
    </ResponseField>

    <ResponseField name="play_with_browser" type="string">
      JSON-encoded Play-with-Browser action list
    </ResponseField>

    ### Response

    ```json theme={null}
    {
      "additionalProperties": false,
      "properties": {
        "success": {
          "title": "Success",
          "type": "boolean"
        },
        "error": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Error"
        },
        "status_code": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Status Code"
        },
        "content_type": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Content Type"
        },
        "data": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Data"
        },
        "is_binary": {
          "default": false,
          "title": "Is Binary",
          "type": "boolean"
        },
        "payload": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Payload"
        }
      },
      "required": [
        "success"
      ],
      "title": "ScrapeWithJsOutput",
      "type": "object"
    }
    ```
  </Accordion>

  <Accordion title="take_screenshot — Capture webpage screenshots: viewport, full-page, or element-specific.">
    ### Parameters

    <ResponseField name="url" type="string" required>
      URL to capture
    </ResponseField>

    <ResponseField name="full_page" type="boolean">
      Capture full page instead of viewport (Default: `false`)
    </ResponseField>

    <ResponseField name="selector" type="string">
      CSS selector for element-specific screenshot
    </ResponseField>

    <ResponseField name="super_proxy" type="boolean">
      Use residential & mobile proxy network
    </ResponseField>

    <ResponseField name="geo_code" type="string">
      Country code for proxy location (e.g. 'us', 'uk', 'de')
    </ResponseField>

    <ResponseField name="regional_geo_code" type="string">
      Regional proxy location: 'europe', 'asia', 'africa', 'oceania', 'northamerica', 'southamerica'
    </ResponseField>

    <ResponseField name="session_id" type="integer">
      Sticky session ID (0-1000000) for IP persistence
    </ResponseField>

    <ResponseField name="device" type="string">
      Device emulation ('desktop', 'mobile', 'tablet')
    </ResponseField>

    <ResponseField name="timeout" type="integer">
      Request timeout in ms (5000-120000)
    </ResponseField>

    <ResponseField name="retry_timeout" type="integer">
      Retry timeout in ms (5000-55000)
    </ResponseField>

    <ResponseField name="disable_retry" type="boolean">
      Disable automatic retry on failure
    </ResponseField>

    <ResponseField name="disable_redirection" type="boolean">
      Disable following redirects
    </ResponseField>

    <ResponseField name="width" type="integer">
      Viewport width (Default: `1920`)
    </ResponseField>

    <ResponseField name="height" type="integer">
      Viewport height (Default: `1080`)
    </ResponseField>

    <ResponseField name="wait_until" type="string">
      Wait condition for render completion
    </ResponseField>

    <ResponseField name="wait_selector" type="string">
      CSS selector to wait for before capturing
    </ResponseField>

    <ResponseField name="custom_wait" type="integer">
      Additional wait time in ms
    </ResponseField>

    <ResponseField name="block_ads" type="boolean">
      Block advertisements
    </ResponseField>

    <ResponseField name="custom_headers" type="boolean">
      Let Scrape.do add default headers
    </ResponseField>

    <ResponseField name="set_cookies" type="string">
      Cookies to send
    </ResponseField>

    ### Response

    ```json theme={null}
    {
      "additionalProperties": false,
      "properties": {
        "success": {
          "title": "Success",
          "type": "boolean"
        },
        "error": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Error"
        },
        "status_code": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Status Code"
        },
        "content_type": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Content Type"
        },
        "data": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Data"
        },
        "is_binary": {
          "default": false,
          "title": "Is Binary",
          "type": "boolean"
        },
        "payload": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Payload"
        }
      },
      "required": [
        "success"
      ],
      "title": "TakeScreenshotOutput",
      "type": "object"
    }
    ```
  </Accordion>

  <Accordion title="scrape_to_markdown — Scrape web pages and convert content to clean, readable markdown format.">
    ### Parameters

    <ResponseField name="url" type="string" required>
      URL to scrape
    </ResponseField>

    <ResponseField name="render" type="boolean">
      Enable JavaScript rendering (Default: `false`)
    </ResponseField>

    <ResponseField name="method" type="string">
      HTTP method (Default: `GET`)
    </ResponseField>

    <ResponseField name="body" type="string">
      Request body for POST/PUT
    </ResponseField>

    <ResponseField name="super_proxy" type="boolean">
      Use residential & mobile proxy network
    </ResponseField>

    <ResponseField name="geo_code" type="string">
      Country code for proxy location (e.g. 'us', 'uk', 'de')
    </ResponseField>

    <ResponseField name="regional_geo_code" type="string">
      Regional proxy location: 'europe', 'asia', 'africa', 'oceania', 'northamerica', 'southamerica'
    </ResponseField>

    <ResponseField name="session_id" type="integer">
      Sticky session ID (0-1000000) for IP persistence
    </ResponseField>

    <ResponseField name="device" type="string">
      Device emulation ('desktop', 'mobile', 'tablet')
    </ResponseField>

    <ResponseField name="timeout" type="integer">
      Request timeout in ms (5000-120000)
    </ResponseField>

    <ResponseField name="retry_timeout" type="integer">
      Retry timeout in ms (5000-55000)
    </ResponseField>

    <ResponseField name="disable_retry" type="boolean">
      Disable automatic retry on failure
    </ResponseField>

    <ResponseField name="disable_redirection" type="boolean">
      Disable following redirects
    </ResponseField>

    <ResponseField name="block_resources" type="boolean">
      Block images, CSS, fonts to speed up loading
    </ResponseField>

    <ResponseField name="block_ads" type="boolean">
      Block advertisements
    </ResponseField>

    <ResponseField name="custom_headers" type="boolean">
      Let Scrape.do inject default headers
    </ResponseField>

    <ResponseField name="set_cookies" type="string">
      Cookies to send
    </ResponseField>

    <ResponseField name="play_with_browser" type="string">
      JSON-encoded Play-with-Browser script
    </ResponseField>

    ### Response

    ```json theme={null}
    {
      "additionalProperties": false,
      "properties": {
        "success": {
          "title": "Success",
          "type": "boolean"
        },
        "error": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Error"
        },
        "status_code": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Status Code"
        },
        "markdown": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Markdown"
        },
        "raw": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Raw"
        }
      },
      "required": [
        "success"
      ],
      "title": "ScrapeToMarkdownOutput",
      "type": "object"
    }
    ```
  </Accordion>

  <Accordion title="get_usage_stats — Get API usage statistics and remaining credits for your Scrape.do account.">
    ### Response

    ```json theme={null}
    {
      "additionalProperties": false,
      "properties": {
        "success": {
          "title": "Success",
          "type": "boolean"
        },
        "error": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Error"
        },
        "status_code": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Status Code"
        },
        "stats": {
          "anyOf": [
            {
              "additionalProperties": true,
              "type": "object"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Stats"
        }
      },
      "required": [
        "success"
      ],
      "title": "GetUsageStatsOutput",
      "type": "object"
    }
    ```
  </Accordion>
</AccordionGroup>

## Limits & Quotas

* Each scrape action exposes 20+ optional knobs (proxy routing,
  geo-targeting, device emulation, cookies, headers, wait
  conditions, viewport). All map to Scrape.do's camelCase query
  string keys via a single `_PARAM_MAP` translation table.
* `take_screenshot` is mutually-exclusive between viewport / full-page
  / element modes — the tool validates that `full_page` and
  `selector` aren't both set.
* Output shape varies per upstream response:
  * JSON → `payload: dict`
  * text/html/markdown → `data: str` with `is_binary=False`
  * image/\* → `data: <base64>` with `is_binary=True`
* 180s timeout for scrape operations (matches legacy); 30s for the
  usage-stats endpoint.

## Related integrations

<CardGroup cols={3}>
  <Card title="Ahrefs" href="/integrations/tools/ahrefs" />

  <Card title="Airweave" href="/integrations/tools/airweave" />

  <Card title="Apify" href="/integrations/tools/apify" />
</CardGroup>

## Links

* [Scrape.do](https://scrape.do)