> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulex.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM node

> Call a language model inside a workflow with templated prompts, {{node_id.field}} references, and optional structured JSON output. Full LLMNodeConfig reference, streaming behavior, managed vs BYOK billing, inputs/outputs, and errors.

export const MediaEmbed = ({id, type = 'screenshot', caption = '', ext, ratio = '16 / 9'}) => {
  const isVideo = type === 'video' || type === 'app_video';
  const resolvedExt = ext || (isVideo ? 'mp4' : type === 'screenshot' ? 'webp' : 'svg');
  const src = 'https://media.modulex.dev/' + id + '.' + resolvedExt;
  const [status, setStatus] = useState('loading');
  const [isDev, setIsDev] = useState(false);
  const [inView, setInView] = useState(false);
  const boxRef = useRef(null);
  useEffect(() => {
    if (typeof window === 'undefined') return;
    const h = window.location.hostname;
    setIsDev(h === 'localhost' || h === '127.0.0.1' || h.endsWith('.mintlify.app'));
  }, []);
  useEffect(() => {
    if (inView) return;
    if (typeof IntersectionObserver === 'undefined') {
      setInView(true);
      return;
    }
    const el = boxRef.current;
    if (!el) return;
    const io = new IntersectionObserver(entries => {
      if (entries.some(e => e.isIntersecting)) {
        setInView(true);
        io.disconnect();
      }
    }, {
      rootMargin: '300px'
    });
    io.observe(el);
    return () => io.disconnect();
  }, [inView]);
  if (status === 'missing') {
    if (!isDev) return null;
    return <div style={{
      display: 'flex',
      flexDirection: 'column',
      justifyContent: 'center',
      gap: '0.4rem',
      padding: '1rem 1.25rem',
      margin: '1.25rem 0',
      width: '100%',
      aspectRatio: ratio,
      boxSizing: 'border-box',
      border: '1px dashed rgba(128,128,128,0.45)',
      borderRadius: '0.75rem',
      background: 'rgba(128,128,128,0.06)',
      color: 'currentColor',
      fontSize: '0.85rem',
      lineHeight: 1.45
    }}>
        <div style={{
      display: 'flex',
      alignItems: 'center',
      gap: '0.5rem',
      opacity: 0.75
    }}>
          <span aria-hidden="true">🎬</span>
          <code style={{
      fontSize: '0.75rem'
    }}>{id}</code>
          <span style={{
      fontSize: '0.65rem',
      textTransform: 'uppercase',
      letterSpacing: '0.04em',
      padding: '0.1rem 0.4rem',
      borderRadius: '0.4rem',
      background: 'rgba(128,128,128,0.18)'
    }}>
            {type}
          </span>
        </div>
        <div style={{
      opacity: 0.9
    }}>{caption || 'Media not uploaded yet.'}</div>
        <div style={{
      fontSize: '0.7rem',
      opacity: 0.5
    }}>
          Upload to R2 as <code>{id}.{resolvedExt}</code> — preview only, hidden in production.
        </div>
      </div>;
  }
  const mediaStyle = {
    display: status === 'loaded' ? 'block' : 'none',
    width: '100%',
    height: 'auto',
    borderRadius: '0.75rem'
  };
  const media = isVideo ? <video src={inView ? src : undefined} autoPlay loop muted playsInline preload="metadata" onLoadedData={() => setStatus('loaded')} onError={() => setStatus('missing')} style={mediaStyle} /> : <img src={inView ? src : undefined} alt={caption} onLoad={() => setStatus('loaded')} onError={() => setStatus('missing')} style={mediaStyle} />;
  return <figure style={{
    margin: '1.25rem 0'
  }}>
      <div ref={boxRef} style={status === 'loaded' ? {
    width: '100%'
  } : {
    width: '100%',
    aspectRatio: ratio,
    borderRadius: '0.75rem',
    background: 'rgba(128,128,128,0.06)'
  }}>
        {media}
      </div>
      {status === 'loaded' && caption ? <figcaption style={{
    marginTop: '0.5rem',
    textAlign: 'center',
    fontSize: '0.85rem',
    opacity: 0.7
  }}>
          {caption}
        </figcaption> : null}
    </figure>;
};

The `llm` node calls a single language model with a system prompt and a user prompt, then writes the model's response into run state under the node's own id. It is the simplest way to add generation, summarization, extraction, classification, or rewriting to a workflow. For a step that can also call integration tools and loop, use the [agent node](/workflow-builder/nodes/agent) instead.

Each `llm` node makes exactly one model call per run (no tool-calling loop). To pass data into the prompt, use `{{node_id.field}}` references — the same reference system documented in [variables & references](/workflow-builder/variables-and-references) and [the workflow engine](/concepts/workflow-engine).

<MediaEmbed id="MX-MEDIA-3070" type="screenshot" caption={"the LLM node configuration panel in the workflow builder"} />

## What the node does

When the engine compiles your workflow, the `llm` node is turned into a single async step ([workflow engine](/concepts/workflow-engine)). At run time the node:

1. Resolves `system_prompt` and `user_prompt` against the current run state, replacing every `{{...}}` token with its resolved value (strings are substituted inline; objects and arrays are JSON-serialized into the string). A reference that is the entire string keeps its native type; an unresolved reference is left intact.
2. Builds a two-message conversation — a system message (only if `system_prompt` is set) followed by a human message containing the resolved user prompt.
3. Calls the configured model once.
4. If `structured_output_schema` is set, requests a structured (JSON) response constrained to that schema; otherwise returns the model's text.
5. Records token usage for billing (managed usage only — see [credit impact](#credit-impact)).
6. Writes the result into run state under the node's `id`.

If `user_prompt` is omitted, the node falls back to the run-state field named `input` (the run's top-level input). If that is also empty, the user message is an empty string.

<Note>
  The output of an LLM node is always stored in run state under the node's `id` — for example a node with `id: "summary_1"` writes to `{{summary_1}}`. The legacy `output_key` field is deprecated and ignored for routing; do not rely on it. See [inputs & outputs](#inputs-and-outputs).
</Note>

## Configuration (`LLMNodeConfig`)

These fields live on the node's `llm_config`. In the builder you set them through the detail panel; over the API they appear inside the node definition (the builder also accepts a wrapped `{config: {...}}` form, which the backend normalizes to `llm_config`).

<ParamField path="llm" type="LLMConfig object" required>
  The model to call. Required. See [the `llm` object](#the-llm-object) below for its fields. This selects both the provider (managed or BYOK) and the specific model.
</ParamField>

<ParamField path="system_prompt" type="string">
  The system message that sets the model's role and instructions. Optional. Supports `{{node_id.field}}` and `{{input}}` references. If omitted, no system message is sent.
</ParamField>

<ParamField path="user_prompt" type="string">
  The human message — the actual task or question. Optional but recommended. Supports `{{node_id.field}}` and `{{input}}` references. If omitted, the node uses the run-state `input` field; if that is empty, the user message is an empty string.
</ParamField>

<ParamField path="structured_output_schema" type="object (JSON Schema)">
  A JSON Schema describing the shape you want the model to return. Optional. When set, the node returns a parsed object matching the schema instead of free text. ModuleX automatically fills in a `title` and `description` at the top of the schema if you omit them (LangChain requires these for tool/function calling). See [structured output](#structured-output).
</ParamField>

<ParamField path="structured_output_strict" type="boolean" default="false">
  Only meaningful when `structured_output_schema` is set. When `true`, the provider enforces strict schema validation: a response that does not exactly match the schema fails instead of being loosely coerced. When `false` (the default), responses are coerced more leniently. Opt in per node when you want the model to fail fast on schema drift.
</ParamField>

<Accordion title="Deprecated fields — do not use in new workflows">
  These fields exist on `LLMNodeConfig` for backward compatibility and are normalized away or ignored. Author new workflows without them.

  <ParamField path="prompt_template" type="string" deprecated>
    Deprecated. Use `user_prompt` instead. If `prompt_template` is set and `user_prompt` is not, the backend copies `prompt_template` into `user_prompt` automatically.
  </ParamField>

  <ParamField path="input_keys" type="string[]" deprecated>
    Deprecated. Use `{{node_id.field}}` references in your prompts instead of listing state keys.
  </ParamField>

  <ParamField path="output_key" type="string" deprecated>
    Deprecated. Output is always stored under the node `id`. Setting this to the literal value `messages` triggers a legacy messages-state code path used by the simple chat runtime, not by ordinary workflow nodes — avoid it.
  </ParamField>
</Accordion>

### The `llm` object

The `llm` field is an `LLMConfig`. It is required and identifies the provider, the model, and (optionally) which stored credential to use.

<ParamField path="llm.integration_name" type="string" required>
  The provider integration on the wire. Required. Use `modulexai` for ModuleX-managed models (billed in credits). For BYOK, use the provider integration name — for example `anthropic`, `openai`, `gemini`, or `xai`. See [LLM providers](/integrations/llm-providers/overview) and [managed vs BYOK](#managed-vs-byok).
</ParamField>

<ParamField path="llm.provider_id" type="string" required>
  The underlying provider id. Required. For example `anthropic`, `openai`, `gemini`, `xai`, or — for managed routing — `openrouter`. Each model in a provider catalog declares its own `provider_id`; match it for the model you pick.
</ParamField>

<ParamField path="llm.model_id" type="string" required>
  The standardized model id, for example `claude-sonnet-4.6`, `claude-haiku-4.5`, or `gpt-5.4-mini`. Required. ModuleX maps this id to the provider's served model. Models marked deprecated or in maintenance are automatically routed to their replacement for managed integrations. Browse available ids on each [provider page](/integrations/llm-providers/overview).
</ParamField>

<ParamField path="llm.temperature" type="number" default="0.4">
  Sampling temperature passed to the model. Optional; defaults to `0.4`. Lower values make output more deterministic; higher values make it more varied.
</ParamField>

<ParamField path="llm.credential_id" type="string">
  A specific stored credential to use for this call. Optional. If omitted, ModuleX resolves a credential for the integration in the current organization. Required in practice for BYOK providers (you must have connected your own key). See [managing credentials](/integrations/managing-credentials).
</ParamField>

### Retry configuration

The `llm` node is retry-wrapped. You can attach a `retry_config` to the node definition itself (not inside `llm_config`) to control how failed calls are retried. If you omit it, the engine applies its default retry policy (2 retries). Errors are only retried when their type is in `retry_on_error_types`.

<ParamField path="retry_config.max_attempts" type="integer" default="3">
  Total attempts including the first. Range 1–10. `1` means no retry; `3` means the initial call plus 2 retries.
</ParamField>

<ParamField path="retry_config.initial_interval" type="number" default="1.0">
  Seconds to wait before the first retry. Range 0.1–60.
</ParamField>

<ParamField path="retry_config.backoff_factor" type="number" default="2.0">
  Multiplier applied to the delay between successive retries (exponential backoff). Range 1–5.
</ParamField>

<ParamField path="retry_config.retry_on_error_types" type="string[]" default="[&#x22;TimeoutError&#x22;, &#x22;ConnectionError&#x22;, &#x22;HTTPError&#x22;]">
  Exception type names that trigger a retry. Errors not in this list fail immediately. A credit-exhaustion stop is not retried.
</ParamField>

See [error handling & retries](/workflow-builder/error-handling-retries) for how retries surface in the run stream.

## Inputs and outputs

### Inputs

The LLM node has no fixed input fields. It reads whatever you reference in `system_prompt` and `user_prompt` from run state:

* `{{node_id}}` — the entire output of an upstream node (typed value: string, object, or array).
* `{{node_id.field}}` — a nested value via dot/bracket path, for example `{{extract_1.results[0].title}}`. An out-of-range index or missing key resolves to nothing and the reference is left intact in the string.
* `{{input}}` — the run's top-level input field.

A reference that is the entire string keeps its native type; a reference embedded inside other text is string-substituted (objects/arrays are JSON-encoded). See [variables & references](/workflow-builder/variables-and-references) for the full resolution model.

### Outputs

The node writes one value into run state under its `id`:

<ResponseField name="{node_id}" type="string | object">
  The model's response.

  <Expandable title="Shape by mode">
    <ResponseField name="text mode" type="string">
      When `structured_output_schema` is not set, the value is the model's text response.
    </ResponseField>

    <ResponseField name="structured mode" type="object">
      When `structured_output_schema` is set, the value is a parsed object conforming to your schema. If the provider's structured call fails, the node falls back to parsing JSON out of the raw text response; if no JSON can be extracted, it stores the raw string instead.
    </ResponseField>
  </Expandable>
</ResponseField>

Downstream nodes read this with `{{node_id}}` (whole value) or `{{node_id.field}}` (a field of a structured object).

## Streaming

How an LLM node's output streams depends on the run surface:

* **Normal workflow nodes** stream at the **node level** over [SSE](/realtime/sse-streaming). When the node finishes, the engine publishes a `node_update` event carrying the node's output. On the wire this event is flat and uses the keys `node` and `output` — for example `{type, node, output}` — not the typed model field names. The run then ends with a `done` event.
* **Token-by-token streaming** is only active on the simple LLM-chat runtime (the messages-state path), not on ordinary workflow `llm` nodes. In a standard workflow you receive the node's result as a single `node_update`, not incremental tokens.

So for a workflow `llm` node, expect one `node_update` for the node, then the next node's events. See [SSE run streaming](/realtime/sse-streaming) for the full event taxonomy and frame format.

<Note>
  The realtime wire dicts differ from the typed event models: the `node_update` frame uses `node` and `output`, and `done` carries only a `message`. Parse each SSE frame as JSON and switch on its `type` field — there is no SSE `event:` line. Details on [SSE run streaming](/realtime/sse-streaming).
</Note>

## Structured output

Set `structured_output_schema` to a JSON Schema to make the node return a parsed object instead of text. ModuleX uses the provider's native structured-output mechanism (tool calling on Anthropic, function calling on OpenAI, and the appropriate method elsewhere) and lets the provider pick the best method for the model you chose.

Behavior to know:

* If your schema omits a top-level `title` or `description`, ModuleX adds them automatically (LangChain requires them to build the tool definition the model sees).
* With `structured_output_strict: false` (default), responses are coerced leniently. With `structured_output_strict: true`, a response that does not exactly match the schema fails instead of being coerced.
* If the structured call fails for any reason, the node falls back to invoking the model normally and extracting JSON from the raw text. If no JSON can be extracted, it stores the raw string — so always validate downstream when correctness matters, or add a [guardrails node](/workflow-builder/nodes/guardrails) to enforce the shape.

<CodeGroup>
  ```json structured_output_schema theme={null}
  {
    "type": "object",
    "properties": {
      "sentiment": {
        "type": "string",
        "enum": ["positive", "neutral", "negative"]
      },
      "summary": { "type": "string" },
      "topics": {
        "type": "array",
        "items": { "type": "string" }
      }
    },
    "required": ["sentiment", "summary"]
  }
  ```
</CodeGroup>

A downstream node can then read `{{classify_1.sentiment}}` or `{{classify_1.topics}}`.

## Managed vs BYOK

The `llm.integration_name` you choose decides who runs the model and how it is billed:

<CardGroup cols={2}>
  <Card title="Managed (modulexai)" icon="server">
    Set `integration_name` to `modulexai`. The call runs through ModuleX-provisioned providers and is **billed in credits** by input/output tokens. No provider key of your own is required. See [ModuleX-managed models](/integrations/llm-providers/modulexai).
  </Card>

  <Card title="BYOK (your own key)" icon="key">
    Set `integration_name` to the provider (for example `anthropic`, `openai`, `gemini`, `xai`) and connect your own credential. Usage is billed directly by that provider with **no ModuleX markup** and is **not** charged in credits (token usage is recorded for analytics only). See [LLM providers](/integrations/llm-providers/overview) and [managing credentials](/integrations/managing-credentials).
  </Card>
</CardGroup>

Both modes use the same `LLMNodeConfig`; only `integration_name` (and the credential) changes.

## Credit impact

There is no fixed per-node credit charge for an LLM node. Charging is metered per model call:

* **Managed (`modulexai`)** — each call records token usage and charges credits for input and output tokens. ModuleX measures usage in [credits](/billing/credits) (the managed-usage billing unit).
* **BYOK** — token usage is recorded for analytics, but **no credits are charged**; you pay your provider directly.

Workflow runs go through the [usage gate](/billing/usage-gating), which admits the run before it starts. The gate is best-effort once a run is executing: a billing problem mid-run surfaces as a node error rather than crashing the run silently. If an organization runs out of credits mid-run, the node's error event carries a stable reason token (`credit_exhausted`) so clients can detect a budget stop deterministically. See [usage gating & limits](/billing/usage-gating) and [credits & metering](/billing/credits).

<Note>
  Workflow run, [Composer](/workflow-builder/composer), [Assistant](/assistant/overview), and managed-knowledge surfaces are gated by the billing admission gate, which can return a `DenialEnvelope` as **402 / 403 / 429**. The flat envelope shape is `{code, layer, key, current, limit, reason}`. See [errors & status codes](/api-reference/errors) and [usage gating](/billing/usage-gating).
</Note>

## Errors

The LLM node surfaces failures through the run's node-error event after retries are exhausted. Common cases:

<Accordion title="Missing or invalid llm_config">
  If an `llm` node has no `llm_config`, compilation fails with a configuration error (`LLM node <id> missing llm_config`). Ensure the `llm` object with `integration_name`, `provider_id`, and `model_id` is set.
</Accordion>

<Accordion title="Provider / network errors">
  Timeouts, connection failures, and HTTP errors from the provider are retried per `retry_config` (defaults: `TimeoutError`, `ConnectionError`, `HTTPError`). When retries are exhausted, the node emits a `node_error` event with `error_type`, `error_message`, the attempt count, and `recoverable: false`, then the run fails. See [error handling & retries](/workflow-builder/error-handling-retries).
</Accordion>

<Accordion title="Credit exhaustion mid-run (managed)">
  If a managed call cannot be billed because credits are exhausted, the node's `node_error` event includes a stable `reason` token, `credit_exhausted`, so a client can match it without parsing free text. Resolve by topping up the [wallet](/billing/wallet) or upgrading your [plan](/billing/plans).
</Accordion>

<Accordion title="Structured output that cannot be parsed">
  If structured output is requested but the provider call fails and no JSON can be extracted from the fallback text, the node stores the raw string instead of an object. Downstream `{{node_id.field}}` references then resolve to nothing. Add a [guardrails node](/workflow-builder/nodes/guardrails) or validate before relying on the shape.
</Accordion>

For the full taxonomy of error-envelope shapes and which surface emits each, see [errors & status codes](/api-reference/errors).

## Full example

A two-node workflow: an upstream node produces a customer message, and an `llm` node classifies it into structured fields.

The first tab is the LLM node definition as it appears inside a workflow. The remaining tabs run a deployed workflow that contains this node. Running a workflow is asynchronous: the run call returns immediately with run metadata (`status` is `running`), and you observe node output and the final result by streaming events. The node's structured object arrives on the `node_update` event for `classify_1`. See [SSE run streaming](/realtime/sse-streaming) and [run a workflow](/guides/run-a-workflow).

<CodeGroup>
  ```json LLM node definition theme={null}
  {
    "id": "classify_1",
    "type": "llm",
    "name": "Classify message",
    "x": 480,
    "y": 160,
    "retry_config": {
      "max_attempts": 3,
      "initial_interval": 1.0,
      "backoff_factor": 2.0
    },
    "llm_config": {
      "llm": {
        "integration_name": "modulexai",
        "provider_id": "openrouter",
        "model_id": "claude-sonnet-4.6",
        "temperature": 0.2
      },
      "system_prompt": "You are a support triage assistant. Classify the customer message precisely.",
      "user_prompt": "Classify this message and summarize it in one sentence:\n\n{{intake_1.message}}",
      "structured_output_schema": {
        "type": "object",
        "properties": {
          "sentiment": { "type": "string", "enum": ["positive", "neutral", "negative"] },
          "priority": { "type": "string", "enum": ["low", "medium", "high"] },
          "summary": { "type": "string" }
        },
        "required": ["sentiment", "priority", "summary"]
      },
      "structured_output_strict": true
    }
  }
  ```

  ```bash cURL theme={null}
  # 1. Start the run (POST /workflows/run). Returns run metadata immediately.
  curl https://api.modulex.dev/workflows/run \
    -H "Authorization: Bearer mx_live_xxxxxxxxxxxxxxxxxxxx" \
    -H "X-Organization-ID: org_4d2b18" \
    -H "Content-Type: application/json" \
    -d '{
          "workflow_id": "wf_7f3a9c",
          "input": {
            "intake_1": { "message": "My invoice double-charged me and I need this fixed today." }
          }
        }'
  # -> { "status": "running", "run_id": "run_8b21", "thread_id": "thr_44", ... }

  # 2. Stream events (SSE). Each frame is `data: <json>`; switch on the `type` field.
  #    The classify_1 result arrives on its node_update event.
  curl -N https://api.modulex.dev/workflows/listen/run_8b21 \
    -H "Authorization: Bearer mx_live_xxxxxxxxxxxxxxxxxxxx" \
    -H "X-Organization-ID: org_4d2b18"
  ```

  ```python Python theme={null}
  import asyncio
  from modulex import Modulex


  async def main():
      async with Modulex(
          api_key="mx_live_xxxxxxxxxxxxxxxxxxxx",
          organization_id="org_4d2b18",
      ) as client:
          # 1. Start the run (returns immediately with run metadata).
          run = await client.executions.run(
              workflow_id="wf_7f3a9c",
              input={
                  "intake_1": {
                      "message": "My invoice double-charged me and I need this fixed today.",
                  },
              },
          )

          # 2. Stream events; classify_1's structured object arrives on its node_update.
          #    SSEEvent carries the discriminator in `.event` and the payload in `.data`.
          async for event in client.executions.listen(run.run_id):
              if event.event == "node_update" and event.data.get("node") == "classify_1":
                  result = event.data["output"]["classify_1"]
                  print(result["sentiment"])  # -> "negative"
                  print(result["priority"])   # -> "high"


  asyncio.run(main())
  ```

  ```javascript JavaScript theme={null}
  import { Modulex } from "modulex-js";

  const client = new Modulex({
    apiKey: "mx_live_xxxxxxxxxxxxxxxxxxxx",
    organizationId: "org_4d2b18",
  });

  // 1. Start the run (returns immediately with run metadata).
  const run = await client.executions.run({
    workflowId: "wf_7f3a9c",
    input: {
      intake_1: {
        message: "My invoice double-charged me and I need this fixed today.",
      },
    },
  });

  // 2. Stream events; classify_1's structured object arrives on its node_update.
  //    Switch on `event.type`; the payload (node, output) lives in `event.data`.
  for await (const event of client.executions.listen(run.run_id)) {
    if (event.type === "node_update" && event.data.node === "classify_1") {
      const result = event.data.output.classify_1;
      console.log(result.sentiment); // -> "negative"
      console.log(result.priority);  // -> "high"
    }
  }
  ```
</CodeGroup>

Every request authenticates with `Authorization: Bearer mx_live_…` plus `X-Organization-ID`. See [authentication](/api-reference/authentication) and the [run-a-workflow guide](/guides/run-a-workflow). To stream the run instead of waiting for the final state, see [SSE run streaming](/realtime/sse-streaming) and [streaming & HITL in the SDKs](/sdks/streaming-hitl).

## Related

<CardGroup cols={2}>
  <Card title="Agent node" icon="robot" href="/workflow-builder/nodes/agent">
    When the step needs to call tools and loop, not just generate once.
  </Card>

  <Card title="Guardrails node" icon="shield-check" href="/workflow-builder/nodes/guardrails">
    Validate JSON shape, regex, and PII on an LLM node's output.
  </Card>

  <Card title="LLM providers" icon="plug" href="/integrations/llm-providers/overview">
    Managed and BYOK providers, model ids, and credentials.
  </Card>

  <Card title="Variables & references" icon="braces" href="/workflow-builder/variables-and-references">
    How `{{node_id.field}}` references resolve against run state.
  </Card>
</CardGroup>