LLM node

The llm node calls a single language model with a system prompt and a user prompt, then writes the model’s response into run state under the node’s own id. It is the simplest way to add generation, summarization, extraction, classification, or rewriting to a workflow. For a step that can also call integration tools and loop, use the agent node instead. Each llm node makes exactly one model call per run (no tool-calling loop). To pass data into the prompt, use {{node_id.field}} references — the same reference system documented in variables & references and the workflow engine.

What the node does

When the engine compiles your workflow, the llm node is turned into a single async step (workflow engine). At run time the node:

Resolves system_prompt and user_prompt against the current run state, replacing every {{...}} token with its resolved value (strings are substituted inline; objects and arrays are JSON-serialized into the string). A reference that is the entire string keeps its native type; an unresolved reference is left intact.
Builds a two-message conversation — a system message (only if system_prompt is set) followed by a human message containing the resolved user prompt.
Calls the configured model once.
If structured_output_schema is set, requests a structured (JSON) response constrained to that schema; otherwise returns the model’s text.
Records token usage for billing (managed usage only — see credit impact).
Writes the result into run state under the node’s id.

If user_prompt is omitted, the node falls back to the run-state field named input (the run’s top-level input). If that is also empty, the user message is an empty string.

The output of an LLM node is always stored in run state under the node’s id — for example a node with id: "summary_1" writes to {{summary_1}}. The legacy output_key field is deprecated and ignored for routing; do not rely on it. See inputs & outputs.

Configuration (`LLMNodeConfig`)

These fields live on the node’s llm_config. In the builder you set them through the detail panel; over the API they appear inside the node definition (the builder also accepts a wrapped {config: {...}} form, which the backend normalizes to llm_config).

LLMConfig object

required

The model to call. Required. See the llm object below for its fields. This selects both the provider (managed or BYOK) and the specific model.

string

The system message that sets the model’s role and instructions. Optional. Supports {{node_id.field}} and {{input}} references. If omitted, no system message is sent.

string

The human message — the actual task or question. Optional but recommended. Supports {{node_id.field}} and {{input}} references. If omitted, the node uses the run-state input field; if that is empty, the user message is an empty string.

object (JSON Schema)

A JSON Schema describing the shape you want the model to return. Optional. When set, the node returns a parsed object matching the schema instead of free text. ModuleX automatically fills in a title and description at the top of the schema if you omit them (LangChain requires these for tool/function calling). See structured output.

boolean

default:"false"

Only meaningful when structured_output_schema is set. When true, the provider enforces strict schema validation: a response that does not exactly match the schema fails instead of being loosely coerced. When false (the default), responses are coerced more leniently. Opt in per node when you want the model to fail fast on schema drift.

Deprecated fields — do not use in new workflows

These fields exist on LLMNodeConfig for backward compatibility and are normalized away or ignored. Author new workflows without them.

string

deprecated

Deprecated. Use user_prompt instead. If prompt_template is set and user_prompt is not, the backend copies prompt_template into user_prompt automatically.

string[]

deprecated

Deprecated. Use {{node_id.field}} references in your prompts instead of listing state keys.

string

deprecated

Deprecated. Output is always stored under the node id. Setting this to the literal value messages triggers a legacy messages-state code path used by the simple chat runtime, not by ordinary workflow nodes — avoid it.

The `llm` object

The llm field is an LLMConfig. It is required and identifies the provider, the model, and (optionally) which stored credential to use.

string

required

The provider integration on the wire. Required. Use modulexai for ModuleX-managed models (billed in credits). For BYOK, use the provider integration name — for example anthropic, openai, gemini, or xai. See LLM providers and managed vs BYOK.

string

required

The underlying provider id. Required. For example anthropic, openai, gemini, xai, or — for managed routing — openrouter. Each model in a provider catalog declares its own provider_id; match it for the model you pick.

string

required

The standardized model id, for example claude-sonnet-4.6, claude-haiku-4.5, or gpt-5.4-mini. Required. ModuleX maps this id to the provider’s served model. Models marked deprecated or in maintenance are automatically routed to their replacement for managed integrations. Browse available ids on each provider page.

number

default:"0.4"

Sampling temperature passed to the model. Optional; defaults to 0.4. Lower values make output more deterministic; higher values make it more varied.

string

A specific stored credential to use for this call. Optional. If omitted, ModuleX resolves a credential for the integration in the current organization. Required in practice for BYOK providers (you must have connected your own key). See managing credentials.

Retry configuration

The llm node is retry-wrapped. You can attach a retry_config to the node definition itself (not inside llm_config) to control how failed calls are retried. If you omit it, the engine applies its default retry policy (2 retries). Errors are only retried when their type is in retry_on_error_types.

integer

default:"3"

Total attempts including the first. Range 1–10. 1 means no retry; 3 means the initial call plus 2 retries.

number

default:"1.0"

Seconds to wait before the first retry. Range 0.1–60.

number

default:"2.0"

Multiplier applied to the delay between successive retries (exponential backoff). Range 1–5.

string[]

Exception type names that trigger a retry. Errors not in this list fail immediately. A credit-exhaustion stop is not retried.

See error handling & retries for how retries surface in the run stream.

Inputs and outputs

Inputs

The LLM node has no fixed input fields. It reads whatever you reference in system_prompt and user_prompt from run state:

{{node_id}} — the entire output of an upstream node (typed value: string, object, or array).
{{node_id.field}} — a nested value via dot/bracket path, for example {{extract_1.results[0].title}}. An out-of-range index or missing key resolves to nothing and the reference is left intact in the string.
{{input}} — the run’s top-level input field.

A reference that is the entire string keeps its native type; a reference embedded inside other text is string-substituted (objects/arrays are JSON-encoded). See variables & references for the full resolution model.

Outputs

The node writes one value into run state under its id:

string | object

The model’s response.

Show Shape by mode

string

When structured_output_schema is not set, the value is the model’s text response.

object

When structured_output_schema is set, the value is a parsed object conforming to your schema. If the provider’s structured call fails, the node falls back to parsing JSON out of the raw text response; if no JSON can be extracted, it stores the raw string instead.

Downstream nodes read this with {{node_id}} (whole value) or {{node_id.field}} (a field of a structured object).

Streaming

How an LLM node’s output streams depends on the run surface:

Normal workflow nodes stream at the node level over SSE. When the node finishes, the engine publishes a node_update event carrying the node’s output. On the wire this event is flat and uses the keys node and output — for example {type, node, output} — not the typed model field names. The run then ends with a done event.
Token-by-token streaming is only active on the simple LLM-chat runtime (the messages-state path), not on ordinary workflow llm nodes. In a standard workflow you receive the node’s result as a single node_update, not incremental tokens.

So for a workflow llm node, expect one node_update for the node, then the next node’s events. See SSE run streaming for the full event taxonomy and frame format.

The realtime wire dicts differ from the typed event models: the node_update frame uses node and output, and done carries only a message. Parse each SSE frame as JSON and switch on its type field — there is no SSE event: line. Details on SSE run streaming.

Structured output

Set structured_output_schema to a JSON Schema to make the node return a parsed object instead of text. ModuleX uses the provider’s native structured-output mechanism (tool calling on Anthropic, function calling on OpenAI, and the appropriate method elsewhere) and lets the provider pick the best method for the model you chose. Behavior to know:

If your schema omits a top-level title or description, ModuleX adds them automatically (LangChain requires them to build the tool definition the model sees).
With structured_output_strict: false (default), responses are coerced leniently. With structured_output_strict: true, a response that does not exactly match the schema fails instead of being coerced.
If the structured call fails for any reason, the node falls back to invoking the model normally and extracting JSON from the raw text. If no JSON can be extracted, it stores the raw string — so always validate downstream when correctness matters, or add a guardrails node to enforce the shape.

{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "neutral", "negative"]
    },
    "summary": { "type": "string" },
    "topics": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["sentiment", "summary"]
}

A downstream node can then read {{classify_1.sentiment}} or {{classify_1.topics}}.

Managed vs BYOK

The llm.integration_name you choose decides who runs the model and how it is billed:

Managed (modulexai)

Set integration_name to modulexai. The call runs through ModuleX-provisioned providers and is billed in credits by input/output tokens. No provider key of your own is required. See ModuleX-managed models.

BYOK (your own key)

Set integration_name to the provider (for example anthropic, openai, gemini, xai) and connect your own credential. Usage is billed directly by that provider with no ModuleX markup and is not charged in credits (token usage is recorded for analytics only). See LLM providers and managing credentials.

Both modes use the same LLMNodeConfig; only integration_name (and the credential) changes.

Credit impact

There is no fixed per-node credit charge for an LLM node. Charging is metered per model call:

Managed (modulexai) — each call records token usage and charges credits for input and output tokens. ModuleX measures usage in credits (the managed-usage billing unit).
BYOK — token usage is recorded for analytics, but no credits are charged; you pay your provider directly.

Workflow runs go through the usage gate, which admits the run before it starts. The gate is best-effort once a run is executing: a billing problem mid-run surfaces as a node error rather than crashing the run silently. If an organization runs out of credits mid-run, the node’s error event carries a stable reason token (credit_exhausted) so clients can detect a budget stop deterministically. See usage gating & limits and credits & metering.

Workflow run, Composer, Assistant, and managed-knowledge surfaces are gated by the billing admission gate, which can return a DenialEnvelope as 402 / 403 / 429. The flat envelope shape is {code, layer, key, current, limit, reason}. See errors & status codes and usage gating.

Errors

The LLM node surfaces failures through the run’s node-error event after retries are exhausted. Common cases:

Missing or invalid llm_config

If an llm node has no llm_config, compilation fails with a configuration error (LLM node <id> missing llm_config). Ensure the llm object with integration_name, provider_id, and model_id is set.

Provider / network errors

Timeouts, connection failures, and HTTP errors from the provider are retried per retry_config (defaults: TimeoutError, ConnectionError, HTTPError). When retries are exhausted, the node emits a node_error event with error_type, error_message, the attempt count, and recoverable: false, then the run fails. See error handling & retries.

Credit exhaustion mid-run (managed)

If a managed call cannot be billed because credits are exhausted, the node’s node_error event includes a stable reason token, credit_exhausted, so a client can match it without parsing free text. Resolve by topping up the wallet or upgrading your plan.

Structured output that cannot be parsed

If structured output is requested but the provider call fails and no JSON can be extracted from the fallback text, the node stores the raw string instead of an object. Downstream {{node_id.field}} references then resolve to nothing. Add a guardrails node or validate before relying on the shape.

For the full taxonomy of error-envelope shapes and which surface emits each, see errors & status codes.

Full example

A two-node workflow: an upstream node produces a customer message, and an llm node classifies it into structured fields. The first tab is the LLM node definition as it appears inside a workflow. The remaining tabs run a deployed workflow that contains this node. Running a workflow is asynchronous: the run call returns immediately with run metadata (status is running), and you observe node output and the final result by streaming events. The node’s structured object arrives on the node_update event for classify_1. See SSE run streaming and run a workflow.

{
  "id": "classify_1",
  "type": "llm",
  "name": "Classify message",
  "x": 480,
  "y": 160,
  "retry_config": {
    "max_attempts": 3,
    "initial_interval": 1.0,
    "backoff_factor": 2.0
  },
  "llm_config": {
    "llm": {
      "integration_name": "modulexai",
      "provider_id": "openrouter",
      "model_id": "claude-sonnet-4.6",
      "temperature": 0.2
    },
    "system_prompt": "You are a support triage assistant. Classify the customer message precisely.",
    "user_prompt": "Classify this message and summarize it in one sentence:\n\n{{intake_1.message}}",
    "structured_output_schema": {
      "type": "object",
      "properties": {
        "sentiment": { "type": "string", "enum": ["positive", "neutral", "negative"] },
        "priority": { "type": "string", "enum": ["low", "medium", "high"] },
        "summary": { "type": "string" }
      },
      "required": ["sentiment", "priority", "summary"]
    },
    "structured_output_strict": true
  }
}

# 1. Start the run (POST /workflows/run). Returns run metadata immediately.
curl https://api.modulex.dev/workflows/run \
  -H "Authorization: Bearer mx_live_xxxxxxxxxxxxxxxxxxxx" \
  -H "X-Organization-ID: org_4d2b18" \
  -H "Content-Type: application/json" \
  -d '{
        "workflow_id": "wf_7f3a9c",
        "input": {
          "intake_1": { "message": "My invoice double-charged me and I need this fixed today." }
        }
      }'
# -> { "status": "running", "run_id": "run_8b21", "thread_id": "thr_44", ... }

# 2. Stream events (SSE). Each frame is `data: <json>`; switch on the `type` field.
#    The classify_1 result arrives on its node_update event.
curl -N https://api.modulex.dev/workflows/listen/run_8b21 \
  -H "Authorization: Bearer mx_live_xxxxxxxxxxxxxxxxxxxx" \
  -H "X-Organization-ID: org_4d2b18"

import asyncio
from modulex import Modulex


async def main():
    async with Modulex(
        api_key="mx_live_xxxxxxxxxxxxxxxxxxxx",
        organization_id="org_4d2b18",
    ) as client:
        # 1. Start the run (returns immediately with run metadata).
        run = await client.executions.run(
            workflow_id="wf_7f3a9c",
            input={
                "intake_1": {
                    "message": "My invoice double-charged me and I need this fixed today.",
                },
            },
        )

        # 2. Stream events; classify_1's structured object arrives on its node_update.
        #    SSEEvent carries the discriminator in `.event` and the payload in `.data`.
        async for event in client.executions.listen(run.run_id):
            if event.event == "node_update" and event.data.get("node") == "classify_1":
                result = event.data["output"]["classify_1"]
                print(result["sentiment"])  # -> "negative"
                print(result["priority"])   # -> "high"


asyncio.run(main())

import { Modulex } from "modulex-js";

const client = new Modulex({
  apiKey: "mx_live_xxxxxxxxxxxxxxxxxxxx",
  organizationId: "org_4d2b18",
});

// 1. Start the run (returns immediately with run metadata).
const run = await client.executions.run({
  workflowId: "wf_7f3a9c",
  input: {
    intake_1: {
      message: "My invoice double-charged me and I need this fixed today.",
    },
  },
});

// 2. Stream events; classify_1's structured object arrives on its node_update.
//    Switch on `event.type`; the payload (node, output) lives in `event.data`.
for await (const event of client.executions.listen(run.run_id)) {
  if (event.type === "node_update" && event.data.node === "classify_1") {
    const result = event.data.output.classify_1;
    console.log(result.sentiment); // -> "negative"
    console.log(result.priority);  // -> "high"
  }
}

Every request authenticates with Authorization: Bearer mx_live_… plus X-Organization-ID. See authentication and the run-a-workflow guide. To stream the run instead of waiting for the final state, see SSE run streaming and streaming & HITL in the SDKs.

Agent node

When the step needs to call tools and loop, not just generate once.

Guardrails node

Validate JSON shape, regex, and PII on an LLM node’s output.

LLM providers

Managed and BYOK providers, model ids, and credentials.

Variables & references

How {{node_id.field}} references resolve against run state.

App overview

Chat

Knowledge

Collaboration

Builder

Nodes

Execution

Co-editing

Assistant Chat

What the node does

Configuration (`LLMNodeConfig`)

The `llm` object

Retry configuration

Inputs and outputs

Inputs

Outputs

Streaming

Structured output

Managed vs BYOK

Managed (modulexai)

BYOK (your own key)

Credit impact

Errors

Full example

Agent node

Guardrails node

LLM providers

Variables & references

​What the node does

​Configuration (LLMNodeConfig)

​The llm object

​Retry configuration

​Inputs and outputs

​Inputs

​Outputs

​Streaming

​Structured output

​Managed vs BYOK

Managed (modulexai)

BYOK (your own key)

​Credit impact

​Errors

​Full example

​Related

Agent node

Guardrails node

LLM providers

Variables & references

What the node does

Configuration (`LLMNodeConfig`)

The `llm` object

Retry configuration

Inputs and outputs

Inputs

Outputs

Streaming

Structured output

Managed vs BYOK

Credit impact

Errors

Full example

Related