Skip to main content
Every Assistant turn streams. When you start a turn, the response does not arrive in one block — the Assistant pushes a series of small JSON frames over a Server-Sent Events (SSE) connection as it thinks, calls tools, reads results, pauses for your input, and finishes. This page is the technical reference for that stream: the frame format on the wire, every event type the Assistant emits, how to consume the stream with listen() in the SDKs, how to cancel a turn, and how streaming in the app differs from streaming from your own code. The Assistant stream is one specific case of ModuleX run streaming. For the transport-level reference that also covers workflow runs, see SSE run streaming. For the SDK helpers that pair streaming with answering pause-for-input questions, see streaming and HITL.

How an Assistant turn streams

A turn is a request-then-listen pattern. You never receive events on the same request that starts the turn — starting a turn and listening to it are two separate calls.
1

Start the turn

Send POST /assistant/chat with your message. The response is a small JSON body (not a stream) that contains the chat_id, a freshly minted run_id, the thread_id, and a stream_url. See how the Assistant works for the full request shape.
2

Open the stream

Open GET /assistant/chat/{chat_id}/listen/{run_id} (this is exactly the stream_url from step 1). The connection stays open and the server pushes frames as the turn progresses.
3

Consume frames until terminal

Read frames and switch on each frame’s type. Keep reading until you receive a terminal frame (done, error, or cancelled), or until the turn pauses for human-in-the-loop input.
4

Resume or finish

If the turn paused with a user_input_request, answer it with POST /assistant/chat/{chat_id}/resume. Resume mints a new run_id and a new stream_url, so you open a fresh stream on the new id. Otherwise the turn is done.
The stream is server-to-client only. There is no way to send data back to the Assistant over the open SSE connection — answering a paused turn, cancelling, and starting the next turn are all separate REST calls. Resume and cancel are covered below and in human-in-the-loop.

The stream endpoint

SSE endpoint
Opens the live event stream for one Assistant run. Returns a text/event-stream response with the headers Cache-Control: no-cache, Connection: keep-alive, and X-Accel-Buffering: no.
string (UUID)
required
The Assistant chat to stream. Must be a chat you own in the current organization, with kind set to assistant.
string (UUID)
required
The run to stream, as returned by POST /assistant/chat or POST /assistant/chat/{chat_id}/resume. A new run_id is minted on every turn and on every resume, so the id is per-turn, not per-conversation. The stable per-conversation id is the thread_id, which equals the chat_id.

Authentication and authorization

Every call uses the standard ModuleX headers — see authentication.
  • Authorization: Bearer mx_live_… — your API key (a Clerk JWT is also accepted; the backend additionally accepts the key in X-API-KEY).
  • X-Organization-ID: org_… — the organization context. Missing this header returns 400.
The Assistant endpoints require the owner or admin role in that organization, enforced by organization_admin_required. The retired member role cannot use the Assistant. See roles and permissions and permissions and limits. Before the server subscribes you to the stream, it runs an ownership check that binds the run_id to your chat_id and organization. A run_id that is not in your chat — including one that belongs to another tenant — returns 404 with Run not found, never 403. There is no existence leak: “does not exist” and “not yours” return the same 404.
A non-2xx status on connect is delivered as an error before the first frame, not as a frame in the stream. In the SDKs this surfaces as a thrown typed exception the moment you start iterating (for example NotFoundError for 404). See error frames vs connection errors.

Action frames: the wire format

The Assistant stream is a data-only SSE stream. Each event is written as a single data: line whose value is a JSON object, terminated by a blank line:
There is no SSE event: line. The discriminator you switch on is the type field inside the JSON, not the SSE event name. This matters if you write your own parser: do not look for a named SSE event; parse the data: payload and read type.
The chat-list sidebar stream (GET /chats/stream) uses the opposite convention — real SSE event: lines such as event: chat_list_updated. That is a different stream for a different purpose and is not the Assistant turn stream. Do not reuse a parser written for one on the other. The full taxonomy is in SSE run streaming.

Flat vs wrapped payloads

Within a single Assistant stream, payloads are inconsistent about nesting. Most Assistant frames are wrapped: the real payload sits under a data key alongside type. The keepalive heartbeat frame is flat — it carries only type and nothing else.
One frame nests one level deeper than the rest: user_input_request. Its frame is {"type":"user_input_request","data":{…the question…}}, so the actual question object is at data.data, not data. A naive read of frame.data.kind returns nothing — read frame.data.data.kind. The Python SDK ships user_input_request_from_event() specifically to hide this; see the HITL frame.

Responses are snake_case

Frame payloads are the backend’s raw JSON, which is snake_case (run_id, tool_call_id, request_id). The SDKs do not convert streamed response fields to camelCase, so the typed event models you consume use snake_case field names. (The JavaScript SDK does convert request bodies camelCase to snake_case on the way out, but never response payloads.)

Event types

These are the frame types the Assistant emits, as actually published by the executor. Switch on type. Every payload field below lives under the frame’s data key unless noted.
object (wrapped)
First frame of the run. Carries run_id, thread_id, workflow_id (always null for the Assistant), workflow_type, and timestamp.
workflow_type is always "composer" on an Assistant run — even though no workflow is involved. It is not a reliable surface discriminator. Do not branch on it to detect “this is an Assistant run.” The reliable surface tag is the request metadata surface field (assistant).
object (wrapped)
A streamed slice of the Assistant’s natural-language reply. The payload is { chunk }, where chunk is a text fragment. Concatenate chunk values in order to build the running reply text.
object (wrapped)
The Assistant has decided to call a tool. Payload: tool (the tool name), input (the arguments object), tool_call_id (a stable id for this call), and timestamp. The Assistant calls one tool at a time within a step. See using tools.
object (wrapped)
The result of a tool_call. Payload: tool, tool_call_id (matches the originating tool_call), output, and timestamp. On a tool error the output carries an error shape such as {error, success: false}. Pair a result to its call by tool_call_id.
object (wrapped)
An internal progress signal the agent emits after repeated tool failures or at language-model-call milestones. Payload is {message, consecutive_failures} or {message, total_llm_calls}. Informational; safe to ignore in most clients.
object (double-wrapped)
The turn paused to ask you something — a choice, a yes/no, free text, or a credential connection. The question object is at data.data (one level deeper than every other frame). After this frame the stream goes quiet with no terminal frame; the run is paused server-side awaiting your answer. See the HITL frame and human-in-the-loop.
object
Published on the old run’s channel when an OAuth credential connection auto-resumes the turn without an explicit resume call. Payload is {new_run_id}. When you see it, switch your stream to the new run_id. See OAuth auto-resume.
object (wrapped) · terminal
The turn finished successfully. Payload carries response (the final reply text), tool_calls (the calls made this turn), usage (input_tokens, output_tokens, total_tokens, llm_calls), and the composer-shaped keys has_workflow_changes (always false for the Assistant) and workflow_tool (always null). This frame ends the stream.
object (flat) · terminal
The turn failed. Payload carries a message. This frame ends the stream. A backend exception mid-turn is emitted as a final error frame rather than dropping the connection. See error frames vs connection errors.
object · terminal
The turn was cancelled, typically by POST /assistant/chat/{chat_id}/cancel. Payload carries a reason. This frame ends the stream. See cancel a turn.
object · history-only · terminal marker
A history-only marker written when a turn pauses for input. It is appended to the run’s history but never published live, so an actively-connected client does not receive it — the live stream simply goes quiet after user_input_request. On a later reconnect the replay uses interrupted to stop cleanly. Treat it as terminal only when replaying history.
object (flat) · keepalive
A {"type":"heartbeat"} keepalive injected roughly every 15 seconds of idle time to keep the connection and any intermediary proxies alive. It carries no payload. Ignore it. The Python SDK filters heartbeats out by default; the JavaScript SDK yields them and you skip them yourself. See SDK heartbeat handling.
The Assistant has no workflow, so the composer-only frames workflow_change and workflow_sync never fire on an Assistant stream. If you are writing one consumer for both the Assistant and the Composer, you can ignore those two types on the Assistant path.

Terminal frames and when the stream ends

A stream ends after one of these terminal frames: done, error, or cancelled. The interrupted marker is also terminal, but only on a history replay — it is never sent live. There is one case where the stream does not end with a terminal frame: a live pause for input. After a user_input_request, the connection stays open and idle (with periodic heartbeats) but no done/error/cancelled arrives, because the run is parked waiting for your answer. Your consumer should stop reading on user_input_request, answer it, and open a fresh stream on the new run_id that resume returns.

Heartbeats, history, and reconnection

Each published frame is also appended to a per-run history list with a 1-hour TTL. When you (re)connect to a run that already has history, the server first replays the stored frames in order, then tails live. If the run already reached a terminal state, the replay yields up to and including the terminal frame and then closes — there is no live producer to wait for. This makes reconnecting within the hour replay-safe. The SDKs do not auto-reconnect; reconnection (and de-duplicating replayed tool_call/tool_result frames by tool_call_id) is the consumer’s job.

The HITL frame: user_input_request

When the Assistant needs a decision, it pauses and emits a user_input_request. The question is at data.data and is discriminated on its kind. The wire contract is shared with the Composer and is verified field-for-field across both SDKs.
You answer with POST /assistant/chat/{chat_id}/resume, sending the request_id, a response discriminated on its own kind, and the llm config. Resume mints a new run_id and stream_url. Only the user who triggered the question may answer it. The complete request/response contract, error codes, and worked examples are in human-in-the-loop and streaming and HITL.

OAuth auto-resume

When a credential_request opens an OAuth connection and you complete it, the OAuth callback resumes the turn for you — you do not call resume manually. The callback publishes a run_resumed frame on the old run’s channel carrying {new_run_id}. Your client should react by switching its stream to new_run_id. This is why Assistant chats share the Composer’s storage and resume machinery.

listen() in the SDKs

Both official SDKs wrap the stream behind a single listen() method that returns an async iterator of parsed frames. The auth headers, the data-only parsing, the snake_case payloads, and the connect-time error handling are all handled for you.
SDK streams are not subject to the client’s retry policy or request timeout. maxRetries and timeout apply to ordinary requests only; listen() opens the connection directly and relies on heartbeats to keep it alive. Cancellation is explicit — see cancel a turn.
The example below starts a turn, opens the stream, and prints text and tool activity until a terminal frame. Authenticate with Authorization: Bearer mx_live_… plus X-Organization-ID — see authentication.
The SDK surfaces differ in shape. The Python SDK yields a normalized SSEEvent whose logical name is on event.event and whose raw payload is on event.data; it exposes event.is_terminal. The JavaScript SDK casts each frame to a typed union and yields the payload object directly, discriminated on evt.type. In both, a user_input_request payload is nested — Python at event.data["data"], JavaScript at evt.data.

Heartbeats and keepalives

The two SDKs treat the heartbeat keepalive differently:
  • Python filters heartbeat (and keepalive) frames out — they are never yielded unless you pass include_heartbeats=True. Its read timeout is disabled on streams, so a long, idle pause-for-input does not time out.
  • JavaScript skips SSE comment lines automatically, but a {"type":"heartbeat"} data frame is yielded as a normal event — your consumer must ignore it (as the example does).

Error frames vs connection errors

There are two distinct error surfaces, and you handle them in different places:
  1. Connection errors happen on connect, before any frame. A non-2xx status throws a typed exception synchronously as you begin iterating — 401 AuthenticationError, 403 PermissionError, 404 NotFoundError, 429 RateLimitError. Wrap the start of iteration in a try/catch.
  2. Mid-stream errors arrive as a normal error frame inside the stream. The parser does not throw on them — your switch must handle type === "error" and treat it as terminal.
For the full SDK error model — classes, retries, and the 410 gap on resume — see errors and retries and streaming and HITL.

Cancel a turn

There is no in-band cancel over the stream. To stop a running turn, make a separate REST call to POST /assistant/chat/{chat_id}/cancel. The server sets a cancel flag, publishes a cancelled frame to the stream, sets the run status to cancelled, and — if the turn was paused on a HITL question — clears the pending question so it is not re-presented. If there is no running turn, the call returns 400 with No active execution to cancel. In the SDKs you also stop reading locally. Closing the iterator (via an AbortController in JavaScript, or by exiting the async with / breaking the loop in Python) stops consuming frames; calling cancel() stops the turn on the server. Do both: stop reading to free the client, and call cancel to stop the work and its metering.
Aborting the local stream in JavaScript stops iteration cleanly and releases the underlying reader, but it does not stop the turn on the server by itself — the work keeps running and keeps metering until you call cancel(). Always pair a local abort with a server-side cancel() when you want the turn to actually stop.

App streaming vs SDK streaming

Both the ModuleX app and the SDKs consume the same backend stream and the same frame types, but the transport mechanics differ. The app’s chat experience is covered in chat overview; the proxy and reconnection behavior described here is an implementation detail you do not configure.

A full worked example: a tool turn from start to finish

The trace below is a complete Assistant turn that calls a discovery tool, calls an integration action, and finishes. Note the data-only frames, the wrapped payloads, and the heartbeat keepalive.
If the same turn needed a credential you had not connected, the execute_integration_tool step would instead produce a user_input_request of kind credential_request; the stream would go quiet, and you would connect the credential (which auto-resumes via run_resumed) or answer with resume. See using tools and human-in-the-loop.

Credit impact

Streaming a turn does not add cost on top of the turn itself. The turn is charged once when it starts, and language-model token usage is recorded separately as the turn runs — these are the same charges described in permissions and limits and credits and metering. Opening the listen stream, receiving frames, reconnecting within the history window, and cancelling do not incur additional run charges. A turn that is cancelled mid-flight still records the language-model tokens it consumed before cancellation.

When the stream returns a billing denial

A turn is admitted by the billing gate before it starts, on POST /assistant/chat — not on the listen call. If your organization’s allowance is exhausted (or a rate or quota limit is hit), that POST is rejected before any run is created, so you never get a stream_url to open. The denial uses the flat DenialEnvelope:
The layer maps to the HTTP status: rate is 429 (with Retry-After and X-RateLimit-* headers), quota is 403, and credit or wallet is 402. This is distinct from the ordinary {"detail": "…"} shape used by validation and ownership errors — the Assistant endpoints can emit either. For the complete error catalog and all three envelope shapes, see errors and status codes and usage gating and limits.

Next steps

Human-in-the-loop

Answer a paused turn: the question and response kinds, the resume call, and the new-run_id contract.

Using tools

How the Assistant discovers, calls, and reports on integration tools — the tool_call and tool_result frames in context.

SSE run streaming

The transport-level reference for ModuleX run streaming, including workflow runs and the full event taxonomy.

Streaming and HITL in the SDKs

Consume streams and answer pause-for-input prompts in the JavaScript and Python SDKs.

Permissions and limits

Who can use the Assistant and the billing and usage limits that apply to each turn.

How the Assistant works

The agentic loop behind the frames: how the Assistant reasons, acts, and decides when it is done.