The Assistant's agentic loop: reason, act, observe

This page is the technical reference for what happens inside a single Assistant turn. It covers the agentic reason to act to observe loop, the storage and execution model the Assistant shares with the AI Composer, the relationship between turns and runs, every way a turn can terminate, the limits that bound it, and the exact credit impact. If you are looking for the product tour first, start with the Assistant overview. The Assistant runs the assistant profile of the same agent core that powers the Composer. The two are separated only by kind and profile: the Composer edits a workflow graph, while the Assistant has no workflow tools at all and acts directly on your connected integration tools and knowledge bases.

The reason, act, observe loop

A turn is an agentic loop. The Assistant alternates between calling the language model (reason) and calling exactly one tool (act), then feeds the tool result back into the model (observe), and repeats until the model produces a final answer with no further tool call. Each pass of the loop produces streamed events you can observe live over SSE:

Reason — the model decides the next step

The Assistant calls the language model with the conversation so far and the available tools. The model streams its text as response_chunk events. It either ends the turn with a final answer, or it decides to call a tool.

Act — one tool runs

If the model asks for a tool, the Assistant emits a tool_call event and runs that single action. Tool calls are serialized: SerialToolCallsMiddleware forces one tool call per model step, so two tools never fire in the same step. The Assistant can call a tool that discovers integrations, runs an integration action, searches knowledge, or asks you a human-in-the-loop question.

Observe — the result feeds back

The Assistant emits a tool_result event carrying the tool’s output (or an error payload on failure) and feeds it back to the model. The loop returns to the reason step.

Repeat until the model is done

The loop continues until the model returns a turn with no tool call. The Assistant then emits a done event with the final response and usage totals, and the turn ends.

The same loop powers the Composer, but the Assistant’s tool set is restricted to discovery (get_available_integrations, get_integration_details, get_organization_credentials), execution (execute_integration_tool), knowledge (search_knowledge), and the HITL tools (ask_user_choice, ask_user_multi_choice, ask_user_yes_no, ask_user_free_text, request_credential). Workflow inspect, build, edit, and run tools are deliberately excluded — a resumed Assistant run can never regain them because the profile is re-derived from the chat’s kind.

Storage and execution model

Assistant and Composer chats share the same conversation store, distinguished by a kind value (assistant vs composer), so they cannot read or operate on each other’s chats. Every service call is scoped by kind. The Assistant intentionally reuses the Composer’s agent factory, executor, HITL interrupt/resume machinery, OAuth auto-resume, and SSE streaming — staying in the shared store is required so that OAuth auto-resume keeps working.

object

The shared chat backing both surfaces.

Show Key fields

string (UUID)

The chat identifier. This value is the conversation checkpoint thread_id, so thread_id == chat_id for the life of the conversation.

string

assistant for Assistant chats, composer for Composer chats. The discriminator that scopes every service call.

string

Defaults to the first 50 characters of the opening message.

string (UUID) | null

The run_id of the in-flight or interrupted run, set synchronously at run start and cleared on completion, failure, or cancel. It is the race-free binding for run-to-chat ownership.

null

Always null for Assistant chats — they are never workflow-bound. Because of this, the executor’s workflow_change and workflow_sync events never fire for the Assistant.

What differs from the Composer at runtime: the Assistant has no subagents (the Composer uses an integration-resolver and a credential-resolver), runs slimmer middleware (it keeps SerialToolCallsMiddleware and SummarizationMiddleware plus provider prompt caching, and skips the todo-list, filesystem, and subagent middleware), and uses an Assistant system prompt with no workflow or focus language. The save, revert, and focus endpoints that the Composer exposes are omitted entirely.

Turns and runs

These two words are not interchangeable; the distinction matters for billing, streaming, and identity.

Turn

One user message answered by one agent loop. A turn starts with POST /assistant/chat and ends when the loop reaches a final answer, pauses for input, is cancelled, or errors. Billing charges exactly one run credit per turn.

Run

A single execution of the loop, identified by a run_id. A turn maps to one run — but when a turn pauses for human-in-the-loop input and you resume it, a new run_id is minted for the continuation. So one logical turn can span more than one run_id.

The three run-id identities

run_id is used at several layers, and you must not assume one identity. The same caution applies to workflow runs — see workflows and runs for the full treatment.

string (UUID)

The identifier used for SSE streaming, status, and history keys. A new run_id is minted on every resume, so it is not stable across a multi-step conversation and is not a conversation-level key.

string (UUID)

The conversation checkpoint thread. thread_id == chat_id and is stable across the whole conversation. Every run in a chat re-enters the same thread.

string

The durable run identifier persisted on the run row, a unique string value. It is the stable durable identifier the docs reference for a run.

How a turn is admitted, charged, and tracked

When POST /assistant/chat is called, the Assistant runs a synchronous, reject-before-write sequence so a denied turn never writes any rows:

Concurrency guards

If the chat already has a pending HITL question, the request returns 409 (answer it first). If a run is already in flight on the chat, the request returns 409 (wait for it or cancel it). A chat holds only one open question and one running turn at a time.

Billing admission gate

The Assistant calls the run-admission gate with the turn key {chat.id}:{run_id}, usage type run, and rate class sync_exec. Assistant turns count under sync_exec unconditionally. On denial it raises a billing error (402 / 403 / 429) before any rows are written. See credit impact.

Persist and charge

On success the Assistant appends your message, sets running_id to the new run_id, commits, and charges exactly one run credit for the turn. Any error on this path releases the admission reservation so it does not leak.

Execute in the background

The loop is scheduled as a background task with profile="assistant" and workflow_id=None. The response returns immediately with a stream_url you open to watch the turn.

The POST /assistant/chat response gives you everything you need to start streaming:

Response

{
  "status": "running",
  "chat_id": "f2b1c0de-1111-2222-3333-444455556666",
  "run_id": "a1b2c3d4-aaaa-bbbb-cccc-dddddddddddd",
  "thread_id": "f2b1c0de-1111-2222-3333-444455556666",
  "stream_url": "/assistant/chat/f2b1c0de-1111-2222-3333-444455556666/listen/a1b2c3d4-aaaa-bbbb-cccc-dddddddddddd"
}

Termination and limits

A turn’s loop ends in exactly one of four ways. Three are terminal for the run; one (HITL pause) keeps the chat alive awaiting your response.

Natural completion — the model stops calling tools

The model returns a turn with no tool call. The Assistant emits a done event carrying the final response, the list of tool_calls, and a usage object (input_tokens, output_tokens, total_tokens, llm_calls). For the Assistant, has_workflow_changes is always false and workflow_tool is always null (the composer-shaped keys are present but empty). The run status becomes completed and running_id is cleared.

Human-in-the-loop pause — the model asks you a question

The model calls a HITL tool (ask_user_* or request_credential), which fires an interrupt. The Assistant writes an interrupt-audit row (outcome="pending"), publishes a user_input_request event, sets a server-side pending sentinel (TTL 7 days), flips run status to interrupted, appends a history-only interrupted marker, and cold-exits the loop awaiting resume. The chat stays open — this is not a terminal run for the conversation. You answer with POST /assistant/chat/{chat_id}/resume, which mints a new run_id and re-enters the same thread. See human-in-the-loop and the HITL resume reference.

Cancellation — you stop the turn

POST /assistant/chat/{chat_id}/cancel sets a server-side cancel flag, publishes a cancelled event, and sets run status to cancelled. If the run was paused on a HITL interrupt, cancel also clears the pending sentinel and flips the audit row to cancelled so the question is not re-presented. Then running_id is cleared. Calling cancel with no active run returns 400 (No active execution to cancel).

Error — the turn fails

On failure the Assistant emits an error event with a sanitized message, followed by a done event with error: true, and sets run status to failed. Language-model token usage accrued before the failure is still recorded (token billing is independent of success).

Loop limits

The loop is bounded so it cannot run forever, and it emits guidance long before it hits the ceiling.

integer

default:"100"

The hard ceiling on agent loop iterations for a single run, enforced by the execution engine. A run that exceeds it stops with an error rather than looping indefinitely. There is no per-turn wall-clock timeout exposed in the run config; the recursion limit is the effective bound.

integer

default:"3"

After this many tool failures in a row, the Assistant publishes a guidance event ({message, consecutive_failures}) so the model — and you — know something is going wrong. The counter resets on the next successful tool call. This nudges the loop, it does not terminate it.

integer

default:"20"

Every multiple of this many language-model calls in a single run, the Assistant publishes a wind-down guidance event ({message, total_llm_calls}) encouraging the model to converge. This is advisory, not a hard stop.

A guidance event is informational. It signals that the loop is struggling (repeated failures) or running long (many LLM calls), but it does not end the turn. Only done, cancelled, error, and the HITL interrupted pause change the run’s lifecycle.

Events emitted during a turn

The turn streams as Server-Sent Events. Each frame is a single data: <json> line with no event: line — discriminate on the JSON type field. The payload for most events is nested under a data key, for example {"type":"response_chunk","data":{"chunk":"..."}}. See streaming responses for how to consume the stream in the app and the SDKs, and the SSE run streaming reference for the transport details.

`type`	Payload	Emitted when
`metadata`	`{run_id, thread_id, workflow_id, workflow_type, timestamp}`	At run start.
`response_chunk`	`{chunk}`	On each model text delta (the reason step).
`tool_call`	`{tool, input, tool_call_id, timestamp}`	When a tool is invoked (the act step).
`tool_result`	`{tool, tool_call_id, output, timestamp}`	When a tool returns (the observe step); on failure `output` is `{error, success: false}`.
`guidance`	`{message, consecutive_failures}` or `{message, total_llm_calls}`	On the failure threshold or every LLM-call milestone.
`user_input_request`	`{kind, request_id, message, ...}`	When a HITL tool fires an interrupt.
`interrupted`	`{request_id, kind}`	History-only terminal marker for the paused run.
`run_resumed`	`{new_run_id}`	On OAuth auto-resume, published on the old run’s channel.
`done`	`{response, has_workflow_changes, tool_calls, workflow_tool, usage}`	On success — terminal.
`error`	`{message}`	On failure, followed by a `done` with `error: true`.
`cancelled`	`{reason}`	After `POST /assistant/chat/{chat_id}/cancel`.
`heartbeat`	`{type: "heartbeat"}`	Keepalive every 15s of inter-event silence.

For Assistant runs, the metadata frame reports workflow_type as composer and workflow_id as null. Use the request’s surface metadata tag (assistant) as the per-surface discriminator, not workflow_type. The workflow_change and workflow_sync events do not fire for the Assistant.

Credit impact

The Assistant meters usage like any other managed run. Two distinct charges apply per turn:

One run credit per turn

The flat run charge is 1 credit per turn, applied through the admission gate at the start of the turn. Resuming a paused turn re-enters through /resume (not /chat), so a resume does not add a second run charge for the same turn.

Token usage, billed separately

Language-model token usage is recorded separately in the executor, based on input and output token counts, regardless of whether the turn succeeds, errors, or is cancelled. This is independent of the flat run credit.

If the loop calls integration tools or searches managed knowledge, those carry their own credit costs documented in credits and metering. Bring-your-own-key model usage is not credited (it is recorded for analytics only).

When a turn is denied

The Assistant is one of the surfaces where the billing admission gate is live. If your organization’s allowance and wallet cannot cover the turn, the gate denies it before any rows are written and the request fails with a flat DenialEnvelope — note this has no detail wrapper, unlike standard FastAPI errors:

DenialEnvelope (402)

{
  "code": "credit_plan_exhausted",
  "layer": "credit",
  "key": null,
  "current": null,
  "limit": null,
  "reason": "credit_plan_exhausted"
}

The layer field maps to the HTTP status: credit and wallet to 402, quota to 403, and rate to 429 (with Retry-After and X-RateLimit-* headers). For the full envelope taxonomy and how it differs from the {detail} shape, see errors and status codes; for the gating rules and per-plan limits, see usage gating and limits and permissions and limits.

A worked example: one turn with a tool call

This walks one full turn end to end: start the turn, stream the loop, watch a tool call and its result, and receive the final answer. Every request uses Authorization: Bearer mx_live_… plus X-Organization-ID — see authentication. As built today the Assistant requires the owner or admin role in the organization; the retired member role is not a current role.

Start the turn

Send one text message. The message must be a JSON string (v1 is text-only — an array or object returns 400). Omit chat_id to start a new chat, or pass it to continue one. If you omit llm, your organization’s saved default is used.

curl -X POST https://api.modulex.dev/assistant/chat \
  -H "Authorization: Bearer mx_live_your_api_key" \
  -H "X-Organization-ID: org_your_organization_id" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "List my GitHub repositories",
    "llm": {
      "integration_name": "openai",
      "provider_id": "openai",
      "model_id": "gpt-4o"
    }
  }'

from modulex import AsyncModulex

client = AsyncModulex(
    api_key="mx_live_your_api_key",
    organization_id="org_your_organization_id",
)

resp = await client.assistant.chat(
    "List my GitHub repositories",
    llm={
        "integration_name": "openai",
        "provider_id": "openai",
        "model_id": "gpt-4o",
    },
)
print(resp.chat_id, resp.run_id)

import { Modulex } from "modulex-js";

const client = new Modulex({
  apiKey: "mx_live_your_api_key",
  organizationId: "org_your_organization_id",
});

const resp = await client.assistant.chat({
  message: "List my GitHub repositories",
  llm: {
    integration_name: "openai",
    provider_id: "openai",
    model_id: "gpt-4o",
  },
});
console.log(resp.chat_id, resp.run_id);

Open the stream

Open the returned stream_url to receive the turn’s events as they happen. Both SDKs expose assistant.listen(chatId, runId) as an async iterator that yields the data payload of each frame.

curl -N https://api.modulex.dev/assistant/chat/{CHAT_ID}/listen/{RUN_ID} \
  -H "Authorization: Bearer mx_live_your_api_key" \
  -H "X-Organization-ID: org_your_organization_id"

async with client.assistant.listen(resp.chat_id, resp.run_id) as stream:
    async for event in stream:
        if event.event == "response_chunk":
            print(event.data["chunk"], end="")
        elif event.event == "tool_call":
            print(f"\n[tool] {event.data['tool']}")
        elif event.event == "done":
            print("\n[final]", event.data["response"])

for await (const event of client.assistant.listen(resp.chat_id, resp.run_id)) {
  if (event.type === "response_chunk") {
    process.stdout.write(event.data.chunk);
  } else if (event.type === "tool_call") {
    console.log(`\n[tool] ${event.data.tool}`);
  } else if (event.type === "done") {
    console.log("\n[final]", event.data.response);
  }
}

Read the loop in the frames

A clean single-tool turn looks like this on the wire (reason, act, observe, then a final answer):

Frame trace

data: {"type":"metadata","data":{"run_id":"a1b2c3d4-...","thread_id":"f2b1c0de-...","workflow_id":null,"workflow_type":"composer","timestamp":"2026-06-20T10:00:01+00:00"}}

data: {"type":"response_chunk","data":{"chunk":"Let me check your GitHub."}}

data: {"type":"tool_call","data":{"tool":"execute_integration_tool","input":{"integration":"github","action":"list_repos"},"tool_call_id":"run-xyz","timestamp":"2026-06-20T10:00:02+00:00"}}

data: {"type":"tool_result","data":{"tool":"execute_integration_tool","tool_call_id":"run-xyz","output":{"success":true,"repos":["acme/api","acme/web"]},"timestamp":"2026-06-20T10:00:03+00:00"}}

data: {"type":"response_chunk","data":{"chunk":"You have 2 repositories: acme/api and acme/web."}}

data: {"type":"done","data":{"response":"You have 2 repositories: acme/api and acme/web.","has_workflow_changes":false,"tool_calls":[{"tool":"execute_integration_tool","input":{"integration":"github","action":"list_repos"},"output":{"success":true,"repos":["acme/api","acme/web"]}}],"workflow_tool":null,"usage":{"input_tokens":1200,"output_tokens":340,"total_tokens":1540,"llm_calls":3}}}

Handle a pause, if one occurs

If the model instead emits a user_input_request (for example, to connect a missing credential), the run pauses and the stream stays open on heartbeats. Answer with assistant.resume(...) — which mints a new run_id you then open a fresh stream on. The full request and response kinds are in human-in-the-loop.

The message field must be a JSON string; an array or object returns 400. Continuing a chat that belongs to another organization returns 404 (ownership is hidden as not-found, not 403). A second turn while one is pending or running returns 409.

Where to go next

Using tools

How the Assistant discovers and calls integration tools in the act step, and how it requests credentials.

Streaming responses

Consume the turn’s events live over SSE in the app and the SDKs.

Human-in-the-loop

The pause-and-resume contract: question kinds, response kinds, and how resume mints a new run.

Permissions and limits

Who can use the Assistant and the billing and usage limits that bound a turn.

Workflows and runs

The run-id identities and the difference between a turn and a run, in the wider ModuleX model.

The Assistant concept

Where the Assistant sits relative to the Composer and the workflow engine.

App overview

Chat

Knowledge

Collaboration

Builder

Nodes

Execution

Co-editing

Assistant Chat

The Assistant's agentic loop: reason, act, observe

The reason, act, observe loop

Storage and execution model

Turns and runs

Turn

Run

The three run-id identities

How a turn is admitted, charged, and tracked

Termination and limits

Loop limits

Events emitted during a turn

Credit impact

One run credit per turn

Token usage, billed separately

When a turn is denied

A worked example: one turn with a tool call

Where to go next

Using tools

Streaming responses

Human-in-the-loop

Permissions and limits

Workflows and runs

The Assistant concept

​The reason, act, observe loop

​Storage and execution model

​Turns and runs

Turn

Run

​The three run-id identities

​How a turn is admitted, charged, and tracked

​Termination and limits

​Loop limits

​Events emitted during a turn

​Credit impact

One run credit per turn

Token usage, billed separately

​When a turn is denied

​A worked example: one turn with a tool call

​Where to go next

Using tools

Streaming responses

Human-in-the-loop

Permissions and limits

Workflows and runs

The Assistant concept

The reason, act, observe loop

Storage and execution model

Turns and runs

The three run-id identities

How a turn is admitted, charged, and tracked

Termination and limits

Loop limits

Events emitted during a turn

Credit impact

When a turn is denied

A worked example: one turn with a tool call

Where to go next