402 / 403 / 429, see
Usage gating & limits.
How the four levers interact
The levers are not independent in practice. Choices that cut cost often also cut latency (fewer managed model calls means fewer billed token batches and fewer network round-trips), and choices that improve reliability — more retries, more guardrails — can add both cost and latency. The table below is the trade-off map this page works through.Cost optimization (credits)
Only managed usage is metered in credits. An operation is managed when it runs through ModuleX’s own provider pool — its integration name ismodulexai (managed model and tool
calls) or modulexdb (managed knowledge), or its API-key source is the platform
environment. Everything that runs on your own provider keys is BYOK and is never
credited (see BYOK trade-offs). So the first cost question is always:
which calls in this workflow are managed?
The cost surface, ranked
These are the metered operations and their charges, from Credits & metering. Optimize the rows that dominate your usage, not the cheap flat fees.run, retrieval, file_ingest, tool base).multiplier (default 1.0).Tactics that actually move the number
Pick the smallest model that passes your guardrails
Trim prompt and completion tokens
0, never a silent 1.0, so cost tracks real
per-model pricing rates. Shorten system prompts, cap completion length, and avoid
re-sending large context on every node. For embeddings, completion tokens are forced to
0 (input-only), so only the text you embed is billed.Reduce tool calls inside loops
1 × multiplier credits. A
conditional node that loops a tool call N times
costs N credits before tokens. Batch where the integration supports it, or gate the
loop behind a condition.Cache and deduplicate retrieval and ingest
1 credit) is charged per call — avoid
issuing one retrieval per item in a loop when a single
knowledge node search would serve the whole batch.Charge once per turn, not per attempt
RUN_CREDIT = 1), and a
resumed turn re-enters through the resume path with source="existing",
which is a no-op for charging. Retries within a node (see Reliability) do
not add run credits; they may add tool or token credits if the retried call is itself
managed.Watch the allowance and the overage floor
Each plan grants a monthly credit allowance (Pro 5,000 / Max 20,000; Free 300 one-time; Enterprise custom). When the allowance is exhausted, a paid org with overage enabled spends down a prepaid wallet; a paid org without overage, or any Free org, is denied. The overage charge for a bucket is reconciled to exactlymax(0, month_sum − floor) credits, where floor = max(plan_limit, overage_baseline_credits) —
so enabling overage mid-month never retroactively bills the spend that preceded the toggle.
Latency
ModuleX runs workflows in the background and streams progress over Server-Sent Events. Understanding that path is the key to reasoning about latency.The run path
A run does not block the HTTP response while it executes.POST /workflows/run performs the
admission gate, mints a run_id, enqueues a background task, and returns immediately with
{run_id, status, ...}. You then attach to the live stream with
GET /workflows/listen/{run_id} over SSE. The same run_id is
reused across a resume, so you can re-listen after a pause.
run_id, created_at). The JS
SDK converts request keys camelCase to snake_case but does not convert responses back,
so read run.run_id, not run.runId.Latency tactics
Attach to the stream early, and survive reconnects
Attach to the stream early, and survive reconnects
POST /workflows/run returns. Attach to
GET /workflows/listen/{run_id} immediately so you observe the first node as it runs.
The stream replays the run’s buffered history on connect, so a late or reconnecting
listener still receives earlier events in order. A {"type":"heartbeat"} keepalive is
injected every 15s of silence — treat it as liveness, not as a workflow event, and do
not let an idle-connection timeout shorter than 15s drop the stream.Stop on terminal events, do not poll
Stop on terminal events, do not poll
done, error, and cancelled. Stop reading when you see one
rather than polling run status in a loop — both SDKs already break iteration on terminal
events. Polling adds latency and round-trips without surfacing progress any sooner.Keep the critical path managed when network proximity matters
Keep the critical path managed when network proximity matters
Defer overage reconciliation off the run's critical path
Defer overage reconciliation off the run's critical path
defer_overage_reconcile=True), so billing
bookkeeping does not sit on the run’s hot path. You do not configure this — it is the
default behavior — but it is why a wallet-funded run is not slower than a plan-funded one.Tune retry intervals so recovery does not dominate run time
Tune retry intervals so recovery does not dominate run time
initial_interval · backoff_factor^(attempt-1) seconds between
attempts. With the defaults (1.0s start, 2.0× factor, 3 attempts) a node that keeps
failing adds 1.0s + 2.0s = 3.0s of pure waiting before it gives up. If a downstream
service recovers quickly, lower initial_interval; if it recovers slowly, fewer attempts
with a larger interval can finish sooner than many short ones. See
Error handling & retries.Reliability
Reliability is mostly a matter of configuring retries deliberately and failing fast on errors that retrying cannot fix.Retry the transient, fail fast on the rest
Each node carries an optionalretry_config. Omit it and the engine applies a built-in
default. The full contract lives in
Error handling & retries; the parameters that
matter for tuning are below.
1 is no retry and 3 is two retries. Range
1-10. More attempts improve recovery but add latency and, for managed calls, credits.0.1-60.0. Lower it for fast-recovering services;
raise it for rate-limited upstreams that need a cooldown.1.0-5.0.
A factor of 1.0 makes the delay constant.Reliability tactics
Add only the error types you can actually recover from
RateLimitError to a tool node’s retry_on_error_types lets it ride out a brief
upstream 429. Adding a non-recoverable type (an authentication failure, a malformed
reference) just wastes attempts and latency. Match the list to the failures the node
really sees.Validate outputs with a guardrails node
Pause for a human on irreversible steps
run_id continues. This trades
latency and a human in the loop for the certainty that an irreversible action was
confirmed.Design retries to be idempotent at the SDK boundary
429, 500, 502, 503). The Python SDK
retries GET/HEAD only; the JS SDK retries all methods. A POST /workflows/run that is
network-retried could therefore start two runs from JS. Make your callers idempotent (for
example, key the trigger on your own request id), because the server-side
Idempotency-Key plumbing does not deduplicate runs — see the caveat below.Stay under the rate limits
Managed runs are admission-gated. Thesync_exec run-rate class is consumed only when the
call authenticates with an API key (auth_method == "api_key"); when that class is
exceeded the gate returns a rate denial. The per-plan sync_exec limits are 10 runs/min
(Free), 150 (Pro), 500 (Max), and unlimited (Enterprise). A 429 carries Retry-After and
X-RateLimit-Limit / -Remaining / -Reset headers; both SDKs honor Retry-After for
backoff. Smooth your call rate against these limits rather than bursting and absorbing 429s.
For the full status taxonomy and every error-envelope shape, see
Errors & status codes and Rate limiting.
BYOK trade-offs
BYOK (bring your own key) means a model, tool, or knowledge call runs on your provider account instead of ModuleX’s managed pool. This is the single biggest cost-versus-control decision in the platform, and it cuts across all three levers above.What changes when you go BYOK
retry_config accordingly.A decision rule
Default to managed for low-volume and prototyping
Move high-volume model calls to BYOK when your provider rate is lower
1.05 margin. If you have a negotiated provider
rate or committed-use discount, a high-volume LLM node is often cheaper on BYOK — at the
cost of owning that provider’s quota and latency.Use BYOK when you need a specific provider, region, or model
Keep knowledge managed unless you already run a vector store
modulexdb) bills retrieval and ingest at 1 credit each. If you
already operate Qdrant,
Pinecone,
MongoDB Atlas, or
Weaviate, a BYOK
knowledge provider removes those credits in
exchange for running the store yourself.feature.byok
entitlement in the backend. The cost trade-off is purely managed-credits versus your provider
bill, not a plan upgrade.A worked tuning pass
Putting the levers together on one workflow — a daily digest that retrieves from a knowledge base, summarizes with a managed model, and posts to Slack:Measure where the credits go
run (1), one retrieval (1), one managed LLM summarize (token-metered, the
dominant cost), and one Slack tool call (1 × multiplier). The LLM node is the row to
optimize first.Cut the dominant cost
Harden the external step
tool node is the only call that leaves the platform, so give it a focused
retry_config — ["ConnectionError", "HTTPError", "RateLimitError"], a short
initial_interval, and max_attempts of 3-4. Leave the in-memory nodes on the
default.Decide managed vs BYOK on volume
Stay under the rate ceiling at scale
sync_exec limit for your
plan, and back off on Retry-After rather than retrying immediately.