> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulex.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Bring your own vector store (BYOK knowledge)

> Connect Qdrant, Pinecone, MongoDB Atlas, or Weaviate as an external knowledge provider and retrieve from it in workflows. BYOK retrieval runs uncosted.

export const MediaEmbed = ({id, type = 'screenshot', caption = '', ext, ratio = '16 / 9'}) => {
  const isVideo = type === 'video' || type === 'app_video';
  const resolvedExt = ext || (isVideo ? 'mp4' : type === 'screenshot' ? 'webp' : 'svg');
  const src = 'https://media.modulex.dev/' + id + '.' + resolvedExt;
  const [status, setStatus] = useState('loading');
  const [isDev, setIsDev] = useState(false);
  const [inView, setInView] = useState(false);
  const boxRef = useRef(null);
  useEffect(() => {
    if (typeof window === 'undefined') return;
    const h = window.location.hostname;
    setIsDev(h === 'localhost' || h === '127.0.0.1' || h.endsWith('.mintlify.app'));
  }, []);
  useEffect(() => {
    if (inView) return;
    if (typeof IntersectionObserver === 'undefined') {
      setInView(true);
      return;
    }
    const el = boxRef.current;
    if (!el) return;
    const io = new IntersectionObserver(entries => {
      if (entries.some(e => e.isIntersecting)) {
        setInView(true);
        io.disconnect();
      }
    }, {
      rootMargin: '300px'
    });
    io.observe(el);
    return () => io.disconnect();
  }, [inView]);
  if (status === 'missing') {
    if (!isDev) return null;
    return <div style={{
      display: 'flex',
      flexDirection: 'column',
      justifyContent: 'center',
      gap: '0.4rem',
      padding: '1rem 1.25rem',
      margin: '1.25rem 0',
      width: '100%',
      aspectRatio: ratio,
      boxSizing: 'border-box',
      border: '1px dashed rgba(128,128,128,0.45)',
      borderRadius: '0.75rem',
      background: 'rgba(128,128,128,0.06)',
      color: 'currentColor',
      fontSize: '0.85rem',
      lineHeight: 1.45
    }}>
        <div style={{
      display: 'flex',
      alignItems: 'center',
      gap: '0.5rem',
      opacity: 0.75
    }}>
          <span aria-hidden="true">🎬</span>
          <code style={{
      fontSize: '0.75rem'
    }}>{id}</code>
          <span style={{
      fontSize: '0.65rem',
      textTransform: 'uppercase',
      letterSpacing: '0.04em',
      padding: '0.1rem 0.4rem',
      borderRadius: '0.4rem',
      background: 'rgba(128,128,128,0.18)'
    }}>
            {type}
          </span>
        </div>
        <div style={{
      opacity: 0.9
    }}>{caption || 'Media not uploaded yet.'}</div>
        <div style={{
      fontSize: '0.7rem',
      opacity: 0.5
    }}>
          Upload to R2 as <code>{id}.{resolvedExt}</code> — preview only, hidden in production.
        </div>
      </div>;
  }
  const mediaStyle = {
    display: status === 'loaded' ? 'block' : 'none',
    width: '100%',
    height: 'auto',
    borderRadius: '0.75rem'
  };
  const media = isVideo ? <video src={inView ? src : undefined} autoPlay loop muted playsInline preload="metadata" onLoadedData={() => setStatus('loaded')} onError={() => setStatus('missing')} style={mediaStyle} /> : <img src={inView ? src : undefined} alt={caption} onLoad={() => setStatus('loaded')} onError={() => setStatus('missing')} style={mediaStyle} />;
  return <figure style={{
    margin: '1.25rem 0'
  }}>
      <div ref={boxRef} style={status === 'loaded' ? {
    width: '100%'
  } : {
    width: '100%',
    aspectRatio: ratio,
    borderRadius: '0.75rem',
    background: 'rgba(128,128,128,0.06)'
  }}>
        {media}
      </div>
      {status === 'loaded' && caption ? <figcaption style={{
    marginTop: '0.5rem',
    textAlign: 'center',
    fontSize: '0.85rem',
    opacity: 0.7
  }}>
          {caption}
        </figcaption> : null}
    </figure>;
};

ModuleX can retrieve from a vector store you already own instead of its managed store. You connect the store once as an org credential, then point a [knowledge node](/workflow-builder/nodes/knowledge) at it by setting `provider_type` to the store's name. ModuleX embeds the query, queries your store over its API, and formats the matches back into the workflow run state.

This is the bring-your-own-key (BYOK) path for retrieval. Unlike [managed knowledge](/platform/knowledge/managed) — which is hosted in `modulexdb` and billed in credits — **BYOK retrieval is uncosted**: ModuleX never charges a retrieval credit when a knowledge node reads from an external provider. See [credits](/billing/credits) for the full metering model.

<Note>
  BYOK here means retrieval only. ModuleX does **not** ingest, chunk, or embed your documents into an external store — you own that pipeline and load vectors into the store yourself. ModuleX reads from a collection you have already populated. Document upload and ingest (`pending` to `processing` to `completed`) apply to managed knowledge bases only; see [managing documents](/platform/knowledge/documents).
</Note>

## Supported providers

Four external vector stores ship as knowledge providers. Each is browsable in the catalog via [knowledge providers](/integrations/knowledge-providers/overview) and has its own connection reference page.

<CardGroup cols={2}>
  <Card title="Qdrant" icon="database" href="/integrations/knowledge-providers/qdrant">
    High-performance vector store with payload filtering. `provider_type` value: `qdrant`.
  </Card>

  <Card title="Pinecone" icon="database" href="/integrations/knowledge-providers/pinecone">
    Managed serverless and pod-based vector store with namespaces. `provider_type` value: `pinecone`.
  </Card>

  <Card title="MongoDB Atlas" icon="database" href="/integrations/knowledge-providers/mongodb-atlas">
    Atlas Vector Search over your existing collections. `provider_type` value: `mongodb_atlas`.
  </Card>

  <Card title="Weaviate" icon="database" href="/integrations/knowledge-providers/weaviate">
    Open-source vector store with cloud and self-hosted instances. `provider_type` value: `weaviate`.
  </Card>
</CardGroup>

The managed store, `modulexdb`, is the fifth provider type and the default. It is documented separately under [managed knowledge](/platform/knowledge/managed) and [modulexdb](/integrations/knowledge-providers/modulexdb). The `provider_type` field on a knowledge node accepts exactly these five values: `modulexdb`, `qdrant`, `pinecone`, `weaviate`, `mongodb_atlas`.

<MediaEmbed id="MX-MEDIA-3412" type="image" caption={"BYOK retrieval data flow diagram."} />

## How retrieval works

A knowledge node configured for an external provider runs the same way on every run surface (builder, [run from chat](/workflow-builder/execution/run-on-chat), and [run via API](/workflow-builder/execution/api-endpoint)):

<Steps>
  <Step title="Resolve the query">
    The node resolves its `query` string, including any [`{{node_id.field}}` references](/workflow-builder/variables-and-references) to upstream node output. If `query_from_input` is `true`, the workflow input is used instead.
  </Step>

  <Step title="Load and decrypt the credential">
    ModuleX loads the org credential named by `credential_id`, scoped to your `X-Organization-ID`, and decrypts the connection fields (URL, API key, connection string) at runtime. See [data security & encryption](/security/data-encryption).
  </Step>

  <Step title="Embed the query">
    External stores receive a query **vector**, not text, so ModuleX embeds the query using the node's `embedding_config`. Use the same embedding model your vectors were created with, or matches will be meaningless. If the embedding model is a managed (`modulexai`) model, the embedding step is billed in credits; a BYOK embedding model is uncosted.
  </Step>

  <Step title="Query your store">
    ModuleX calls your store's query API with the vector, `collection_name`, `top_k`, `min_score`, and any `filters` / `namespace`. No retrieval credit is reserved or charged for this step.
  </Step>

  <Step title="Format and write to state">
    Matches are normalized into chunks and shaped by `output_format`, then written to run state under the node's `id` (the standard [node output convention](/workflow-builder/nodes/overview)).
  </Step>
</Steps>

<Warning>
  All four external providers require a query embedding vector. You **must** set `embedding_config` on the knowledge node — omitting it raises a runtime error: `embedding_config required for providers that don't handle embeddings`. This includes Weaviate: although Weaviate supports `nearText`, the ModuleX adapter always sends a precomputed vector.
</Warning>

## Connecting a provider

You connect an external store as an organization credential, then reference that credential from a knowledge node. Connecting requires the **owner** or **admin** role; the retired `member` role cannot create credentials. See [roles & permissions](/security/roles-permissions).

### In the app

Open the integration in [knowledge providers](/integrations/knowledge-providers/overview), choose Connect, and fill in the connection fields for that store (below). ModuleX validates the connection with a live test call before saving, then stores the fields encrypted. Manage the saved credential later under [managing credentials](/integrations/managing-credentials).

<MediaEmbed id="MX-MEDIA-3413" type="screenshot" caption={"Connecting an external vector store as a knowledge provider."} />

### Connection fields per provider

Each provider uses the `custom` auth type. The fields below are exactly the credential fields the store's auth schema declares — pass them inside `auth_data` when connecting via the API.

<Tabs>
  <Tab title="Qdrant">
    <ParamField path="url" type="string" required>
      URL of your Qdrant instance, including the port. Example: `https://xyz-abc.aws.cloud.qdrant.io:6333`. Not sensitive.
    </ParamField>

    <ParamField path="api_key" type="string">
      API key for Qdrant Cloud. Optional for local instances. Sensitive.
    </ParamField>

    The connection test lists collections via `GET {url}/collections`.
  </Tab>

  <Tab title="Pinecone">
    <ParamField path="api_key" type="string" required>
      Your Pinecone API key (format `pcsk_...`). Sensitive.
    </ParamField>

    <ParamField path="environment" type="string" required>
      Pinecone environment, for example `us-east-1-aws` or `gcp-starter`. Not sensitive.
    </ParamField>

    The connection test lists indexes via `GET https://api.pinecone.io/indexes`.
  </Tab>

  <Tab title="MongoDB Atlas">
    <ParamField path="connection_string" type="string" required>
      MongoDB Atlas connection string, for example `mongodb+srv://user:password@cluster.mongodb.net/`. Sensitive (it carries credentials).
    </ParamField>

    The connection test lists databases on the cluster.
  </Tab>

  <Tab title="Weaviate">
    <ParamField path="url" type="string" required>
      URL of your Weaviate instance, for example `https://your-cluster.weaviate.cloud`. Not sensitive.
    </ParamField>

    <ParamField path="api_key" type="string">
      API key for Weaviate Cloud. Optional for local instances. Sensitive.
    </ParamField>

    The connection test checks cluster status.
  </Tab>
</Tabs>

### Connecting via the API

Create the credential with `POST /credentials`. The credential type is auto-detected from the body: send `auth_type: "custom"` plus the provider's connection fields under `auth_data`, and set `integration_name` to the provider name (`qdrant`, `pinecone`, `mongodb_atlas`, or `weaviate`). Set `make_default: true` to make it the org's default credential for that provider. Every request authenticates with `Authorization: Bearer mx_live_…` and `X-Organization-ID`. A successful create returns `201`.

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://api.modulex.dev/credentials \
    -H "Authorization: Bearer mx_live_xxx" \
    -H "X-Organization-ID: org_123" \
    -H "Content-Type: application/json" \
    -d '{
      "integration_name": "qdrant",
      "display_name": "Prod Qdrant",
      "auth_type": "custom",
      "make_default": true,
      "auth_data": {
        "url": "https://xyz-abc.aws.cloud.qdrant.io:6333",
        "api_key": "qdrant-api-key-..."
      }
    }'
  ```

  ```python Python theme={null}
  from modulex import Modulex

  client = Modulex(
      api_key="mx_live_xxx",
      organization_id="org_123",
  )

  credential = await client.credentials.create(
      integration_name="qdrant",
      display_name="Prod Qdrant",
      auth_type="custom",
      make_default=True,
      auth_data={
          "url": "https://xyz-abc.aws.cloud.qdrant.io:6333",
          "api_key": "qdrant-api-key-...",
      },
  )
  print(credential.credential_id)
  ```

  ```javascript JavaScript theme={null}
  import { Modulex } from "modulex-js";

  const client = new Modulex({
    apiKey: "mx_live_xxx",
    organizationId: "org_123",
  });

  const credential = await client.credentials.create({
    integrationName: "qdrant",
    displayName: "Prod Qdrant",
    authType: "custom",
    makeDefault: true,
    authData: {
      url: "https://xyz-abc.aws.cloud.qdrant.io:6333",
      apiKey: "qdrant-api-key-...",
    },
  });
  console.log(credential.credentialId);
  ```
</CodeGroup>

<Accordion title="Validate before you save (POST /credentials/test-temporary)">
  To check connection fields without persisting a credential, post them to `POST /credentials/test-temporary` with `integration_name`, `auth_type`, and `auth_data`. The response reports validity and the test method:

  ```json theme={null}
  {
    "is_valid": true,
    "message": "Credential is valid for qdrant",
    "tested_at": "2026-06-21T12:00:00",
    "test_method": "api_call",
    "integration_name": "qdrant",
    "auth_type": "custom",
    "status_code": 200,
    "cost_level": "free"
  }
  ```

  `test_method` is one of `api_call`, `basic`, or `none`. To re-test a credential you already saved, post to `POST /credentials/{credential_id}/test`. See [managing credentials](/integrations/managing-credentials) for the full credential lifecycle (create, set-default, rotate, delete).
</Accordion>

## Configuring the knowledge node

Once the credential exists, set the [knowledge node](/workflow-builder/nodes/knowledge) `provider_type` to the store and reference the credential. The full node configuration follows.

### Parameters

<ParamField path="credential_id" type="string" required>
  The credential connecting to the external store. Must belong to the request's organization.
</ParamField>

<ParamField path="provider_type" type="string" default="modulexdb">
  The store to query. One of `modulexdb`, `qdrant`, `pinecone`, `weaviate`, `mongodb_atlas`. Set it to the external provider's name for BYOK retrieval.
</ParamField>

<ParamField path="query" type="string" required>
  The search query. Supports [`{{node_id.field}}` references](/workflow-builder/variables-and-references) so the query can come from upstream node output.
</ParamField>

<ParamField path="query_from_input" type="boolean" default="false">
  When `true`, use the workflow input as the query instead of `query`. ModuleX reads the first of `query`, `question`, `input`, `user_input`, or `message` from run state. Prefer a `{{...}}` reference in `query` instead; this flag is retained for backward compatibility.
</ParamField>

<ParamField path="collection_name" type="string" required>
  The collection / index / class to search. **Required for every external provider** — omitting it raises `collection_name is required for external provider`. The field maps to the store's own concept: a Qdrant collection, a Pinecone index, a Weaviate class, or a MongoDB collection. Not used by `modulexdb`.
</ParamField>

<ParamField path="namespace" type="string">
  A namespace within the collection, for stores that support one (Pinecone). Passed through to the store's query.
</ParamField>

<ParamField path="embedding_config" type="object" required>
  How ModuleX embeds the query before sending the vector to your store. **Required for all four external providers.** See the embedding fields below.
</ParamField>

<ParamField path="top_k" type="integer" default="5">
  Number of results to retrieve. Range `1` to `50`.
</ParamField>

<ParamField path="min_score" type="number" default="0.3">
  Minimum similarity score, `0.0` to `1.0`. Scores are normalized to this range per the store's metric before filtering.
</ParamField>

<ParamField path="max_tokens" type="integer" default="2000">
  Maximum tokens in the formatted context string, applied only when `output_format` is `context` or `both`. Range `100` to `10000`.
</ParamField>

<ParamField path="filters" type="object">
  Provider-specific filter conditions passed to the store's query (for example a Qdrant `filter`, a Pinecone metadata filter, or a MongoDB MQL pre-filter).
</ParamField>

<ParamField path="document_ids" type="array">
  Restrict results to specific document IDs. **Native (`modulexdb`) knowledge bases only** — ignored by external providers; use `filters` instead.
</ParamField>

<ParamField path="output_format" type="string" default="context">
  Shape of the result written to state. One of `chunks` (matched chunks with metadata), `context` (a single RAG-ready string), or `both`.
</ParamField>

<ParamField path="include_metadata" type="boolean" default="true">
  Include each chunk's metadata in the results.
</ParamField>

<ParamField path="include_source" type="boolean" default="true">
  Include a source header (`[n] (Source: …, Score: …)`) in the formatted context. For external providers the source is read from `metadata.filename` or `metadata.source`, falling back to `Unknown`.
</ParamField>

#### Embedding config fields

<ParamField path="embedding_config.integration_name" type="string" required>
  The embedding provider's integration name, for example `openai`, `voyage`, or `cohere`.
</ParamField>

<ParamField path="embedding_config.provider_id" type="string" required>
  The provider identifier, for example `openai` or `voyage`.
</ParamField>

<ParamField path="embedding_config.model_id" type="string" required>
  The embedding model, for example `text-embedding-3-small`. Match the model your stored vectors were created with.
</ParamField>

<ParamField path="embedding_config.credential_id" type="string">
  The credential for the embedding provider. If omitted, ModuleX uses the organization's **default** credential for `integration_name`. If no default exists, the node fails with `No default credential found for integration`.
</ParamField>

### Output (written to run state)

The node writes a single object to run state under its node `id`, regardless of provider:

<ResponseField name="total_results" type="integer">
  Number of matches returned by the store.
</ResponseField>

<ResponseField name="chunks" type="array">
  Present when `output_format` is `chunks` or `both`; otherwise `null`. Each chunk:

  <Expandable title="chunk object">
    <ResponseField name="content" type="string">The chunk text, extracted from the match payload.</ResponseField>
    <ResponseField name="score" type="number">Normalized similarity score, `0.0` to `1.0`.</ResponseField>
    <ResponseField name="metadata" type="object">The match metadata, or `null` when `include_metadata` is `false`.</ResponseField>
    <ResponseField name="id" type="string">The store's match identifier.</ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="context" type="string">
  Present when `output_format` is `context` or `both`; otherwise `null`. A single string joining the top chunks (token-budgeted by `max_tokens`), each prefixed with `[n]` and, when `include_source` is `true`, a `(Source: …, Score: …)` header.
</ResponseField>

<Note>
  When the query resolves to empty or the node errors at run time, it does **not** crash the run. Instead it writes `{chunks: [], context: "", total_results: 0, error: "<message>"}` to state so downstream nodes can branch on it. Combine this with a [conditional node](/workflow-builder/nodes/conditional) to handle no-result cases.
</Note>

## Worked example

A two-node workflow: an [LLM node](/workflow-builder/nodes/llm) rewrites the user question, then a knowledge node retrieves from a Qdrant collection using that rewritten query and an OpenAI embedding model. The `query` references the LLM node's output via `{{rewrite.text}}`.

<CodeGroup>
  ```json Knowledge node config theme={null}
  {
    "id": "retrieve",
    "type": "knowledge",
    "config": {
      "provider_type": "qdrant",
      "credential_id": "a1b2c3d4-0000-0000-0000-000000000000",
      "collection_name": "product_docs",
      "query": "{{rewrite.text}}",
      "top_k": 8,
      "min_score": 0.4,
      "output_format": "both",
      "include_metadata": true,
      "include_source": true,
      "filters": { "must": [ { "key": "lang", "match": { "value": "en" } } ] },
      "embedding_config": {
        "integration_name": "openai",
        "provider_id": "openai",
        "model_id": "text-embedding-3-small",
        "credential_id": null
      }
    }
  }
  ```

  ```json Pinecone variant theme={null}
  {
    "id": "retrieve",
    "type": "knowledge",
    "config": {
      "provider_type": "pinecone",
      "credential_id": "a1b2c3d4-0000-0000-0000-000000000000",
      "collection_name": "product-docs",
      "namespace": "v2",
      "query": "{{rewrite.text}}",
      "top_k": 8,
      "output_format": "context",
      "embedding_config": {
        "integration_name": "openai",
        "provider_id": "openai",
        "model_id": "text-embedding-3-small"
      }
    }
  }
  ```

  ```json Run state after the node theme={null}
  {
    "retrieve": {
      "total_results": 8,
      "chunks": [
        {
          "id": "chunk-4271",
          "score": 0.83,
          "content": "To install the CLI, run npm install -g modulex-cli ...",
          "metadata": { "filename": "install.md", "lang": "en" }
        }
      ],
      "context": "[1] (Source: install.md, Score: 0.83) To install the CLI, run npm install -g modulex-cli ..."
    }
  }
  ```
</CodeGroup>

A later node reads the retrieved context with `{{retrieve.context}}` (for `context`/`both`) or iterates `{{retrieve.chunks}}` (for `chunks`/`both`). See [variables & references](/workflow-builder/variables-and-references) for the reference syntax and [run a workflow](/guides/run-a-workflow) for triggering the run end to end.

## Credit impact

<Card title="BYOK retrieval is uncosted" icon="circle-check">
  A knowledge node reading from `qdrant`, `pinecone`, `weaviate`, or `mongodb_atlas` never reserves or charges a retrieval credit. Only managed (`modulexdb`) retrieval is billed, at one retrieval credit per call. See [credits](/billing/credits) and [usage gating](/billing/usage-gating).
</Card>

Two cost nuances remain:

* **Run credit.** Every workflow run still costs the flat run credit, independent of which provider a knowledge node uses. See [credits](/billing/credits).
* **Embedding cost.** If `embedding_config` points at a managed (`modulexai`) embedding model, embedding the query is billed in credits like any managed token usage. Point `embedding_config` at a BYOK embedding credential to keep the whole retrieval path uncosted.

Because BYOK retrieval is never gated, an external-provider knowledge node does **not** emit the billing [`DenialEnvelope`](/api-reference/errors) (`402` / `403` / `429`). The billing gate that protects managed retrieval is skipped for BYOK. Your store's own quotas and rate limits still apply, and they surface as run errors (see below), not as ModuleX billing denials.

## Errors

External-provider knowledge nodes fail at run time rather than at request validation. A failure is captured into the node's state object as an `error` field; the run continues so you can branch on it.

| Condition                                          | Message                                                                                       |
| -------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| `collection_name` missing for an external provider | `collection_name is required for external provider: <type>`                                   |
| `embedding_config` missing                         | `embedding_config required for providers that don't handle embeddings`                        |
| `credential_id` not found in the org               | `Credential not found: <id>`                                                                  |
| Embedding `credential_id` not found                | `Embedding credential not found: <id>`                                                        |
| No default embedding credential and none specified | `No default credential found for integration '<name>'`                                        |
| Store auth or connectivity failure                 | The provider adapter raises a connection, authentication, or query error, surfaced in `error` |

Validation errors on the credential API (for example a malformed body, or MCP-only fields) return the standard `{detail}` `HTTPException` shape with status `400` or `500`. For the full envelope taxonomy across surfaces, see [errors & status codes](/api-reference/errors).

## Related

<CardGroup cols={2}>
  <Card title="Knowledge providers" icon="layer-group" href="/integrations/knowledge-providers/overview">
    Browse the managed and external knowledge providers ModuleX can use.
  </Card>

  <Card title="Knowledge node" icon="square-terminal" href="/workflow-builder/nodes/knowledge">
    The node that retrieves from any provider inside a workflow.
  </Card>

  <Card title="Managed knowledge" icon="server" href="/platform/knowledge/managed">
    The credit-billed modulexdb store, with ingest and search.
  </Card>

  <Card title="Knowledge & RAG" icon="book-open" href="/concepts/knowledge-rag">
    The retrieval model behind chats and workflows.
  </Card>
</CardGroup>