Knowledge providers: managed and external vector stores

A knowledge provider is the vector store ModuleX searches when a workflow or chat needs retrieval. ModuleX supports two families: the managed store (modulexdb), where ModuleX hosts your embeddings and chunks and bills retrieval in credits, and external (bring-your-own) stores — Qdrant, Pinecone, MongoDB Atlas, and Weaviate — that you connect with your own credentials and query without ModuleX markup. This page is the exhaustive reference for both: how each is identified, the exact fields you supply to connect, the knowledge node contract that consumes them, and the retrieval billing model. For the conceptual model of retrieval-augmented generation in ModuleX, see Knowledge & RAG. To create and manage a managed knowledge base in the app, see Knowledge overview and Managed knowledge (modulexdb). To wire a provider into a graph, see the Knowledge node.

Knowledge bases and their provider credentials are organization-scoped. Every knowledge endpoint and every credential operation requires the owner or admin role (organization_admin_required); the member role is retired. The organization is selected by the X-Organization-ID header on every request — see Org context & X-Organization-ID.

The two provider families

A provider is identified on the wire by a provider_type value. The backend enumerates exactly five (KnowledgeProviderType): one managed and four external.

`provider_type`	Family	Display name	How you connect	Retrieval billing
`modulexdb`	Managed	modulexdb (managed)	Auto-created when you create a knowledge base	Billed in credits
`qdrant`	External (BYOK)	Qdrant	`custom` credential (URL + API key)	Uncosted
`pinecone`	External (BYOK)	Pinecone	`custom` credential (API key + environment)	Uncosted
`mongodb_atlas`	External (BYOK)	MongoDB Atlas Vector Search	`custom` credential (connection string)	Uncosted
`weaviate`	External (BYOK)	Weaviate	`custom` credential (URL + API key)	Uncosted

Per-provider connection guides

This page is the cross-provider reference. For the field-by-field connect steps, open the page for your store: modulexdb (managed), Qdrant, Pinecone, MongoDB Atlas, and Weaviate.

Managed: modulexdb

The managed store is the default. When you create a knowledge base without choosing an external provider, ModuleX provisions a modulexdb knowledge base: it parses, chunks, and embeds your documents, stores the chunk vectors in its own managed vector store, and serves vector, hybrid, and RAG-context search over them. You do not connect anything — the store is created for you, and an internal credential is linked to the knowledge base automatically (the “native knowledge base = credential” pattern). A knowledge base counts as managed when its embedding_config.integration_name is modulexai. Managed retrieval and managed ingest are metered in credits (see Retrieval billing below). Choose modulexdb when you want ModuleX to own ingestion and storage end to end and you do not already run a vector database.

External: bring your own vector store

An external provider connects ModuleX to a vector database you already operate. ModuleX does not ingest or store documents for an external provider — you populate and maintain your own collection/index, and ModuleX only queries it at run time. Because ModuleX provides no storage and no embeddings for these, external retrieval is uncosted (your vector-database vendor and your embedding provider bill you directly — this is bring-your-own-key usage; see BYOK in the glossary). All four external providers connect with a custom auth credential. Each declares its own connection fields and exposes catalog actions (query, plus a list/describe pair) so the catalog UI can show what the provider can do. Choose an external provider when you already have vectors in Qdrant, Pinecone, MongoDB Atlas, or Weaviate and want to retrieve over them without re-ingesting into ModuleX.

Important capability boundary. External providers query an existing collection; they have no ModuleX-side ingestion pipeline. The document upload, chunking, embedding, and processing flows described in Managing documents apply to managed knowledge bases only. You are responsible for keeping an external collection populated and its embedding model consistent with the one you configure on the ModuleX side.

How a provider is connected

The connection mechanism differs by family.

Managed (modulexdb): create a knowledge base

Create a knowledge base — in the app under Knowledge, or with POST /knowledge-bases. ModuleX provisions the managed store, auto-discovers an embedding-capable credential if you do not supply one, and links an internal modulexdb credential to the new knowledge base. There is nothing else to connect. See Build a RAG knowledge base.

External: create a credential for the provider

Create a custom credential for the provider with its connection fields (URL, API key, connection string — see the per-provider tables below), in the app under Settings → Credentials, or with POST /credentials. ModuleX encrypts the secret at rest and tests the connection before saving where a test endpoint is declared. See Managing credentials.

Reference the provider from a knowledge node

In a workflow, add a Knowledge node, set its provider_type, and point credential_id at the credential (or, for managed, the auto-linked knowledge-base credential). For external providers, also set collection_name and an embedding_config. The node configuration is documented in The knowledge node contract below.

Creating an external-provider credential

External providers use the custom auth schema. Create the credential with the provider’s name and an auth_data object holding its fields. Auth in every request follows the standard model: Authorization: Bearer mx_live_… plus X-Organization-ID.

curl -X POST https://api.modulex.dev/credentials \
  -H "Authorization: Bearer mx_live_xxx" \
  -H "X-Organization-ID: org_123" \
  -H "Content-Type: application/json" \
  -d '{
    "integration_name": "qdrant",
    "auth_type": "custom",
    "display_name": "Prod Qdrant",
    "make_default": true,
    "auth_data": {
      "url": "https://xyz-abc.aws.cloud.qdrant.io:6333",
      "api_key": "qdrant-api-key-..."
    }
  }'

from modulex import ModuleX

client = ModuleX(api_key="mx_live_xxx", organization_id="org_123")

credential = client.credentials.create(
    integration_name="qdrant",
    auth_type="custom",
    display_name="Prod Qdrant",
    make_default=True,
    auth_data={
        "url": "https://xyz-abc.aws.cloud.qdrant.io:6333",
        "api_key": "qdrant-api-key-...",
    },
)

import { ModuleX } from "modulex";

const client = new ModuleX({
  apiKey: "mx_live_xxx",
  organizationId: "org_123",
});

const credential = await client.credentials.create({
  integration_name: "qdrant",
  auth_type: "custom",
  display_name: "Prod Qdrant",
  make_default: true,
  auth_data: {
    url: "https://xyz-abc.aws.cloud.qdrant.io:6333",
    api_key: "qdrant-api-key-...",
  },
});

The exact SDK method surface for credential creation may vary between the JavaScript and Python SDKs; the credential resource is not guaranteed to be at parity across both. Confirm against the SDK ⇄ API parity matrix and fall back to the REST call above when a method is absent. The cURL request is the authoritative shape.

Connection fields by provider

Each external provider declares its connection fields in its manifest under a custom auth schema. The fields below are the exact auth_data keys you supply.

Qdrant
Pinecone
MongoDB Atlas
Weaviate

string

required

URL of your Qdrant instance, including the port. Not treated as a secret. Example: https://xyz-abc.aws.cloud.qdrant.io:6333.

string

API key for Qdrant Cloud. Optional for local/unauthenticated instances. Stored encrypted. Example format: qdrant-api-key-....

ModuleX validates the connection by listing collections (GET {url}/collections, expecting a result field in the response). Catalog actions: query, list_collections, get_collection_info.

string

required

Your Pinecone API key. Stored encrypted. Example format: pcsk_....

string

required

Pinecone environment. Not treated as a secret. Examples: us-east-1-aws, gcp-starter.

ModuleX validates the connection by listing indexes (GET https://api.pinecone.io/indexes, expecting an indexes field). Catalog actions: query, list_indexes, describe_index. Pinecone queries accept an optional namespace.

string

required

MongoDB Atlas connection string. Stored encrypted (it embeds your credentials). Format: mongodb+srv://user:password@cluster.mongodb.net/.

ModuleX validates by listing databases. Catalog actions: query, list_databases, list_collections. A query targets a database, collection, vector-search index_name, and the path field that holds the vectors.

string

required

URL of your Weaviate instance. Not treated as a secret. Example: https://your-cluster.weaviate.cloud.

string

API key for Weaviate Cloud. Optional for local/unauthenticated instances. Stored encrypted. Example format: weaviate-api-key-....

ModuleX validates by checking cluster status. Catalog actions: query, list_classes, get_class_stats. Weaviate is the one provider that can search from text directly (nearText) when a text2vec module is enabled, so a query embedding may be optional; otherwise supply a query_vector.

The connection fields above arrive in each manifest under a fields array, not the setup_environment_variables array used by tool and LLM-provider auth schemas. If you read the raw manifest from the catalog API, expect fields for the four knowledge providers. The manifest’s provider_type value (external) is dropped during catalog sync and never appears in API responses — do not rely on it.

Browsing providers in the catalog

The catalog exposes knowledge providers as read-only metadata for discovery; it is not the execution surface. List providers with GET /integrations/knowledge-providers or fetch one with GET /integrations/knowledge-providers/{provider_name}. Both require the owner or admin role and X-Organization-ID. The detail endpoint returns the provider’s actions and its auth_schemas (with OAuth flags enriched where applicable — not relevant to these custom-auth providers).

curl 'https://api.modulex.dev/integrations/knowledge-providers' \
  -H 'Authorization: Bearer mx_live_xxx' \
  -H 'X-Organization-ID: org_123'

from modulex import ModuleX

client = ModuleX(api_key="mx_live_xxx", organization_id="org_123")
providers = client.integrations.list(type="knowledge_provider")

import { ModuleX } from "modulex";

const client = new ModuleX({ apiKey: "mx_live_xxx", organizationId: "org_123" });
const providers = await client.integrations.list({ type: "knowledge_provider" });

string

Unique provider id, e.g. qdrant, pinecone, mongodb_atlas, weaviate.

string

Human-readable name, e.g. MongoDB Atlas Vector Search.

string

Short summary of the provider.

string

Always knowledge_provider for these entries.

string[]

Tags such as Vector Database, semantic-search.

object[]

The provider’s catalog actions (returned by the detail endpoint), each with name, description, parameters, and a derived output_schema.

Show action object

string

Action id, e.g. query, list_indexes.

string

What the action does.

object

Map of parameter name to {type, description, required, default}.

object

JSON-schema description of the action’s result shape.

object[]

The auth variants the provider supports. For all four external providers this is a single custom schema whose connection fields live under fields (see the warning above).

The catalog SDK method names and exact filter arguments may differ between SDKs and are not guaranteed to be at parity. Treat the cURL request as authoritative; verify SDK signatures against the SDK ⇄ API parity matrix.

The knowledge node contract

A Knowledge node is how a provider is consumed inside a workflow. The node configuration (KnowledgeNodeConfig) is shared across families; a handful of fields apply only to external providers.

string

required

Credential for the knowledge base. For managed, this is the internal credential auto-linked to the knowledge base; for external, the custom credential you created for the provider.

string

default:"modulexdb"

One of modulexdb, qdrant, pinecone, weaviate, mongodb_atlas.

string

required

The search query. Supports {{nodeId.path}} references to upstream node outputs for dynamic queries.

boolean

default:"false"

When true, use the workflow input as the query instead of the query field.

string

Collection/index/class name on the external store. Required for external providers; ignored by modulexdb.

string

Namespace for Pinecone (or similar). External only.

integer

default:"5"

Number of results to retrieve. Range 1–50.

number

default:"0.3"

Minimum similarity-score threshold, 0.0–1.0. Score scales are normalized per provider, so the effective floor differs by vector store.

integer

default:"2000"

Maximum tokens in the formatted context string (for the context output format). Range 100–10000.

object

Provider-specific filter conditions applied to the search.

string[]

Restrict retrieval to specific document ids. Managed knowledge bases only.

string

default:"context"

One of chunks (individual chunks with metadata), context (a single formatted RAG context string), or both.

boolean

default:"true"

Include chunk metadata in results.

boolean

default:"true"

Include source-document info in the formatted context.

object

Embedding settings for providers that do not embed internally. Required for external vector databases (Qdrant, Pinecone, MongoDB Atlas, and Weaviate when not using a text vectorizer), because ModuleX must turn the query text into a vector before searching your store.

Show embedding_config object

string

required

Embedding provider integration, e.g. openai, cohere.

string

required

Provider identifier, e.g. openai.

string

required

Embedding model id, e.g. text-embedding-3-small.

string

Credential for the embedding provider. If omitted, the organization’s default credential for that integration is used.

Match the embedding model in embedding_config to the model your external collection was indexed with. ModuleX embeds the query with the model you configure; if it differs from the one used to build the collection (or the dimension does not match), similarity scores are meaningless and retrieval returns poor or empty results. For managed (modulexdb) knowledge bases this is handled for you — ingest and query use the same configured embedding model.

Known limitation. External-provider retrieval is implemented in the workflow engine’s knowledge node, which embeds the query (per embedding_config) and calls the provider adapter. If an external-provider knowledge node returns “External provider not yet supported”, retry it through the standard knowledge node path, which fully supports external providers. Managed (modulexdb) retrieval is supported on every path.

Retrieval billing

Whether a provider is metered depends entirely on the family.

Operation	Managed (modulexdb)	External (BYOK)
Document ingest (parse → chunk → embed)	`FILE_INGEST_BASE = 1` credit, plus per-chunk embedding token cost	Not applicable — you ingest into your own store
Retrieval (search / hybrid / retrieve-context)	`RETRIEVAL_BASE = 1` credit, plus query-embedding token cost	Uncosted by ModuleX

A knowledge base is billed only when it is managed — that is, when embedding_config.integration_name == "modulexai". One credit equals $0.01. External (BYOK) retrieval is not credited; your vector-database vendor and your embedding provider bill you directly. For the full credit model see Credits & metering.

On the interactive knowledge API, a managed-store credit or rate denial returns the flat billing DenialEnvelope as 402 / 403 / 429 (reject-before-write — no search runs). Inside a workflow run, the managed-retrieval gate is best-effort so a billing hiccup never crashes an in-flight run. The error envelope shapes are documented on Errors & status codes; the gate itself on Usage gating & limits.

Next steps

Managed knowledge (modulexdb)

Set up ModuleX-hosted storage and retrieval, billed in credits.

Connect an external store

Bring your own Qdrant, Pinecone, MongoDB Atlas, or Weaviate.

Knowledge node

Wire a provider into a workflow and shape its output.

Build a RAG knowledge base

End to end: create a knowledge base, ingest documents, and query it.

Using

Providers

Building

Tools

Knowledge providers: managed and external vector stores

The two provider families

Per-provider connection guides

Managed: modulexdb

External: bring your own vector store

How a provider is connected

Creating an external-provider credential

Connection fields by provider

Browsing providers in the catalog

The knowledge node contract

Retrieval billing

Next steps

Managed knowledge (modulexdb)

Connect an external store

Knowledge node

Build a RAG knowledge base

​The two provider families

Per-provider connection guides

​Managed: modulexdb

​External: bring your own vector store

​How a provider is connected

​Creating an external-provider credential

​Connection fields by provider

​Browsing providers in the catalog

​The knowledge node contract

​Retrieval billing

​Next steps

Managed knowledge (modulexdb)

Connect an external store

Knowledge node

Build a RAG knowledge base

The two provider families

Managed: modulexdb

External: bring your own vector store

How a provider is connected

Creating an external-provider credential

Connection fields by provider

Browsing providers in the catalog

The knowledge node contract

Retrieval billing

Next steps