Skip to main content
Every agent run costs a model call, and Providers is where you see which models this account can call, what each is costing, and whether it is working. The Komodor Agentic Operation Platform (KAOP) ships with managed models so agents run without any setup. You can also register your own provider account or route a self-hosted agent directly through an LLM gateway you manage.

Three tiers, one table

The page is one flat table of models, not a tab per tier. Each row is a model with its provider, who manages it, its upstream model id, its spend, its usage, and its status — so the tier is just a column you can sort and filter by. A registered account with no models yet still gets a row, so it is visible rather than something you have to remember exists. Above the table sits the account’s gateway budget. That strip is the only place the managed tier’s account-level numbers appear.

Bringing your own provider

Supported providers are OpenAI, Anthropic, Azure OpenAI, Azure AI Foundry, Bedrock, Vertex AI, Gemini, Mistral, Groq, DeepSeek, OpenRouter, Together AI, xAI, Cohere, Ollama, Baseten, Nebius Token Factory, Nscale, and a hosted vLLM endpoint.
Azure AI Foundry is a separate provider from Azure OpenAI, not another name for it — Foundry is a different API. This matters for Claude: a Claude deployment on Foundry speaks Anthropic’s Messages API but authenticates the Azure way, and only the Azure AI Foundry provider routes it correctly. A non-Claude model registered under the same key takes Foundry’s OpenAI-compatible path, so one key covers your whole Foundry catalog. Enter the API key and API base URL (https://<resource>.services.ai.azure.com) and the wizard lists the deployments that key can reach; an endpoint behind your own API gateway may publish no model list at all, so the model id can always be typed by hand. A Foundry resource reachable only inside your network can serve self-hosted agents through Managed by me instead.
1

Credential

Name the account and supply its API key. The key is stored so model discovery can authenticate with it later.
2

Models

Register one or more upstream model ids against that credential. Nothing is written until this step submits — the credential is created, then each model against it, as one action.
3

Verify

Each new deployment is health-checked, and you can talk to it directly to confirm it answers.
Verify is a step of its own rather than a result banner, because after the write “did this work” is a genuinely different question from “what should I create” — and a reader who has finished creating still has to answer it. To add models to an account you already registered, use Add models on that credential rather than walking the wizard again.
Registering your own provider does not replace the managed tier — both stay callable, and an agent picks a model from either.

Using your own LLM gateway

The Managed by me — your own LLM gateway tier is for self-hosted agents. The worker calls your gateway directly with your key: neither that inference key nor the inference request passes through Komodor’s model gateway. The upstream provider key stays inside your gateway as well. KAOP supports LiteLLM, Hosted vLLM, Ollama, and Azure AI Foundry gateways. The first three use an OpenAI-compatible interface. Azure AI Foundry uses its own discovery route and is distinct from registering a public Azure AI Foundry account under Add a new key.
Keeping Komodor out of the inference path does not turn off run reporting. Prompts, model spans, outputs, and other evidence emitted by the worker still follow the account’s normal data-handling rules.

Choose how KAOP reads gateway metadata

Both modes keep inference between your worker and your gateway. The difference is whether KAOP can read metadata from the gateway. The scoped key must be read-only and non-admin. It is stored encrypted, used only for metadata operations, never delivered to a worker, and never used for inference. Do not enter your gateway’s admin key or an upstream provider key.

Register a customer-managed gateway

1

Choose Managed by me

Open Providers, choose Add provider, then select Managed by me — your own LLM gateway. This tier can be used only by agents you host yourself.
2

Describe the gateway

Choose its kind and strict or scoped mode, then enter a source name and gateway base URL. For a private endpoint, select the Outpost that can reach it; leave the Outpost unset for a publicly reachable gateway.
3

Supply model access

In strict mode, enter the model ids and choose Add gateway. In scoped mode, enter a read-scoped key and choose Continue, select the discovered models your agents may use, then choose Save gateway.
An Outpost carries only metadata operations between KAOP and a private gateway: model discovery, verification, and spend reads. It does not proxy inference traffic — the self-hosted worker still reaches the gateway directly for every inference request, over its own network. Make sure the gateway host is allowed by the selected Outpost, and that the worker’s Kubernetes environment has its own network route to the gateway’s base URL. If the gateway uses a private or custom certificate authority, configure trust with caBundle.existingConfigMap — see A run fails with a certificate error.

Editing a gateway

Changing a gateway’s base URL carries to every agent already bound to it — each is repointed to the new address on its next run, with no redeploy needed. In an account running more than one gateway, only the agents on the one you edited move; an agent you already moved to a different gateway stays where it is. Each self-hosted worker needs network access to the new address.

Use the gateway from an agent

When you build or edit a self-hosted agent, select a model under Managed by me — including a self-hosted agent deployed from the catalog, whose provider can be changed after deploy the same way. Supply the Gateway API key in the wizard, or select I’ll provide the API key myself in the helm command and add it when you install the agent. Either way the key stays in your cluster; KAOP keeps no copy. In the chart, set llm.mode: byo and supply the key either by referencing a Secret you already manage, or by letting the chart render one for you:
With existingSecret set, the chart reads your Secret directly and creates no Secret for the key. Use llm.anthropicApiKey.value instead if you want the chart to render the Secret for you from an inline value. An agent on a Managed by Komodor or Your providers model uses a credential KAOP holds instead — set llm.mode: cloud with llm.credentialRef:
The worker fetches this credential from the control plane at boot; it never appears in your values file. Set llm.mode explicitly when switching between these — a helm upgrade that carries over a stale value from the other mode can bind the wrong credential.

Next steps

Credentials & secrets

How a provider key is stored and delivered to a run.

Outposts

Reach a private gateway without opening an inbound path.

Agent spend & attribution

What the fleet costs, attributed per run, agent, and model.

Model Evaluation

Decide whether a model is good enough for a given job.