Three tiers, one table
The page is one flat table of models, not a tab per tier. Each row is a model with its provider,
who manages it, its upstream model id, its spend, its usage, and its status — so the tier is just a
column you can sort and filter by. A registered account with no models yet still gets a row, so it
is visible rather than something you have to remember exists.
Above the table sits the account’s gateway budget. That strip is the only place the managed tier’s
account-level numbers appear.
Bringing your own provider
Supported providers are OpenAI, Anthropic, Azure OpenAI, Azure AI Foundry, Bedrock, Vertex AI, Gemini, Mistral, Groq, DeepSeek, OpenRouter, Together AI, xAI, Cohere, Ollama, Baseten, Nebius Token Factory, Nscale, and a hosted vLLM endpoint.Azure AI Foundry is a separate provider from Azure OpenAI, not another name for it — Foundry is a
different API. This matters for Claude: a Claude deployment on Foundry speaks Anthropic’s Messages
API but authenticates the Azure way, and only the Azure AI Foundry provider routes it
correctly. A non-Claude model registered under the same key takes Foundry’s OpenAI-compatible path,
so one key covers your whole Foundry catalog. Enter the API key and API base URL
(
https://<resource>.services.ai.azure.com) and the wizard lists the deployments that key can
reach; an endpoint behind your own API gateway may publish no model list at all, so the model id can
always be typed by hand. A Foundry resource reachable only inside your network can serve self-hosted
agents through Managed by me instead.1
Credential
Name the account and supply its API key. The key is stored so model discovery can authenticate
with it later.
2
Models
Register one or more upstream model ids against that credential. Nothing is written until this
step submits — the credential is created, then each model against it, as one action.
3
Verify
Each new deployment is health-checked, and you can talk to it directly to confirm it answers.
Registering your own provider does not replace the managed tier — both stay callable, and an agent
picks a model from either.
Using your own LLM gateway
The Managed by me — your own LLM gateway tier is for self-hosted agents. The worker calls your gateway directly with your key: neither that inference key nor the inference request passes through Komodor’s model gateway. The upstream provider key stays inside your gateway as well. KAOP supports LiteLLM, Hosted vLLM, Ollama, and Azure AI Foundry gateways. The first three use an OpenAI-compatible interface. Azure AI Foundry uses its own discovery route and is distinct from registering a public Azure AI Foundry account under Add a new key.Choose how KAOP reads gateway metadata
Both modes keep inference between your worker and your gateway. The difference is whether KAOP can read metadata from the gateway.
The scoped key must be read-only and non-admin. It is stored encrypted, used only for metadata
operations, never delivered to a worker, and never used for inference. Do not enter your gateway’s
admin key or an upstream provider key.
Register a customer-managed gateway
1
Choose Managed by me
Open Providers, choose Add provider, then select Managed by me — your own LLM
gateway. This tier can be used only by agents you host yourself.
2
Describe the gateway
Choose its kind and strict or scoped mode, then enter a source name and gateway base URL. For a
private endpoint, select the Outpost that can reach it; leave the Outpost unset for a publicly
reachable gateway.
3
Supply model access
In strict mode, enter the model ids and choose Add gateway. In scoped mode, enter a
read-scoped key and choose Continue, select the discovered models your agents may use, then
choose Save gateway.
caBundle.existingConfigMap — see
A run fails with a certificate error.
Editing a gateway
Changing a gateway’s base URL carries to every agent already bound to it — each is repointed to the new address on its next run, with no redeploy needed. In an account running more than one gateway, only the agents on the one you edited move; an agent you already moved to a different gateway stays where it is. Each self-hosted worker needs network access to the new address.Use the gateway from an agent
When you build or edit a self-hosted agent, select a model under Managed by me — including a self-hosted agent deployed from the catalog, whose provider can be changed after deploy the same way. Supply the Gateway API key in the wizard, or select I’ll provide the API key myself in the helm command and add it when you install the agent. Either way the key stays in your cluster; KAOP keeps no copy. In the chart, setllm.mode: byo and supply the key either by referencing a Secret you already
manage, or by letting the chart render one for you:
existingSecret set, the chart reads your Secret directly and creates no Secret for the key.
Use llm.anthropicApiKey.value instead if you want the chart to render the Secret for you from an
inline value.
An agent on a Managed by Komodor or Your providers model uses a credential KAOP holds
instead — set llm.mode: cloud with llm.credentialRef:
llm.mode explicitly when switching between these — a helm upgrade that carries over a
stale value from the other mode can bind the wrong credential.
Next steps
Credentials & secrets
How a provider key is stored and delivered to a run.
Outposts
Reach a private gateway without opening an inbound path.
Agent spend & attribution
What the fleet costs, attributed per run, agent, and model.
Model Evaluation
Decide whether a model is good enough for a given job.