> ## Documentation Index > Fetch the complete documentation index at: https://docs.komodor.com/llms.txt > Use this file to discover all available pages before exploring further. # Troubleshooting > The problems people actually hit — offline agents, 401s, missing credentials, integration failures, stuck runs — and the checks that fix them. This page collects the failures that come up most often in the Komodor Agentic Operation Platform (KAOP), each written as what you see followed by what to check, in the order that resolves it fastest. Almost every case below is diagnosable from a run's own evidence or from an agent's presence record, so the checks tell you where to look rather than what to guess. Before working through a section: if the same failure has happened more than once, Insights has already grouped it, categorized it from structural proof, and attached the failure reason verbatim. Start there and you often skip the diagnosis entirely. ## An agent shows offline **What you see:** the agent sits in **Fleet** as **Offline**, or a draft never goes live. Presence is a round trip, not a report: the control plane pings each open worker connection every **30 seconds**, and the worker's reply is the only thing that updates presence. An agent reads offline once nothing has been heard for **90 seconds** — three missed pings. A draft goes live automatically on its worker's first contact; there is no separate activation step. So "offline" almost always means the worker is not reaching the control plane. Check, in order: Read the worker's own startup logs. A worker that failed to boot never registers, and the agent stays a draft rather than going offline. Workers are outbound-only — they dial the control plane over HTTPS and nothing needs to reach into them. Confirm `AGENTOPS_URL` points at your workspace host and that outbound HTTPS is allowed from wherever the worker runs. A worker authenticates with `Authorization: Bearer `. If the token was rotated or revoked, the old value stops working immediately. Generate a fresh one from the agent's **Permissions** tab, under **Worker token**, and restart the worker with it. A worker may also send `X-Agent-Id`. When it does, the value must be the same agent the token belongs to — a mismatch is refused. If in doubt, omit the header and let the token identify the agent. The agent's **Metadata** tab shows **Last heartbeat**. A timestamp from two minutes ago and one from last week are different problems. A revoked token and a mismatched agent id both return the same `401`, deliberately — distinguishing them would let anyone probe which tokens exist. The specific cause is in the control plane's logs, not in the response. ### Online, but nothing runs An agent can be present and still unable to receive work. Two cases, both visible on [Fleet health](/manage-your-agents/optimize/fleet-health): * **Online, 0 live replicas** — the agent is heartbeating but has nothing serving. This is the normal reading for an agent whose candidate arm is alive while its production arm is dead. * **Heartbeat-online, channel-unreachable** — presence is fine but the delivery channel is not connected. A run fails with a message saying the worker is online but never connected a channel. Restart the worker. ## API or webhook requests return 401 or 403 **What you see:** a JSON `401` or `403` instead of the response you expected. | Caller | What it must send | What a rejection means | | ------------------ | ------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **API client** | `Authorization: Bearer ` | The key is missing, malformed, expired, or revoked — or the identity behind it is. A key stops working the moment its member is suspended or its service account is disabled. | | **Worker** | `Authorization: Bearer `, optionally `X-Agent-Id` | See the offline-agent section above. | | **Webhook sender** | The endpoint's configured credential — by default `X-Webhook-Token` | The token or signature did not match. If the token was rotated, the old one is dead immediately; copy the new one from the endpoint. | Webhook endpoints verify the *sender's* credential, not an API key. Two defaults are worth knowing: a GitHub-type endpoint expects `X-Hub-Signature-256`, and a bearer-type endpoint expects `Authorization`. If the configured header is absent from a request, KAOP also accepts `Authorization: Bearer ` as a fallback — which is what makes alerting systems that cannot send custom headers work. A webhook whose signing credential has been deactivated fails verification even though the token value is unchanged. Check the credential's status, not just the token. Other `403` responses you may meet: cross-account views are read-only, some routes require operator or system-admin standing, and a suspended account is refused outright with its status attached. ## A run fails with a missing credential **What you see:** a run fails, or an agent reports that a tool is unavailable, and the reason names a credential. KAOP shows the failure reason **exactly as it arrived** — `Missing Datadog credentials. Set DD_API_KEY and DD_APP_KEY…` reads as that sentence rather than as a generic label. So the first move is to read the reason, not to interpret it. Then check, in order: * **Is the credential bound to this agent?** A credential is usable only by the agents it is bound to. Binding is per agent, not per workspace. * **Does the binding still resolve?** A connection whose credential was deleted or emptied surfaces as **credential missing** on the connection's health badge. * **Is it a bot-only connection?** Some MCP servers sign in as the person rather than as the app. If the connection was authorized for the app only, KAOP says so and asks you to reconnect the integration including your own user access, then test again. * **Are you inside a run?** Credentials are delivered to a run. A search or tool helper called outside a run scope has no transport to ask over and will say so. KAOP never exposes a credential's value — not to you, not to the agent's logs, and not into run evidence. A run that needs a secret receives it in memory for the duration of the run. That is why diagnosing a credential problem is about *bindings and status*, never about reading the value back. ## An integration or MCP server is failing **What you see:** a tool call fails, or a connection carries a warning badge. A connection has two independent facts, and reading the wrong one sends you the wrong way: * Its **lifecycle status** — pending, active, error, revoked, or disabled. * Its **last check** — whether the credential actually worked the last time it was exercised. The badge on a connection reports the second, and the labels are precise: | Badge | What it means | | ------------------------------ | ------------------------------------------------- | | **credential verified** | It worked, checked within the last day. | | **verified earlier** | It worked, but the check is more than a day old. | | **not checked** | It has never been verified. | | **permission denied** | Authentication or authorization failed. | | **unreachable** | The provider could not be reached. | | **timed out** | The provider did not answer in time. | | **provider returned an error** | It reached the provider, which returned an error. | | **no usable credential** | There is nothing to authenticate with. | | **cannot be verified** | No safe probe exists for this provider. | | **not working** | It failed, with no more specific reason. | | **unproven** | The check was skipped. | **Nothing re-checks a connection in the background.** A badge reports the last time the connection was actually exercised — by its creation, by a **Test**, or by a change to it. A **credential verified** badge from three weeks ago is a record of three weeks ago. Press **Test** before you conclude a connection is fine. Reconnecting or updating a connection's secret clears the previous verdict, so the badge returns to unchecked until you test again. ## A trigger does not fire **What you see:** a schedule or an endpoint exists, but no run appears. * **Is the trigger enabled?** A schedule is only registered while it is enabled and has a cron expression. Disabling it unregisters it without deleting it. * **Is the agent enabled?** A disabled agent does not fire triggers. It still answers manual and chat invocations, which is why a disabled agent can look healthy while its schedule is silent. * **Is the cron expression what you meant?** An invalid expression is rejected at write time with the expression quoted back at you. So is a cron-and-timezone pair with no single UTC equivalent, which is how daylight-saving edge cases surface. * **Did the request pass verification?** A request that fails the endpoint's authentication is rejected before any run is created — see the `401`/`403` section above. * **Is the target enabled?** A request aimed at a disabled agent or a disabled workflow is refused with a conflict rather than queued. * **Is the body the right shape?** For an endpoint, the **whole request body** is the agent's input — unlike the invoke and schedule APIs, which wrap the object under an `input` key. * **Does the target exist?** A missing endpoint, trigger, agent, or workflow is a not-found response, which is worth distinguishing from an authentication failure. A channel rule fires on a mention by default, and messages posted by bots do not fire it unless the rule explicitly allows bots. That is the usual reason an automated Slack post is ignored while a human's identical message works. ## A run is stuck or failed with no obvious cause **What you see:** a run sits queued, or fails before any agent work appears. KAOP classifies these from structural signals, so the category tells you where to look: | Category | What it proves | Where to go | | ------------------------ | ------------------------------------------------------------------ | --------------------------------------------------------------- | | **Agent run failed** | The agent ran and its handler raised. | The run's evidence — the spans and logs show what it was doing. | | **Worker unreachable** | The run failed with no agent span at all. The agent never started. | Fleet health, then the offline-agent section above. | | **Tool call failed** | A tool span ended in error. | The failing integration's health, above. | | **Stuck unclaimed** | It sat queued past its lease without a worker claiming it. | Fleet health — usually no online worker for that agent. | | **Worker offline** | Work is queued or claimed while presence is down. | Fleet health — bring the fleet back or scale it up. | | **Unclassified failure** | It failed with no structural proof of a cause. | The run's evidence. The verbatim reason is the lead. | Two timeouts explain most runs that end without a cause: * A run that nothing can serve is failed after it has waited too long in the queue — and much sooner when the target agent is already known to be offline, draft, or failed to deploy. * A run that has been claimed is failed on an absolute wall clock, so a worker that hangs cannot hold a run forever. ## Knowledge search returns nothing **What you see:** an agent's knowledge search comes back empty, or the Knowledge page warns that search is unavailable. * **No embedding provider configured.** Semantic search needs one for your deployment; without it nothing can be indexed or searched, and the search returns an explicit message saying the embedding backend is not configured or could not be reached. Ask your Komodor contact to configure one. * **The document is not indexed yet.** A freshly uploaded document is indexing until processing finishes, and only indexed documents are searchable. A document in error never indexed — re-index it. * **The filters are too narrow.** Searches can be restricted by service, environment, document type, and sensitivity. An over-specific filter can exclude every match. * **Wrong knowledge base.** Search is scoped to your workspace, and an agent retrieves only from the knowledge bases it is meant to use. An empty result is not an error. When search is working and nothing matched, you get an empty list — which is a different fact from search being unavailable, and the response distinguishes them. ## Errors you will meet, by status code | Code | What it means here | | ----- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `400` | A malformed value — for example an API key expiry that is not an ISO 8601 timestamp, carries no timezone offset, or is in the past. | | `401` | Authentication failed. Which credential depends on the caller — see above. | | `403` | Authenticated but not permitted, or a webhook credential that did not verify. | | `404` | The thing does not exist, or a capability is not enabled for your workspace. A view that is off reads as not found rather than as forbidden. | | `409` | A conflict or a quota. Creating an agent past the cap is a `409` — see [Limits & quotas](/reference/limits-quotas). | | `413` | A payload over its size limit — a knowledge document, a batch, or a run artifact. | | `422` | Validation failed. An invalid cron expression and a malformed worker registration both land here. | | `503` | A dependency was briefly unavailable, such as the credential store. Retry. | There is no rate limiting in KAOP's control plane, and nothing returns `429`. If you see a `429`, it came from an upstream provider an agent called, not from KAOP. ## Still stuck? Gather the run id and the exact error text, then see [Support](/reference/support) for what else to include. ## Next steps Whether what you hit was a cap rather than a bug. How to read the trail a failing run left. What to send, and where. ## Related topics - [Incident management & troubleshooting](/modules-and-workflows/sre/incidents.md) - [Modules & workflows](/modules-and-workflows/overview.md) - [Knowledge graph](/manage-your-agents/optimize/knowledge-graph.md) - [Operating a self-hosted install](/get-started/on-prem/operating.md) - [What's new](/reference/changelog.md)