Skip to main content
Every run in the Komodor Agentic Operation Platform (KAOP) already keeps its own durable evidence, but that evidence lives in KAOP. This page shows how to also emit the same activity as standard OpenTelemetry (OTel) traces, logs, and metrics to your own backend, what the spans and metrics contain, and how the two views complement each other.

​
Two places the same activity shows up

Export does not replace the in-product evidence. It puts agent behavior on the same screen as the services those agents operate on, so an agent’s tool call and the upstream latency spike it caused sit in one trace.

​
Enabling export

Instrumentation ships with both the control plane and every worker built on the Python SDK, and it does nothing until you point it at a collector. There is no KAOP-specific configuration format — it reads the standard OpenTelemetry environment variables, so it works with any OTLP-compatible backend and there is nothing to unlearn.
1

Point the worker at a collector

Set OTEL_EXPORTER_OTLP_ENDPOINT on the worker’s process or container. Setting it is what turns export on.
2

Add whatever your backend needs

Most managed backends authenticate with a header.
3

Name the service, if the default is not what you want

A worker defaults its own service name to agentops-worker-<agent>, derived from the agent’s name. Set OTEL_SERVICE_NAME to override it.
4

Restart the worker and look for the first trace

Trigger a run. The trace appears in your backend under the service name above, rooted at the run’s own span.
Other standard OTEL_* variables are honored as well.
Export is off until an endpoint is set, and nothing leaves your worker before then. A worker run from a laptop with no OTel configuration pays nothing for the instrumentation being present.

​
What a worker emits

​
Spans

A run produces a span tree you can read as the agent’s decision path. Every worker span also carries resource attributes identifying the worker and the agent it runs: agentops.worker.id, agentops.worker.name, and agentops.agent.id.

​
Metrics

All KAOP metrics live under a single agentops. namespace, so one facet in your backend isolates the whole product.

​
Logs

A worker’s log records are exported alongside its spans, and each record carries the trace and span it belongs to — so you can pivot from a slow span to the lines the agent wrote while it was executing.
Worker log lines also reach KAOP independently of OTel, which is why a run’s evidence shows its logs whether or not you export anything. Komodor has no access to a worker you host, so the worker ships its own lines to the control plane as part of the run.

​
Choosing what a worker forwards to KAOP

Two variables control the KAOP-side log forwarding, independent of OTel export: Raise AGENTOPS_CP_LOG_LEVEL on a chatty worker if run evidence is noisier than it is useful; set AGENTOPS_CP_LOG_CAPTURE_ALL while debugging a worker whose own libraries hold the answer.

​
What the control plane emits

The control plane is instrumented the same way and turns on with the same variables. Its spans carry the resolved caller identity, so traces are sliceable per workspace and per principal, and authorization decisions appear as their own spans with the action, resource, and decision attached. Two attribute namespaces are used and only two:
  • usr.* for human principals — the user id, email, and name.
  • agentops.* for everything else — the account, the principal, the authentication path, and authorization decisions.
Identity attributes are stamped on the root span. Two opt-out settings widen that to child spans: AGENTOPS_OTEL_SPAN_BAGGAGE_ENABLED propagates non-identifying ids (account and principal) onto every span in the trace, and AGENTOPS_OTEL_ALL_SPANS_IDENTITY_ENABLED adds the principal kind and display name. Email and name deliberately stay on the root span only, to bound how much personal data reaches your telemetry store.

​
What to build with it

  • Alert on agent failure rate the way you alert on service error rate. agentops.run.failed against agentops.run.completed, split by agent.
  • Watch model latency separately from agent latency. A run that got slower because one provider got slower is a different problem from a run that took more turns.
  • Correlate a tool call with the system it touched. The agentops.tool.call span sits in the same trace as your own service’s spans when your instrumentation propagates context.
  • Track token volume as an operational signal. Token counters move before a spend chart does.
Do not use exported metrics as your billing record. Spend attribution is computed in KAOP from each run’s reported usage, including its cost source classification — see Agent spend attribution. Token counters tell you volume; they do not tell you money.

​
Next steps

Runs & evidence

The in-product view of the same activity, and what a span tree contains.

Fleet health

The rollup KAOP builds from run activity.

Agent spend attribution

Where cost comes from and how it is attributed.

Python SDK

Building a worker — instrumentation is included, not added.