Alert Intelligence

Alert Intelligence in the Komodor Agentic Operation Platform (KAOP) owns the quality of your alerting itself: which monitors fire usefully, which fire constantly and get ignored, and what to change about each one. Every monitor ends up carrying a classification and the rationale behind it, so the list doubles as a work queue.

What the module takes on

Monitors drift: a threshold set correctly stops matching the service it watches, and the monitor fires often enough to be acknowledged without being read. Deciding which ones to change means pulling each monitor’s firing history and classifying what it fired on, then proposing a specific change against that evidence. The module does that continuously and hands you the proposal — a tuned threshold, a dedupe rule, a suppression window — with the history behind it.

Monitors from every source, in one list

Monitors are read from every connected monitoring provider — Datadog, PagerDuty, Grafana, Opsgenie, CloudWatch, New Relic — and consolidated into one list, with the number of sources folded in shown beside the monitor count. Consolidation surfaces a class of duplicate page that is otherwise invisible. The same underlying condition alerting from two different tools cannot be detected from inside either one; it is only apparent to something reading every source together.

How a monitor is classified

Each monitor carries four measurements: what share of its fires were true positives, what share were actionable, how often it fired and flapped over the last seven days, and its acknowledgement rate. Correctness and actionability are independent axes, and a high acknowledgement rate beside a low true-positive rate shows a monitor being cleared without being read. On top of those, each monitor carries one classification — true positive, false positive, actionable, not actionable, or mixed — and the list filters on it as a single facet. A decision is made on the percentages rather than on the label alone.

What a proposal contains

A classification produces a proposed change, and the proposal is specific enough to argue with. It names the kind of optimization — tune a threshold, widen the evaluation window, dedupe, open a silence window, split the monitor, or retire it — then the concrete change, the estimated benefit, and the risk that comes with it. Quieting a monitor always costs something, and a proposal that says what it costs is a decision you can make; one that only promises fewer pages is a decision you are guessing at. A threshold raised with a deploy grace window buys a large drop in paging and pays for it in slower detection of mild regressions — both halves are stated, and both are yours to weigh. Each proposal carries a state of its own, so the list works as a queue rather than a report you read once and close. A ready proposal can be applied directly or queued for approval, which moves it to Fix pending. The agent that produced it is named on it. Which of the two a given operator may do is governed by their role — see Roles & permissions.

The record of past changes

An alerting change is a hypothesis about what should page someone, and its effect is measurable. Change history records each one with its date, what changed, who changed it — a person or the tuning agent — the pages and fires before and after, and a verdict on whether it worked. The record keeps the changes that made things worse alongside the ones that worked. A change that looked reasonable when it was proposed is legible as a mistake only from the before-and-after.

Measuring the result

A page reduction over the last 30 days is reported against the fixes actually applied, with the count of fixes behind it. Alongside sit the figures that make it legible: monitors awaiting a decision, monitors classified as false-positive or not actionable, and monitors classified as actionable or true positive.

The workflow

The module ships one workflow, Alert noise reduction: an intake step backfilling each monitor’s fire, page and flap history; an analysis step classifying every monitor and recording the rationale; and an optimization step proposing the change, with the trade-off it carries stated so you can see what you lose by quieting it. The workflow carries its own operating metrics on the Workflows tab: bound agents, monitors covered, accuracy, 24-hour cost, latency and error rate.

Prerequisites

Enabling it

Alert Intelligence is enabled per account rather than shipped on. It appears in the AI SRE family under Modules → Overview; enabling it adds Alert Intelligence to the main menu alongside its monitor list. Until a provider is connected and an agent is bound to each step, the module is enabled but inert — the monitor list is empty because nothing has read a provider yet.

Next steps