Before you start
- Slack is connected to your KAOP account, through KAOP’s Slack app or your own Slack app connected for chat. Both work the same way. See Connecting Slack. If you connected your own Slack app for chat, mention that app wherever this page says KAOP’s Slack app.
- The app is in the channel.
- The app answers you there. If your Slack connection is limited to certain users or channels, KAOP replies only to those.
- At least one incident workflow is active. See Incident management & troubleshooting.
- Your Slack account is linked to your KAOP user. The first time you start, stop or ask about an incident, KAOP replies with a connect prompt that only you can see. That request is not kept: once you have linked, mention the app again. A Stop pressed before you linked can stay on Stopping…, so stop the incident from its incident page instead. Slack Connect guests from another organization cannot act on incidents.
Roles and custom grants are described in
Roles & permissions.
Start an incident
1
Mention the app with incident
In a channel or a thread, mention KAOP’s Slack app, then
incident, then what’s wrong:
incident checkout-api returns 500s since the 10:30 deploy. incident must be the first word
after the mention. Under an alert another app posted, such as Datadog or Grafana, reply in the
alert’s thread with the mention and incident alone: the alert says what’s wrong.2
Check the start card
KAOP reacts with 👀 and posts a card only you can see: Start an incident? It names the
workflow KAOP picked and why, and under an alert it also shows the alert’s labels and severity.
The dropdown lists the active workflows you can start, so you can pick another.
3
Press Start
Nothing is created until you press Start. Cancel removes the card. If you edit your
message before pressing Start, KAOP uses the edited text.
- The only workflow you can start.
- Under an alert, the workflow whose alert source is the app that posted it.
- The workflow last started in this channel.
Starting under an alert
An incident started under an alert takes the alert’s title, severity and labels, so a workflow’s filter on those fields sees what the alert says. Labels are the alert’skey:value tokens, such as
env:prod. Your own words become the incident’s description.
When KAOP does not start a new incident
Follow it in the thread
The incident reports in the thread you started it from, with one short reply at each milestone: when it starts, when it finds a likely cause, when the agent has a question, when a fix is applied, and when it ends. If an agent’s tool call needs approval, its approval card is posted in the thread as well. An incident opened by an alert routed to a workflow reports in the alert’s thread the same way. If a Slack notification sink handles the workflow’s questions, the questions and approval cards go to that sink’s channel instead, and this thread gets a note pointing there. An incident also keeps a thread in each Slack channel a notification sink posts to; see Notification sink threads. The reaction on your message shows where it stands, one at a time: 👀 picked up, ⏳ working, ✅ resolved, ❌ failed or stopped, 🟡 needs review. Slack’s own status under the thread shows KAOP working while the investigation runs, with Slack’s Stop button beside it. It clears while the incident waits for a person and when it ends. A live line in the thread shows what the investigation is doing: what it is checking, which agent and for how long, or that it waits for a decision, for up to 2 hours. When it ends, the line reads 🏁 Investigation finished in time, or ⏹ Investigation stopped after time.
If the workflow also notifies a Slack channel, that channel gets the full summary when the workflow
reaches its conclusion. When it is the channel the incident was started in, the conclusion is posted
once, in the thread. A resolve by a person, from the thread, the incident page or the co-pilot,
reaches the incident’s thread only, not the workflow’s notification channel.
Continue, Resolve and Retry
Continue picks the investigation up from the step that stopped, Retry starts a workflow that failed to start again, and Resolve closes the incident. A button acts as the person who presses it: it needs a linked Slack account and permission to manage incidents, and anyone else gets a private refusal. Once pressed, the buttons give way to Continued by, Retried by or Resolved by and the person’s name. If two people press at once, only one press goes through. Every press is recorded in the audit log under the person’s name.Stop it
While KAOP is working, press Slack’s Stop button under the thread. KAOP stops the running step and posts ⏹️ Stopped by you, with Continue and Resolve, so the incident can pick up again later. The incident’s timeline records who asked to stop it. Anyone with permission to manage incidents can stop one, not only the person who started it. Only the person who pressed Stop sees these replies:Ask about it
Mention KAOP’s Slack app with your question in the incident’s thread, for examplewhat was the likely cause?. The answer is posted in the thread and comes from that incident: its
alert, findings and runs, and the incident’s chat. It ends with From incident and the
incident’s name, linked, beside a link to the run that answered. Asking never changes the
incident’s findings or status.
The question runs as you, so you need a linked Slack account and permission to view incidents and
use chat. If the answer needs a tool call approved, the approval card appears in the thread.
A reply in the thread that does not mention the app gets no answer.
Slack and the incident page share one conversation. A question asked in Slack shows in the
incident’s Ask about this incident card, labelled with the asker’s name and via Slack (yours
reads You · via Slack), and in Chat. A question asked on the incident page is not posted to
Slack. The answer in Slack is public to the channel and can draw on the whole incident chat,
including questions asked on the page.
Notification sink threads
A Slack notification sink keeps one thread per incident in each of its channels. The first message about an incident starts the thread, and each later event the sink is subscribed to adds a short reply. This is separate from the thread an incident was started from, described under Follow it in the thread.
Questions. One sink posts the question card people answer in Slack: the first Slack sink by
name subscribed to
incident.approval_needed, in its first channel. Approval cards for an agent’s
tool calls go there too. Other subscribed Slack sinks post A decision is waiting: with
Answer in KAOP. Only people who linked their Slack account to KAOP can answer.
While it runs. A thread started by the investigation shows the same reaction, Slack status with
its Stop button, and live line as the incident’s own thread. Every sink thread also gets 🛠️
Fixed when a fix is applied, and its first message counts repeats of the alert.
Asking in the thread. Mention the app in the thread to ask about the incident, as in
Ask about it. It never starts a second incident there. A reply that doesn’t
mention the app is treated like any other message in the channel.
Exceptions. No sink thread starts in the channel the incident was started from; its own thread
covers it. If KAOP cannot post in the thread, the sink posts the event as its own message.
Next steps
Channels
Connect Slack, and route alerts from other apps to an incident workflow.
Incident management & troubleshooting
Set up the incident workflows you start from Slack.
Roles & permissions
Who can start, stop and ask about incidents.
Your own Slack app
Run incidents in Slack through an app you own.