Resolve: An Incident From First Alert to Postmortem
An illustrative night. A database starts refusing connections. Within minutes, three monitoring tools fire about every service that depends on it: latency, error rates, failed health checks, a queue backing up.
The on-call engineer gets thirty pages for one problem. The first half hour goes into working out that they're the same problem. The cause hasn't been touched yet.
CloudThinker Resolve is built for that half hour and everything after it. Our line for it is "Finds the why. In minutes." This post follows one incident through every stage, from the first raw event to the postmortem.
Resolve is the incident module of our Frontier Investigation Agents.
Why the usual pipeline breaks
Most incident tooling handles two ends well. Alerting tools fire. Ticketing tools track. The middle, where someone works out what's actually wrong, is a person with twelve browser tabs.
Correlation rules help, but they only group what someone thought to group. Runbooks help, but only when someone already knows which one applies. And the postmortem gets written days later, from memory.
We wanted one flow where detection, investigation, the fix and the follow-up share the same record.
The four stages
01Detect
Pulse
Raw events are suppressed, clustered and routed into an Incident.
02Analyze
Root-cause analysis
Read-only subagents test competing hypotheses and return a verdict.
03Resolve
Scoped Action
A matching runbook runs, waits for approval or is denied.
04Validate
Follow-up and memory
One follow-up check, and lessons for the next investigation.
Resolve runs the platform loop, Detect, Analyze, Resolve, Validate, end to end. The loop post covers why each stage exists. Here's what each one does to our incident.
Detect: Pulse turns thirty pages into one problem
Pulse is the Detect stage. It takes raw provider events from webhook, polling and chat sources, then strips known noise through eight suppression layers: deduplication, rate limits, snoozes, prior verdicts, noise signatures, flapping checks, cascade suppression and severity normalization.
The cascade layer matters most for our night. It suppresses downstream symptoms already explained by a parent problem, so the queue and health-check alerts don't each become their own case.
What survives is grouped into a Cluster, one candidate Signal. Pulse classifies it and creates an Incident when severity or the actionable assessment qualifies it.
01Forming
Collecting
Pulse is still gathering related raw events into the Cluster.
02Active
A current problem
The Cluster represents something happening now.
03Routed
An open Incident
New matching Signals join this Cluster instead of starting another.
04Resolved
Closed
The only final state. Resolving an Incident does not get you here.
A critical single-signal Cluster can route straight from Forming without waiting for correlation. Resolving an Incident doesn't close its Cluster. If the same problem comes back, a matching Signal opens a successor Incident and the Cluster follows it.
The Needs you lane shows unacknowledged high or critical forming Clusters with no owner. That's the lane to watch when nobody has picked anything up.
Analyze: the investigation starts on its own
Pulse creates the Incident before analysis begins. Whether analysis starts automatically is one toggle, On Duty.
With On Duty on, eligible Pulse-created Incidents start root-cause analysis (RCA) automatically. With it off, Pulse still detects and files Incidents, and RCA waits for someone to click Investigate. Turning it back on doesn't start a backlog.
RCA runs in three phases: Context, Analysis and Resolution. In Analysis, CloudThinker starts read-only subagents to test competing hypotheses against logs, metrics, traces, changes and dependencies. A typical investigation uses two to four, with a maximum of eight. Inside an Investigation walks through this in detail.
If several alerts share a likely cause, Resolve groups them under one parent incident and investigates there once.
The result is a root cause with a causal graph, the hypotheses tested (including the ruled-out ones), the evidence, a timeline and recommended remediation. Confidence shows as High (0.8 or above), Medium (0.5 to below 0.8) or Low, and it sits next to the evidence. It's context, not a guarantee.
Resolve: a fix with an effect attached
A matching, enabled runbook can become a Scoped Action. Runbooks come from Confluence, GitHub, GitLab or a Markdown upload.
Every mutating command in a runbook has one of three effects: Allow, Require Approval or Deny. A command with no matching effect defaults to Require Approval, so an unrecognized command from an external source waits for a person. Changes go through Auto Mode, so you decide what runs on its own and what waits for approval.
For our database incident, the remediation might be a connection-pool change in a repository. Resolve can send that as a draft fix merge request, which your team reviews like any other.
Validate: one follow-up check, and memory
When a runbook includes a verification command, CloudThinker can schedule one follow-up check and record whether the expected result held.
What it learned can go into incident memory as an Observation, a Recollection or a Lesson. Each can be Verified, Unverified or Failed. A Failed entry becomes counter-evidence. Lessons come back at the start of a later investigation, before hypothesis work begins.
Where the work lands
01Signals in
Where alerts come from
- WebhookPushed by the alert source
- PollingA feed Resolve checks on a schedule
- ChatEvents from a messaging channel
02CloudThinker Resolve
One incident record
- ClusterNoise removed, one candidate Signal
- InvestigationVerdict, evidence, ruled-out paths
- RemediationRunbook effects decide what runs
Revisions, not second reports
03Outcomes out
Where the team works
- Slack war roomOpened above a severity you set
- Jira follow-upA ticket for the remaining work
- Draft fix MRReviewed like any other change
- PostmortemWritten on request
Resolve sends results to the tools your team already uses. In Resolve → Integrations, each platform has Connections, Signals and Outcomes. Outcomes include an incident update, a Slack war room, a Jira follow-up, a draft fix merge request and a postmortem.
A war room is a dedicated incident channel with its own minimum severity: Critical, High or All. Set it so routine incidents don't open a channel. In a Slack war room, a message only runs when the sender is linked to an active CloudThinker user who can view incidents and use chat.
The postmortem is written on request, not automatically. It's an outcome, not a lifecycle state, so asking for one doesn't change the Incident's status.
Design decisions
Detection doesn't wait for analysis. The Incident exists before RCA starts, and pausing On Duty never stops detection. A paused investigator shouldn't make a system look quiet.
One record per incident. Re-investigating updates the same record. A changed conclusion is recorded as a root-cause revision, not a second report.
Accuracy is judged by people. Resolve → Insights has an Accuracy & trust view built on human verdicts only. Agent confidence doesn't count as proof. When someone disagrees, they pick a reason: wrong root cause, wrong service, fix wouldn't work, false positive or something else.
Honest limits
Not every incident reaches every stage. An investigation can end without a root cause (Not found), a runbook may not match, and post-fix verification depends on the runbook and the follow-up the agent writes.
On hold isn't a final outcome. When CloudThinker needs information, access or a decision, the Incident moves to Needs your decision and waits.
The postmortem doesn't write itself at close. Someone has to ask for it.
Try it
Connect one alert source in Resolve → Integrations, or use Log Incident if you don't have one yet. Then try:
"Investigate the checkout latency incident and tell me which hypotheses you ruled out, and why."
"Open a draft merge request for the fix you recommended, and don't run anything in production."
"Write the postmortem for this incident with the timeline, the root cause and what was ruled out."
Start a trial or book a walkthrough.