From Runbooks to Hours Saved: Automations That Prove Their Own Value

Automation programs stall on two questions: what to automate first, and whether it paid off. How to pick candidates from your own Jira toil and alert history, turn prose runbooks into skills with /create-skill, walk them through on reads only, and measure the time each run saves.

Updated Nov 2, 2026

Cover Image for From Runbooks to Hours Saved: Automations That Prove Their Own Value

From Runbooks to Hours Saved: Automations That Prove Their Own Value

This is part six of our Connection Value series, which follows what happens after you connect your stack to CloudThinker, from the first five minutes to every month after.

Every Sunday at 02:10, the orders consumer stalls, queue depth passes 10,000, and someone gets paged. They open the runbook in Confluence, restart the consumer, watch the lag drain, post in the incident channel, and go back to bed. It has happened eleven Sundays in a row. Everyone agrees it should be automated. Nobody has done it, because nobody can say whether it's a better use of a sprint than the other thirty things on the list.

Automation programs tend to die on two questions. The first is "what should we automate first?" Teams either automate whatever someone complained about last week, or spend a quarter debating a roadmap while the toil piles up. The second is "did it pay off?" Six months in, someone asks what the effort saved, and the honest answer is a shrug and an anecdote.

Both have answers in data you already generated earlier in this series. Your Jira toil analysis showed which repetitive work eats your team's hours. Your PagerDuty alert audit showed which pages recur and which a machine could triage. This post covers what to do with that picture in CloudThinker: a shortlist ranked by your own evidence, /create-skill to turn existing runbooks into runnable skills, Automation templates for the routine chores, and an hours-saved count you can audit.


Why automation backlogs stall

The two questions are hard for structural reasons.

  • The backlog is ranked by volume of complaint. The loudest toil gets automated, not the most expensive. A chore that costs one engineer two hours every week but never wakes anyone up stays manual for years.
  • The knowledge is written as prose. Runbooks were written for a person to read and interpret. Turning one into a script means re-deriving every step, every tool call and every "only if" by hand, then testing it. That's a sprint, so it doesn't happen.
  • Scripts have no guardrails by default. A cron job that restarts a consumer will restart it in production on the one night it shouldn't. Teams know this, so they don't ship the script.
  • Value is never measured at the source. Nobody records "this run saved 20 minutes" when it happens. Six months later the number is reconstructed from memory, and nobody believes it.

Each of the pieces below targets one of those.


Your top 10, not everyone's top 10

Once you have a few connections in place, ask CloudThinker for an automation shortlist built from three inputs:

01Connected stack

What is reachable

No cloud account, no snapshot cleanup. No repo, no PR triage.

02Jira toil

What recurs

Ticket clusters from part four, like eighteen staging resizes in a quarter.

03Alert patterns

What resolves itself

Pages from part five that fire on a schedule and close the same way.

04Output

Your top 10

Ranked by your evidence, with estimated hours and how each one would run.

Nothing on the list runs until you enable it.
  1. Your connected stack: what's actually reachable. There's no point recommending snapshot cleanup if no cloud account is connected, or PR triage if no repo is.
  2. Your Jira toil analysis: the recurring ticket clusters from part four. If eighteen tickets last quarter were "resize the staging database," that's a candidate.
  3. Your alert patterns: the recurring, self-resolving and predictable pages from part five. An alert that fires every Sunday at 2 a.m. and always resolves the same way is asking to become an automation.

The output is a ranked "top 10 automations for you" list. Each entry shows what it would do, which findings or tickets justify it, a rough estimate of hours saved per month, and what it would need to write under Auto Mode. Nothing on the list is running. It's a menu ranked by your own evidence, and you build each one you want as an automation or skill in Customize.

Once enabled, it acts within Auto Mode, so you decide what runs on its own and what waits for approval. The ranking answers "what first?", and Auto Mode means saying yes to an automation isn't the same as handing it the keys.


From runbook to skill with /create-skill

Most teams have already written their automations, as prose. They sit in Confluence and wikis as runbooks: "check the queue depth, if it's over 10,000 restart the consumer, then verify the lag drops, then post in the incident channel." The knowledge exists. It just needs a human at 3 a.m. to execute it.

/create-skill turns that prose into a runnable skill. Start it in chat, paste the runbook text, answer its questions, and CloudThinker drafts a SKILL.md file with:

  • Steps parsed. Each instruction becomes an explicit, ordered step. Ambiguous ones ("check if it looks healthy") get flagged for you to sharpen instead of silently guessed at.
  • Tools mapped. "Check the queue depth" becomes a concrete call against your connected observability tool. "Restart the consumer" maps to the actual Kubernetes or cloud operation through your existing connections.
  • Guardrails generated. Preconditions ("only in non-prod", "only if the alert is still firing"), rate limits and stop conditions are drafted for your review.
  • A reads-only walk-through. Before the skill acts, ask CloudThinker to run its read steps and tell you what each write step would do, without making the change.

01Input

Runbook prose

A Confluence page or pasted text, written for a human at 3 a.m.

02Parse

Ordered steps

Each instruction becomes a step. Vague ones get flagged, not guessed.

03Map

Real tools

"Check the queue" becomes a query against your connected monitoring.

04Guard

Preconditions

Environment limits, rate limits and stop conditions, drafted for review.

05Check

Walk-through

Real reads, and what each write would do, shown before it acts.

From runbook prose to a reviewed SKILL.md, drafted with /create-skill.

The usual alternative is an engineer hand-writing and testing a script over a sprint, which is why converting runbooks you already have is where the early hours-saved numbers tend to come from.

One runbook, converted

Take the Sunday consumer stall. The Confluence runbook says, in full: "If orders queue depth is over 10,000, restart the orders consumer. Check that lag is going down. Post in #incidents."

CloudThinker asks about two things before it drafts anything. "Restart the orders consumer" doesn't say which environment, and "check that lag is going down" doesn't say for how long or what counts as down. You answer: staging and production both, and lag must fall below 1,000 within ten minutes. The draft skill comes back looking like this (illustrative, trimmed):

---
name: restart-stalled-orders-consumer
description: Restart the orders consumer when queue depth stays above 10,000 and the consumer has stopped making progress. Use when QueueDepthHigh fires for the orders queue.
---

# Restart stalled orders consumer

## Preconditions

- QueueDepthHigh is still firing for queue="orders"
- Consumer throughput has been zero for at least 5 minutes (stalled, not just slow)
- No restart of this deployment in the last 30 minutes

## Steps

1. Read queue depth and consumer throughput from Prometheus:
   `max(queue_depth{queue="orders"})` and
   `sum(rate(messages_consumed_total{consumer="orders"}[5m]))`
2. Restart the consumer:
   `kubectl rollout restart deployment/orders-consumer -n orders`
3. Wait for `kubectl rollout status deployment/orders-consumer -n orders`
4. Verify depth falls below 1,000 within 10 minutes. If not, stop and page on-call.
5. Post a summary with before and after depth in #incidents.

## Stop conditions

- Any precondition fails
- Second restart needed within the same alert

The precondition on throughput is the one the human runbook never spelled out. Without it, the skill would restart a consumer that's slow because of a traffic spike, which fixes nothing. CloudThinker suggested it from the alert history (every past Sunday showed throughput at exactly zero), and you confirmed it.

The reads-only walk-through then reads real queue depth and throughput from Prometheus and shows what the restart step would do, without running it. Once you put the skill to work, the next Sunday's run goes through Auto Mode like every other write. If the restart waits for approval, the on-call gets an approval card with the evidence and the proposed restart.

One design decision worth stating plainly: SKILL.md is an open, portable format. Your skills are plain files. You can read them, diff them, put them in version control, and take them with you. They aren't black-box configurations that exist only in a vendor's database. If you've ever inherited an automation platform where nobody could say what a job actually did, you know why that matters. For the deeper walkthrough of the format, see our runbook automation guide.


Scheduled operations from a template or a sentence

Not everything needs a custom skill. Automations start from a template (a weekly cost report, a daily cost anomaly alert, a daily security drift check, a weekly incident digest, a monthly idle resource sweep, a weekly expiry check) or from one sentence in chat, and they cover the chores every team has and nobody enjoys: the maintenance that gets done in January, skipped in March, and rediscovered during an audit. Each one should say up front which permissions it needs and whether it writes. Some examples:

OperationCadencePermissions neededWrites
Orphaned snapshot cleanupWeeklyRead + delete on snapshotsYes, through Auto Mode
Unattached volume reportDailyRead-only on volumesReport only, no writes
Access key rotation reminderMonthlyRead-only on IAMReport only, no writes
Weekend scale-down of non-prodFri eveningRead + scale on tagged non-prod resourcesYes, through Auto Mode
Log retention enforcementWeeklyRead + update on log group policiesYes, through Auto Mode
Stale PR and branch reportWeeklyRead-only on reposReport only, no writes
TLS certificate expiry checkDailyRead-only on certificates/load balancersReport only, no writes
Idle dev database stopNightlyRead + stop on tagged dev databasesYes, through Auto Mode

The pattern is deliberate. Anything read-only produces a report and touches nothing, so it needs no approval. Anything that changes infrastructure goes through Auto Mode, so you decide which runs need a person to confirm them. The weekend scale-down of non-prod is the easy one to let run: reversible, scoped by tags, and boringly consistent. It's also usually worth real money. It was one of the two biggest line items in our AWS cost automation post.


An hours-saved count you can check

This is the part that answers "did it pay off?" Give every automation an hours-saved estimate per run, written into the summary its last step posts. Pick a figure per operation (a snapshot cleanup run might be worth 0.5 hours, an automated alert triage 0.25) from what the equivalent manual work takes your team, and change it when it turns out wrong.

01Per run

An estimate

Each operation gets a figure, like 0.5 hours for a snapshot cleanup.

02Per team

A correction

If the figure is wrong for you, change it and use the new one from then on.

03Per month

A roll-up

Every line ties to the run behind it, so the total can be checked.

A value claim you can challenge, line by line.

These add up to a monthly roll-up you can ask CloudThinker for: hours saved by operation, by team, by connection. Every line ties back to a specific run (what ran, when, against what, and who approved it), so the total is a number you can defend in a budget review, not a marketing figure. Pull up any month, open any line, and check the arithmetic yourself.

That cuts both ways, on purpose. If an automation runs constantly but its estimate looks inflated, you'll see it and correct it. A value claim you can't challenge is a vibe. This one is a ledger.


Why this beats the usual automation backlog

Backlog and hand-written scriptsEvidence shortlist, skills and an hours log
What gets automated firstWhatever was loudest last weekRanked by your tickets, pages and connected stack
From runbook to automationA sprint of scripting and testingA drafted skill, reviewed before it acts
GuardrailsWhatever the script author rememberedPreconditions and stop conditions drafted for review
First production runAll at once, on mergeGated by Auto Mode, one run at a time
FormatScripts or vendor-only job configsPlain SKILL.md files you can version and move
Proof of valueReconstructed from memoryMonthly roll-up, each line tied to a run

The deeper difference is that evidence runs through the whole loop. The same tickets and pages that justify an automation also feed its preconditions, and the same run records that show what happened also produce the value number. Nobody has to guess at either end.


What this won't do on its own

"Automation platform" plus "your production infrastructure" should raise exactly these questions:

  • Nothing on the shortlist enables itself. The shortlist and the templates are menus. Every automation is opt-in, per operation. Connecting a new account or repo never silently activates anything.
  • Read-only connections stay read-only. Report-style operations run on the read access you granted at connection time. Anything that writes needs you to grant write permissions and enable the operation, and every write it makes goes through Auto Mode.
  • Commands run in CloudThinker's Sandbox. A drafted skill does nothing until you save it and put it to work, and even then it acts within Auto Mode.
  • Hours-saved figures are estimates, and should be labeled that way. Set them from what the manual work really takes and revisit them, so the monthly number reflects your reality rather than flattering assumptions.
  • It won't decide what's safe to automate. The ranking says "this recurs, and here's the evidence." Whether an operation should run at all in your environment is your call, and the answer is no until you enable it.

Getting started

Once Jira and PagerDuty are connected, a few things to ask:

"Show me my top 10 automation candidates, with the tickets or alerts behind each one and the estimated hours saved per month."

"Convert our 'orders queue stalled' runbook from Confluence into a skill. Flag anything ambiguous, and walk me through what it would do before I use it."

"How many hours did automations save last month, by team? Show me the three operations with the highest estimates and the runs behind them."


From backlog to ledger

The two questions that kill automation programs share a root cause: guesswork. Guessing what to automate, then guessing whether it worked. Once your connections feed a toil analysis and an alert audit, neither needs a guess. The shortlist is ranked by your own tickets and pages, and the hours-saved count is built from your own runs.

The final part of this series looks at why every new connection makes the others smarter: how the same findings compound as the picture fills in.

Open CloudThinker, follow the connection guide to connect your stack, and ask your first question. See your own top 10 and convert your first runbook before the meeting where someone asks what automation actually saved.

Related reading

About the author

ST

Steve Tran

Founder & CTO, CloudThinker

Steve Tran is the founder and CTO of CloudThinker, which builds Frontier Investigation Agents for production cloud operations. He writes about incident investigation, cloud cost, and the practices teams use to safely run autonomous AI agents against production AWS, GCP, and Azure environments.

cloudthinkeragenticopsautomationrunbooksaiagentdevopssre