From Runbooks to Hours Saved: Automations That Prove Their Own Value
This is part six of our Connection Value series, which follows what happens after you connect your stack to CloudThinker, from the first five minutes to every month after.
Every Sunday at 02:10, the orders consumer stalls, queue depth passes 10,000, and someone gets paged. They open the runbook in Confluence, restart the consumer, watch the lag drain, post in the incident channel, and go back to bed. It has happened eleven Sundays in a row. Everyone agrees it should be automated. Nobody has done it, because nobody can say whether it's a better use of a sprint than the other thirty things on the list.
Automation programs tend to die on two questions. The first is "what should we automate first?" Teams either automate whatever someone complained about last week, or spend a quarter debating a roadmap while the toil piles up. The second is "did it pay off?" Six months in, someone asks what the effort saved, and the honest answer is a shrug and an anecdote.
Both have answers in data you already generated earlier in this series. Your Jira toil analysis showed which repetitive work eats your team's hours. Your PagerDuty alert audit showed which pages recur and which a machine could triage. This post covers what to do with that picture in CloudThinker: a shortlist ranked by your own evidence, /create-skill to turn existing runbooks into runnable skills, Automation templates for the routine chores, and an hours-saved count you can audit.
Why automation backlogs stall
The two questions are hard for structural reasons.
- The backlog is ranked by volume of complaint. The loudest toil gets automated, not the most expensive. A chore that costs one engineer two hours every week but never wakes anyone up stays manual for years.
- The knowledge is written as prose. Runbooks were written for a person to read and interpret. Turning one into a script means re-deriving every step, every tool call and every "only if" by hand, then testing it. That's a sprint, so it doesn't happen.
- Scripts have no guardrails by default. A cron job that restarts a consumer will restart it in production on the one night it shouldn't. Teams know this, so they don't ship the script.
- Value is never measured at the source. Nobody records "this run saved 20 minutes" when it happens. Six months later the number is reconstructed from memory, and nobody believes it.
Each of the pieces below targets one of those.
Your top 10, not everyone's top 10
Once you have a few connections in place, ask CloudThinker for an automation shortlist built from three inputs:
01Connected stack
What is reachable
No cloud account, no snapshot cleanup. No repo, no PR triage.
02Jira toil
What recurs
Ticket clusters from part four, like eighteen staging resizes in a quarter.
03Alert patterns
What resolves itself
Pages from part five that fire on a schedule and close the same way.
04Output
Your top 10
Ranked by your evidence, with estimated hours and how each one would run.
- Your connected stack: what's actually reachable. There's no point recommending snapshot cleanup if no cloud account is connected, or PR triage if no repo is.
- Your Jira toil analysis: the recurring ticket clusters from part four. If eighteen tickets last quarter were "resize the staging database," that's a candidate.
- Your alert patterns: the recurring, self-resolving and predictable pages from part five. An alert that fires every Sunday at 2 a.m. and always resolves the same way is asking to become an automation.
The output is a ranked "top 10 automations for you" list. Each entry shows what it would do, which findings or tickets justify it, a rough estimate of hours saved per month, and what it would need to write under Auto Mode. Nothing on the list is running. It's a menu ranked by your own evidence, and you build each one you want as an automation or skill in Customize.
Once enabled, it acts within Auto Mode, so you decide what runs on its own and what waits for approval. The ranking answers "what first?", and Auto Mode means saying yes to an automation isn't the same as handing it the keys.
From runbook to skill with /create-skill
Most teams have already written their automations, as prose. They sit in Confluence and wikis as runbooks: "check the queue depth, if it's over 10,000 restart the consumer, then verify the lag drops, then post in the incident channel." The knowledge exists. It just needs a human at 3 a.m. to execute it.
/create-skill turns that prose into a runnable skill. Start it in chat, paste the runbook text, answer its questions, and CloudThinker drafts a SKILL.md file with:
- Steps parsed. Each instruction becomes an explicit, ordered step. Ambiguous ones ("check if it looks healthy") get flagged for you to sharpen instead of silently guessed at.
- Tools mapped. "Check the queue depth" becomes a concrete call against your connected observability tool. "Restart the consumer" maps to the actual Kubernetes or cloud operation through your existing connections.
- Guardrails generated. Preconditions ("only in non-prod", "only if the alert is still firing"), rate limits and stop conditions are drafted for your review.
- A reads-only walk-through. Before the skill acts, ask CloudThinker to run its read steps and tell you what each write step would do, without making the change.
01Input
Runbook prose
A Confluence page or pasted text, written for a human at 3 a.m.
02Parse
Ordered steps
Each instruction becomes a step. Vague ones get flagged, not guessed.
03Map
Real tools
"Check the queue" becomes a query against your connected monitoring.
04Guard
Preconditions
Environment limits, rate limits and stop conditions, drafted for review.
05Check
Walk-through
Real reads, and what each write would do, shown before it acts.
The usual alternative is an engineer hand-writing and testing a script over a sprint, which is why converting runbooks you already have is where the early hours-saved numbers tend to come from.
One runbook, converted
Take the Sunday consumer stall. The Confluence runbook says, in full: "If orders queue depth is over 10,000, restart the orders consumer. Check that lag is going down. Post in #incidents."
CloudThinker asks about two things before it drafts anything. "Restart the orders consumer" doesn't say which environment, and "check that lag is going down" doesn't say for how long or what counts as down. You answer: staging and production both, and lag must fall below 1,000 within ten minutes. The draft skill comes back looking like this (illustrative, trimmed):
---
name: restart-stalled-orders-consumer
description: Restart the orders consumer when queue depth stays above 10,000 and the consumer has stopped making progress. Use when QueueDepthHigh fires for the orders queue.
---
# Restart stalled orders consumer
## Preconditions
- QueueDepthHigh is still firing for queue="orders"
- Consumer throughput has been zero for at least 5 minutes (stalled, not just slow)
- No restart of this deployment in the last 30 minutes
## Steps
1. Read queue depth and consumer throughput from Prometheus:
`max(queue_depth{queue="orders"})` and
`sum(rate(messages_consumed_total{consumer="orders"}[5m]))`
2. Restart the consumer:
`kubectl rollout restart deployment/orders-consumer -n orders`
3. Wait for `kubectl rollout status deployment/orders-consumer -n orders`
4. Verify depth falls below 1,000 within 10 minutes. If not, stop and page on-call.
5. Post a summary with before and after depth in #incidents.
## Stop conditions
- Any precondition fails
- Second restart needed within the same alert
The precondition on throughput is the one the human runbook never spelled out. Without it, the skill would restart a consumer that's slow because of a traffic spike, which fixes nothing. CloudThinker suggested it from the alert history (every past Sunday showed throughput at exactly zero), and you confirmed it.
The reads-only walk-through then reads real queue depth and throughput from Prometheus and shows what the restart step would do, without running it. Once you put the skill to work, the next Sunday's run goes through Auto Mode like every other write. If the restart waits for approval, the on-call gets an approval card with the evidence and the proposed restart.
One design decision worth stating plainly: SKILL.md is an open, portable format. Your skills are plain files. You can read them, diff them, put them in version control, and take them with you. They aren't black-box configurations that exist only in a vendor's database. If you've ever inherited an automation platform where nobody could say what a job actually did, you know why that matters. For the deeper walkthrough of the format, see our runbook automation guide.
Scheduled operations from a template or a sentence
Not everything needs a custom skill. Automations start from a template (a weekly cost report, a daily cost anomaly alert, a daily security drift check, a weekly incident digest, a monthly idle resource sweep, a weekly expiry check) or from one sentence in chat, and they cover the chores every team has and nobody enjoys: the maintenance that gets done in January, skipped in March, and rediscovered during an audit. Each one should say up front which permissions it needs and whether it writes. Some examples:
| Operation | Cadence | Permissions needed | Writes |
|---|---|---|---|
| Orphaned snapshot cleanup | Weekly | Read + delete on snapshots | Yes, through Auto Mode |
| Unattached volume report | Daily | Read-only on volumes | Report only, no writes |
| Access key rotation reminder | Monthly | Read-only on IAM | Report only, no writes |
| Weekend scale-down of non-prod | Fri evening | Read + scale on tagged non-prod resources | Yes, through Auto Mode |
| Log retention enforcement | Weekly | Read + update on log group policies | Yes, through Auto Mode |
| Stale PR and branch report | Weekly | Read-only on repos | Report only, no writes |
| TLS certificate expiry check | Daily | Read-only on certificates/load balancers | Report only, no writes |
| Idle dev database stop | Nightly | Read + stop on tagged dev databases | Yes, through Auto Mode |
The pattern is deliberate. Anything read-only produces a report and touches nothing, so it needs no approval. Anything that changes infrastructure goes through Auto Mode, so you decide which runs need a person to confirm them. The weekend scale-down of non-prod is the easy one to let run: reversible, scoped by tags, and boringly consistent. It's also usually worth real money. It was one of the two biggest line items in our AWS cost automation post.
An hours-saved count you can check
This is the part that answers "did it pay off?" Give every automation an hours-saved estimate per run, written into the summary its last step posts. Pick a figure per operation (a snapshot cleanup run might be worth 0.5 hours, an automated alert triage 0.25) from what the equivalent manual work takes your team, and change it when it turns out wrong.
01Per run
An estimate
Each operation gets a figure, like 0.5 hours for a snapshot cleanup.
02Per team
A correction
If the figure is wrong for you, change it and use the new one from then on.
03Per month
A roll-up
Every line ties to the run behind it, so the total can be checked.
These add up to a monthly roll-up you can ask CloudThinker for: hours saved by operation, by team, by connection. Every line ties back to a specific run (what ran, when, against what, and who approved it), so the total is a number you can defend in a budget review, not a marketing figure. Pull up any month, open any line, and check the arithmetic yourself.
That cuts both ways, on purpose. If an automation runs constantly but its estimate looks inflated, you'll see it and correct it. A value claim you can't challenge is a vibe. This one is a ledger.
Why this beats the usual automation backlog
| Backlog and hand-written scripts | Evidence shortlist, skills and an hours log | |
|---|---|---|
| What gets automated first | Whatever was loudest last week | Ranked by your tickets, pages and connected stack |
| From runbook to automation | A sprint of scripting and testing | A drafted skill, reviewed before it acts |
| Guardrails | Whatever the script author remembered | Preconditions and stop conditions drafted for review |
| First production run | All at once, on merge | Gated by Auto Mode, one run at a time |
| Format | Scripts or vendor-only job configs | Plain SKILL.md files you can version and move |
| Proof of value | Reconstructed from memory | Monthly roll-up, each line tied to a run |
The deeper difference is that evidence runs through the whole loop. The same tickets and pages that justify an automation also feed its preconditions, and the same run records that show what happened also produce the value number. Nobody has to guess at either end.
What this won't do on its own
"Automation platform" plus "your production infrastructure" should raise exactly these questions:
- Nothing on the shortlist enables itself. The shortlist and the templates are menus. Every automation is opt-in, per operation. Connecting a new account or repo never silently activates anything.
- Read-only connections stay read-only. Report-style operations run on the read access you granted at connection time. Anything that writes needs you to grant write permissions and enable the operation, and every write it makes goes through Auto Mode.
- Commands run in CloudThinker's Sandbox. A drafted skill does nothing until you save it and put it to work, and even then it acts within Auto Mode.
- Hours-saved figures are estimates, and should be labeled that way. Set them from what the manual work really takes and revisit them, so the monthly number reflects your reality rather than flattering assumptions.
- It won't decide what's safe to automate. The ranking says "this recurs, and here's the evidence." Whether an operation should run at all in your environment is your call, and the answer is no until you enable it.
Getting started
Once Jira and PagerDuty are connected, a few things to ask:
"Show me my top 10 automation candidates, with the tickets or alerts behind each one and the estimated hours saved per month."
"Convert our 'orders queue stalled' runbook from Confluence into a skill. Flag anything ambiguous, and walk me through what it would do before I use it."
"How many hours did automations save last month, by team? Show me the three operations with the highest estimates and the runs behind them."
From backlog to ledger
The two questions that kill automation programs share a root cause: guesswork. Guessing what to automate, then guessing whether it worked. Once your connections feed a toil analysis and an alert audit, neither needs a guess. The shortlist is ranked by your own tickets and pages, and the hours-saved count is built from your own runs.
The final part of this series looks at why every new connection makes the others smarter: how the same findings compound as the picture fills in.
Open CloudThinker, follow the connection guide to connect your stack, and ask your first question. See your own top 10 and convert your first runbook before the meeting where someone asks what automation actually saved.
