Claude Code and Codex on Your Observability Stack vs. CloudThinker
Many engineering teams already run Datadog, Grafana, PagerDuty or Sentry, and many of their engineers already use Claude Code or OpenAI Codex. Many observability vendors now publish an MCP server, so connecting a coding agent to the stack takes minutes. That raises a fair question for platform and SRE leads: is that enough for production operations, or does it need a purpose-built platform like CloudThinker?
This guide compares the two setups on the criteria that matter in production: what starts an investigation, whose access it uses, how it reaches a conclusion, how changes are controlled, and what record it leaves. It ends with when to use each, and how to use them together.
Product details for Claude Code and Codex come from their public documentation, and CloudThinker details from the CloudThinker documentation, as of November 5, 2026.
The two setups
Setup A: your observability stack plus a coding agent. An engineer runs Claude Code or Codex in a terminal or IDE. It reaches your tools through MCP servers, such as the servers Datadog, Sentry and PagerDuty publish and Grafana's open-source server, and through CLIs like kubectl and aws. Project files (CLAUDE.md for Claude Code, AGENTS.md for Codex) give it instructions for each repository.
01Your stack
Observability tools
DatadogMetrics, logs, monitors
GrafanaDashboards and alerts
PagerDutyPages and on-call
Alerts page a person
02Engineer's machine
Claude Code or Codex
MCP serversOne per tool, set up by each user- Cloud and cluster CLIsCredentials already on the machine
- Project instructionsCLAUDE.md or AGENTS.md per repository
Starts when someone opens a session
03Output
One engineer's session
- AnswerIn the terminal or IDE
- ChangeA diff or a command to run
- RecordA transcript on that machine
Ends when the session ends
Setup B: CloudThinker. Your monitoring tools send alerts to Resolve, CloudThinker's incident product, through webhook presets for tools like Datadog, Grafana and PagerDuty, or through Amazon EventBridge for CloudWatch alarms. CloudThinker's Frontier Investigation Agents investigate inside a workspace that holds your connections, approvals, memory and audit trail, and report to your team.
01Your stack
Observability tools
- DatadogWebhook preset
- GrafanaWebhook preset
- PagerDutyWebhook preset
Alerts reach Resolve directly
02CloudThinker workspace
Resolve
- PulseDrops noise, groups related alerts
- InvestigationTests each explanation against evidence
- ApprovalsEvery fix checked before it runs
- MemoryRead at the start of the next incident
Starts on the alert
03Your team
Slack and the incident page
VerdictVerified, Inferred or Cause unproven- FixProposed, then approved by policy
- RecordTimeline, approvals and audit log
Shared by everyone on the incident
Summary comparison
| Criterion | Observability stack + Claude Code or Codex | CloudThinker |
|---|---|---|
| Built for | Writing and changing code with a developer at the keyboard | Investigating production issues and proposing fixes for approval |
| What starts an investigation | An engineer opens a session and asks | An alert from your existing tools |
| Where it runs | The engineer's machine (Codex can also run cloud tasks) | CloudThinker Cloud, with commands in a managed Sandbox |
| Access to production | Credentials on the machine, plus each user's MCP tokens | Connections defined once per workspace, scoped at the provider |
| How it concludes | A general agent loop, guided by the prompt | Each explanation tested against evidence, with a labeled verdict |
| Change control | Per-user permission settings; admins can enforce managed settings | Workspace approvals: a built-in blocklist, Rules, approval mode and approvers |
| Record | A session transcript on the machine; cloud logs show the engineer | Investigation timeline, approvals and organization-wide Audit Logs |
| What it remembers | Project instructions and what each engineer knows | Workspace and incident memory, read at the start of each investigation |
| Collaboration | One engineer's session | A shared incident page, Incident Room and Slack war room |
| Strongest at | The code change once the cause is known | Finding the cause and handing over the fix |
The rest of this guide explains each row.
Detailed comparison
What starts an investigation
Setup A. Nothing happens until a person opens a session. The alert pages an engineer, the engineer opens a laptop, describes the problem, and the agent starts from that description. Coverage outside working hours depends on someone being awake and at a keyboard, and the quality of the first prompt shapes the whole investigation.
Setup B. The alert itself starts the work. Pulse, Resolve's detection layer, drops duplicates and known noise and groups related alerts into one cluster, so one outage is one item. While On duty is on, CloudThinker triages each actionable cluster, opens an incident and investigates it without waiting for a person. People are brought in through On-call when the incident needs them.
Access and credentials
Setup A. When a coding agent runs kubectl or aws on a laptop, it uses the credentials already on that machine, which are usually the engineer's own and broader than an investigation needs. Each MCP server is set up per user, typically with that user's API key or OAuth sign-in. Access is therefore defined in as many places as there are engineers, and it's hard to answer "what can the agent reach?" for the team as a whole.
Setup B. Access is defined once, as connections in a workspace. The credential behind each connection is the ceiling: if an AWS role is read-only, no prompt or approval can make an agent change that account. Connection credentials are stored encrypted and injected into CloudThinker's Sandbox only when a command runs. They never go to the model or to anyone's laptop. Secrets and personal identifiers that appear in tool output are replaced with placeholders before the model sees them. Where Your Keys Live covers the details.
How it reaches a conclusion
Setup A. Claude Code and Codex are general agents. They follow the prompt, run the queries they judge useful, and write an answer. Both can be steered well with good instructions, but the method and the standard of evidence change with each engineer and each session. The output is prose in a terminal, with no consistent format or confidence label.
Setup B. Resolve runs a structured root cause analysis. Every result follows the same format: a Current assessment with an evidence label, a causal graph, prioritized remediation, and an Investigation timeline that lists each explanation tested and whether it was confirmed or ruled out.
| Evidence label | Meaning |
|---|---|
| Verified | Confirmed against data from your systems |
| Inferred | Reasoned from the evidence, not directly confirmed |
| Cause unproven | The verdict rests on an assumption; treat it as a lead |
When the evidence doesn't support any root cause, the incident ends as Not Found instead of a guess. Admins can mark each diagnosis right or wrong, and Resolve Insights tracks root-cause accuracy on your own incidents.
Change control
Setup A. Both coding agents have serious controls, designed for a developer's machine. Claude Code has allow, ask and deny permission rules (deny wins), permission modes including an auto mode where a classifier reviews actions, shell sandboxing, hooks, and managed settings that administrators can enforce. Codex has sandbox modes (read-only, workspace-write, danger-full-access), approval policies, network access off by default in workspace-write, and managed configuration through requirements.toml. These controls govern a session on a machine. They don't define who on the team may approve a production change, or route that approval to the right people.
Setup B. Every change an agent attempts passes the workspace's approvals, set through Auto Mode. A built-in blocklist refuses calls that should never run, such as stopping CloudTrail logging or deleting the kube-system namespace, and no person or Rule can override it. Rules, the approval mode and approver groups (including a two-person rule) decide what runs and who approves. The policy belongs to the workspace, not to one engineer's configuration, and can be exported as a file and imported into other workspaces.
Audit and accountability
Setup A. The session transcript lives on the engineer's machine. Your cloud provider logs the API calls under the engineer's identity, so the record shows who owned the credential, not which session, prompt or agent made the call, or whether anyone reviewed it first.
Setup B. Each investigation keeps its full timeline on the incident page, and each approved change records who approved it. On Scale and above, Audit Logs gives one timeline of user, agent and system actions across every workspace, with the agent named as the actor.
Context and learning over time
Setup A. Context comes from project instruction files and from what the engineer types. What one engineer's session learns about your systems isn't available to the next engineer unless someone writes it down.
Setup B. Context belongs to the workspace. CloudThinker keeps workspace memory about your environment, reads your knowledge bases, and writes incident memory at the end of each investigation. A new investigation reads that memory before it forms a hypothesis, so when the same service fails the same way again, it starts from what your team already learned.
Collaboration during an incident
Setup A. The investigation lives in one person's terminal. Sharing it means copying output into Slack or a ticket.
Setup B. The investigation lives on a shared incident page. With On-call turned on, CloudThinker opens an Incident Room for the responders and can open a Slack war room, so everyone works from the same evidence.
Benchmark: same model, different harness
The public SREGym-Lite leaderboard runs agents against injected production faults and scores whether they diagnose and fix them end to end. It lists Claude Code, Codex and CloudThinker on the same models, which isolates the effect of the system around the model.
| Model | Agent | End to end | Time to mitigate | Tokens per run |
|---|---|---|---|---|
| Claude Opus 5 | CloudThinker | 76.5% | 759 s | 1.56M |
| Claude Opus 5 | Claude Code | 70.6% | 466 s | 1.86M |
| GPT-6 Astra | CloudThinker (low effort) | 100.0% | 342 s | 736K |
| GPT-6 Astra | Codex (max effort) | 96.1% | 288 s | 719K |
| GPT-6 Astra | Codex (medium effort) | 90.2% | 138 s | 377K |
Public SREGym-Lite leaderboard, checked November 5, 2026.
On the same model, CloudThinker resolves more incidents end to end. The coding agents finish their runs faster. Read the results with three caveats:
- SREGym-Lite has 17 faults, injected into open-source demo applications. A gap of a few points is a small number of incidents.
- A benchmark measures one investigation at a time. It doesn't measure what starts the investigation, who approves the fix, or what's recorded, which is where most of the differences in this guide sit.
- The leaderboard changes as agents submit new runs. Check the current board before you rely on these numbers.
When to use each
| Situation | Better fit |
|---|---|
| Writing, refactoring or reviewing the code for a fix | Claude Code or Codex |
| Ad-hoc exploration by an engineer at the keyboard, in non-production | Claude Code or Codex |
| Alerts that arrive outside working hours | CloudThinker |
| Production access that must be scoped, approved and audited for the whole team | CloudThinker |
| Several teams that need to share what past incidents taught them | CloudThinker |
| A consistent, evidence-labeled root cause for every incident | CloudThinker |
The two setups solve different problems. A coding agent turns intent into a change in a repository. CloudThinker turns a production signal into a verified cause and a proposed fix, under rules the whole team shares.
How to use them together
CloudThinker connects to Claude through the Claude Connector, a remote MCP server. Claude Code can read your incidents, cost findings, code reviews, security findings and automations, and start CloudThinker runs that follow your workspace's approvals. Sign-in uses your CloudThinker login, so SSO and MFA apply.
To add it to Claude Code:
claude mcp add --transport http cloudthinker https://app.cloudthinker.io/api/mcp
Then run /mcp in Claude Code, select cloudthinker, sign in, pick a workspace and click Allow.
01Alert
Your monitor fires
Datadog, Grafana or PagerDuty sends the alert to Resolve.
02Investigate
CloudThinker finds the cause
A verdict with evidence and a proposed fix, in the incident page and Slack.
03Code
Claude Code writes the change
The engineer pulls the incident into Claude Code through the Claude Connector.
04Ship
Review and approval
The change goes through code review and your approval policy.
A typical flow: CloudThinker investigates the alert and posts a verified root cause. The engineer who owns the service asks Claude Code to read that incident and write the fix in the repository. The change then goes through code review and your approval policy before it reaches production.
The Claude Connector works with Claude clients only today: claude.ai, the Claude desktop app and Claude Code. Codex users can work from the incident page and Slack.
Evaluation checklist
Use these questions to evaluate either setup, or any other AI SRE tool:
| Question | What to look for |
|---|---|
| What starts an investigation at 3 a.m.? | The alert, not a person opening a laptop |
| Whose credentials does the agent use? | Credentials defined for the agent, scoped at the provider, not an engineer's own |
| Can you list everything the agent can reach? | One place that answers for the whole team |
| How does it say it isn't sure? | A labeled verdict, and a way to end without a root cause |
| Who approves a production change, and where is that set? | A team policy, not a per-user setting |
| What does the audit trail show? | The agent as the actor, the approver, and the evidence |
| What does the next investigation know about the last one? | Memory that persists across people and sessions |
Limitations of this comparison
- CloudThinker publishes this guide. We've described Claude Code and Codex from their documentation and kept to what it states.
- Claude Code and Codex release often. Some details here may change after November 5, 2026.
- A coding agent can be wrapped in your own automation: triggered by alerts, given scoped service credentials, and logged centrally. That's possible, and it means building and running the platform layer described in Setup B yourself. Why a Coding Tool Can't Safely Connect to Production covers what that layer has to include.
Frequently asked questions
Can't we give Claude Code or Codex read-only credentials?
Yes, and you should. Read-only credentials limit what the agent can change. They don't change what starts an investigation, how evidence is labeled, who approves a fix, or what the team remembers next time.
Does CloudThinker replace Claude Code or Codex?
No. Keep using them for code. CloudThinker covers the production side: detection, investigation, approved changes and the record. The Claude Connector links the two.
Which tools does CloudThinker work with?
Resolve has webhook presets for monitoring and alerting tools such as Datadog, Grafana and PagerDuty, a custom source for anything else, and EventBridge for CloudWatch alarms. Investigations use the cloud, cluster, observability and code connections in your workspace.
