Does this work for models outside Amazon Bedrock?

Yes. Agents read usage and logs from self-hosted models on EKS or SageMaker AI too, as long as the logs and metrics reach a tool you connect.

Do agents read our prompts and responses?

Only if you connect the logs that contain them, and only in your account. You can start with usage metrics and cost tags alone.

How do you avoid a fix that hurts answer quality?

Proposed changes are replayed on past traffic and scored before rollout. You set the quality bar a change must clear.

Can agents change models or capacity on their own?

Only where your policy allows it. Most teams start with agents proposing and keep every rollout behind an approval.

AWS Partner

SOC 2 Reports

Frontier Investigationfor AI Workloads

Model spend, latency and failures on Bedrock and GPUs traced to the cause.

Connect the accounts where your models run. Agents pick up every token spike, latency regression and failed inference, trace it to the prompt, model, route or GPU behind it, and hand the owning team a fix sized in dollars and milliseconds.

Book a Demo See how it works

Amazon Bedrock usage alertcause found

support-assistant token spend up 4x since Monday

Spend: $1,840/day against a 30-day average of $460/day
Change: Release 2.6 added full ticket history to every prompt
Cause: Average input grew from 3,100 to 14,800 tokens
Evidence: Invocation logs, prompt diff, cost tags
Fix: Summarize history to 2,000 tokens. Saves about $1,300/day

Investigated in 3m. Waiting for ML lead approval

[The work behind every AI workload]

Models ship every week.Nobody can say why the bill moved.

01The manual work

Investigate

Spend shows up as one Bedrock line. Someone digs through invocation logs, prompt changes and GPU metrics to find which feature caused it.

02The agent handoff

Cause found

Frontier agents tie spend, latency and errors to the prompt, model, route or instance behind them, and prepare a fix.

03Your engineers’ role

Approve

Set the cost and quality limits. Review the proposed change and decide what reaches users.

[Where CloudThinker fits]

Your stack stays.Agents work inside it.

01AI workloads

Already in your AI stack

Amazon Bedrock and SageMaker AIModel calls, endpoints and spend
Azure OpenAI ServiceDeployments and token usage
Google Vertex AIEndpoints and quotas
GPU nodes on KubernetesUtilization and queue depth
LangfuseTraces and prompt versions

No change to your models

02Cost and health

Your system of record

AWS Cost Explorer and BudgetsAI spend by team and model
DatadogLatency and error rates
Prometheus and GrafanaGPU and serving metrics
SentryApplication errors by release

Source of truth stays put

03Investigation

CloudThinker

Spot the spikeTokens, latency or errors off baseline
Trace the changeThe prompt, model or code change behind it
ReplayThe fix scored on past traffic first
ProposeA change with cost and quality impact

Read-only by default

04Response

Spend under control

SlackCause and cost posted to the team
GitHub and GitLabPull request with the fix
PagerDutyPage when latency breaks the SLO
JiraFollow-up for the model owner

Fix runs on approval

Logos show common stacks. CloudThinker connects to each one through read-only access you approve.

[Example scenario]

Wednesday, 10:15. The AI bill is four times last week’s.Quality looks the same to users.

A 30-engineer SaaS company running a support assistant on Amazon Bedrock and a recommendation model on GPU nodes in EKS. AI spend is reported to the CFO weekly.

10:15
Spend alert firesSignal
AWS Budgets: Bedrock spend for support-assistant above the weekly threshold on day 3.
10:16
Agent picks it upAgent
Pulls Bedrock invocation logs, token counts by application tag, and every release of the assistant this week.
10:18
Cause, with proofAgent
Release 2.6 sends the full ticket history with every prompt. Average input went from 3,100 to 14,800 tokens.
10:19
Fix proposed and testedAgent
Summarize history to 2,000 tokens. Replayed 200 past tickets: answer quality score unchanged. Saves about $1,300 a day.
11:00
ML lead approvesYour team
Checks the replay results, approves the change behind a 10% rollout first.
15:00
Saving verifiedAgent
Full rollout. Tokens per request back to 3,400 on average. Daily spend on track for $520.

#ai-platform4 messages

AWS Budgets10:15
support-assistant Bedrock spend at 112% of the weekly budget.
CloudThinker10:19
Cause: release 2.6 (PR #1407) adds full ticket history to each prompt. Input tokens 3,100 to 14,800. Fix: summarize to 2,000 tokens, MR !96. Replay of 200 tickets shows no quality drop. Saves about $1,300/day.
ML lead11:00
Replay looks good. Approved at 10%.
CloudThinker15:00
Rolled out to 100%. 3,400 tokens per request, no change in CSAT or escalations. Weekly spend back under budget.

from budget alert to known cause: 4 min
daily saving in this example: $1.3k
past tickets replayed before rollout: 200

An illustrative example. Team, systems and times are representative, not a specific customer.

[Frontier investigation agents]

Every token has an owner.Every spike has a reason.

Agents read model usage, releases and infrastructure together, so a spend alert arrives with the feature that caused it, and a fix arrives tested against real traffic.

Trace token spend: Usage broken down by application, feature, model and release, so every dollar maps to a change.
Find the slow path: Latency traced to the model, retrieval step, tool call or retry loop that added it.
Right-size GPUs: Idle and underused GPU nodes found, with the instance type and schedule that fit the load.
Hand over a fix: Prompt, routing and capacity changes tested on past traffic and run only at the approval level you set.

[What changes]

Same team. Same tools.Far less of the work by hand.

Moment	Today	With frontier agents
A jump in AI spend	Seen on next month’s bill	Investigated the day it starts
Finding the cause	Invocation logs read by hand	Traced to the release, prompt or route behind it
Cutting cost	A guess that might hurt quality	A change replayed on past traffic first
GPU capacity	Sized once and left running	Matched to real load, with idle nodes flagged
Reporting to finance	One line item called AI	Spend by feature, team and model

[Integrations]

Connects to the rest of your stack.Read-only to start.

Amazon Bedrock
Amazon SageMaker AI
Amazon EKS
Amazon CloudWatch
AWS Cost Explorer
AWS Budgets
Datadog
Grafana
Prometheus
New Relic
Sentry
PagerDuty
Slack
GitHub
GitLab

[Adoption path]

One pilot.Then company-wide.

The rollout follows the four phases of the AWS Cloud Adoption Framework, so it fits the plan your cloud team already runs.

01Envision
Pick one AI feature
Connect the account and logs behind one model-powered feature, read-only. Let agents investigate in shadow mode.
02Align
Agree cost and quality limits
Set the spend thresholds, the quality checks a change must pass, and who approves rollouts.
03Launch
Roll out feature by feature
Add each AI feature and GPU workload on the same policies, report format and audit trail.
04Scale
Make it the default
New AI features launch with cost tags and investigation on. Findings feed prompt and routing standards.

[AWS guidance]

Built the way AWS says to run it.Reviews start from a known reference.

[Trust and control]

Agents do the work.Your team keeps control.

You approve every change: Agents propose. Nothing touches production until someone on your team says yes, and you set that rule per system.
Every action on the record: Each step is logged, attributed and reversible, ready for your auditors.
Certified for enterprise: SOC 2 Type II and ISO 42001, with reports in our trust center.
Runs where you need it: In our cloud, through AWS Marketplace, or inside your own account.

[Questions]

What teams askbefore they start.

Does this work for models outside Amazon Bedrock?: Yes. Agents read usage and logs from self-hosted models on EKS or SageMaker AI too, as long as the logs and metrics reach a tool you connect.
Do agents read our prompts and responses?: Only if you connect the logs that contain them, and only in your account. You can start with usage metrics and cost tags alone.
How do you avoid a fix that hurts answer quality?: Proposed changes are replayed on past traffic and scored before rollout. You set the quality bar a change must clear.
Can agents change models or capacity on their own?: Only where your policy allows it. Most teams start with agents proposing and keep every rollout behind an approval.

[Go deeper]