The economics made voice support a practical AI use case. In Peter Bar’s July 2025 talk, more than 80% of support teams still used phone support, and more than one-third of customer-service interactions happened by phone. A US call handled by a human cost about 7–12; Bar estimated that voice AI could be at least five times cheaper.

Intercom also had a production foundation. Its wider Fin platform served 5,000+ customers with a 56% average resolution rate, reaching 70–80% for some customers. These figures describe Fin overall, not Fin Voice traffic; Voice call volume and peak concurrency were not disclosed.

Fin Voice therefore extended an established AI support platform into a channel that was widely used and expensive to operate. Doing that required more than an LLM endpoint: telephony, speech processing, RAG, workflow integration, controlled deployment, observability, evaluation, and human handoff all became part of the system.

How Intercom introduced Fin Voice

Intercom began with knowledge-based questions already handled by its text agent. Call transcripts showed that help-center content could answer many phone requests.

Intercom first placed Fin Voice in after-hours support, where it replaced voicemail without changing the daytime workflow. This gave support teams real calls to review while limiting the effect of mistakes. Fin Voice later moved onto several customers’ main phone lines. Intercom’s current deployment guide recommends an initial 5–10% of calls.

Figure 1: Intercom moved from voicemail replacement to an after-hours pilot and then controlled production traffic.

Why 100 days was possible

Fin Voice reused four production building blocks:

1. Fin’s existing text-agent runtime and agent behavior.

2. A RAG system already connected to customer knowledge bases.

3. Intercom’s native phone product and call-routing workflows.

4. An installed customer base that could test the product and provide feedback.

The team was not building an AI platform, knowledge pipeline, and telephony product from zero. Its main work was integrating those systems and adapting the experience to real-time voice. That reuse explains more of the 100-day timeline than model choice alone.

Intercom Fin Voice solution architecture

This teaching diagram combines the public talk with Intercom’s deployment and telephony documentation. It is not an official internal diagram.

Figure 2: Fin Voice connected the real-time runtime to telephony, customer-service workflows, deployment controls, and evaluation data.

The system can be read as five layers:

1. Telephony and session ingress: Calls enter through Intercom Phone, PSTN forwarding, or SIP. External providers can register a phone number and call ID through the Fin Voice API, linking the carrier session to an Intercom conversation.

2. Real-time runtime: Speech -> STT -> LLM + RAG -> TTS -> Speech. Intercom initially used a real-time API to test quickly and retained it after evolving the stack. The chained design exposed text and retrieval for grounding and debugging; direct speech-to-speech offered more natural audio but less control.

3. Knowledge and actions: RAG retrieved approved help content. Current documentation also describes closed-beta Voice Procedures for API actions such as refunds or subscription changes; this was not part of the original 2025 disclosure.

4. Workflow and handoff: Rollout percentage, office hours, customer attributes, call history, and IVR branches determined which calls reached Fin. Unresolved calls transferred with a summary, transcript, and intent.

5. State and evaluation: Intercom stored recordings, live transcripts, summaries, outcomes, routing state, and call duration. Internal tools joined this evidence with logs for troubleshooting and evaluation.

The architecture distributed risk across the full call path. Carrier, transcription, retrieval, model, action, speech, and transfer failures could each affect the caller’s outcome.

Voice is not chat with sound

A chatbot can pause or return several paragraphs. On a phone call, silence feels broken and long answers are difficult to remember.

Figure 3: Voice requires different latency, answer length, and interaction patterns.

Intercom changed three behaviors:

• Hide unavoidable latency: Simple answers arrived in about one second. For requests taking three or four seconds, the agent acknowledged the caller while working.

• Break up long answers: Troubleshooting steps were delivered in short chunks with confirmation between steps.

• Teach natural conversation: Callers often began with IVR-style keywords. Natural responses encouraged them to speak in complete sentences.

The LLMOps loop

Intercom delivered a browser voice playground within roughly four weeks. Support managers could test Fin against their own knowledge base before connecting a live number.

Figure 4: Test, deploy, observe, evaluate, and improve every major change.

The operating loop worked as follows:

1. Representative conversations were tested in the playground.

2. Changes reached after-hours or percentage-based production traffic.

3. The team reviewed recordings, transcripts, logs, handoffs, and outcomes.

4. Failures were traced to prompts, knowledge, routing, or runtime components.

5. Evaluations ran again before traffic increased.

The primary outcome metric was resolution rate: calls completed without human help. This included explicit confirmation and assumed resolution when a caller disconnected after an answer and did not call again within 24 hours.

Intercom started with manual spreadsheet evaluations and automated more of the process over time. An LLM judge helped review transcripts, but human review and deterministic checks still mattered.

Challenges and responses

ChallengeWhat Intercom didRisky first deploymentStarted after hours and increased traffic gradually.Slow complex requestsPlayed a short acknowledgement instead of silence.Long spoken instructionsSplit answers into steps and asked for confirmation.AI could not resolve the callEscalated to the correct team with a transcript summary.Hard-to-debug failuresConnected recordings, transcripts, logs, outcomes, and evaluations.

What the case reveals for AI Platform Engineers

1. Platform reuse created the schedule advantage. RAG, telephony, data, and the text agent already existed.

2. The model was one runtime component. Testing, deployment, tracing, evaluation, handoff, and rollback surrounded inference.

3. Quality existed at call level. STT, retrieval, generation, TTS, routing, and audio quality all affected the outcome.

4. Handoff was part of the architecture. Context and ownership had to survive the transfer.

5. AI behavior followed progressive delivery. Exposure moved from playground tests to after-hours calls and then production percentages.

6. Business metrics differed from model metrics. Resolution, escalation, repeat calls, and cost per outcome mattered more than tokens alone.

The public material does not describe every internal control. In a comparable production system, the remaining platform scope would normally include latency SLOs, end-to-end tracing, safe retries, provider fallback, PII controls, prompt-injection defenses, versioned prompts and knowledge, representative evaluation sets, and rollout stop conditions.

Bottom line

Fin Voice was a production system rather than a model endpoint. Intercom’s delivery speed came from existing platform capabilities; its production adoption came from workflow integration, controlled exposure, call-level evidence, and human handoff.

Sources

• Peter Bar, Shipping an Enterprise Voice AI Agent in 100 Days, AI Engineer, July 2025.

• Intercom, Deploy Fin Voice.

• Intercom, Integrating Fin Voice using call forwarding.

• Intercom, Use Fin Voice in phone workflows.

• Intercom, Fin Voice FAQs.

Accuracy note

This article uses the public talk and Intercom documentation. Wider Fin metrics are not presented as Fin Voice traffic. The architecture is a teaching reconstruction, not a complete view of Intercom’s internal implementation.

AI DevOps is becoming real — but it is much bigger than asking ChatGPT to generate Terraform or Kubernetes YAML.

We are starting to see Agentic DevOps, AI SREs, autonomous incident investigation, agentic CI/CD pipelines, agentic skills, AI Platform Engineering and production AI infrastructure emerge at the same time.

Watch this complete episodes to get my insights on each of these points and where I see its going based on my 20years of experience in Ops/Devops and what I have been experiencing since last two years of working on AI + Devops intersection. Following is the research I did which you could use as a reference.

In this episode, I research what is happening across the AI + DevOps ecosystem and share my perspective on where I think this is heading.

We explore:

• AI DevOps vs Agentic DevOps vs AIOps

• AI for DevOps vs DevOps for AI

• AI SRE and autonomous incident investigation

• GitHub Agentic Workflows and agentic CI/CD

• AI-powered release readiness and root cause analysis

• Agentic Skills and turning runbooks into executable knowledge

• Coding agents such as Claude, Codex and Copilot

• Agent harnesses and provider-agnostic agents

• Hermes Agent and declarative agent architectures

• MCP vs traditional CLI tools

• Safety, approvals, RBAC and human-in-the-loop systems

• GitOps as a safety mechanism for autonomous agents

• AI Platform Engineering

• Kubernetes for AI workloads

• GPU infrastructure, DRA, Kueue and AI scheduling

• LLM observability

• AI FinOps

• Production AI Engineering

• Where DevOps, SRE and Platform Engineering careers may be heading

One of the biggest changes I see is this:

The valuable DevOps skill is moving from remembering commands to applying judgment.

AI can increasingly generate syntax and execute tasks. But someone still needs to understand architecture, reliability, security, cost, production systems, guardrails and when an AI agent should — or should not — be allowed to act.

That creates an enormous opportunity for experienced DevOps, Platform Engineering and SRE professionals.

If you’re working in DevOps, SRE, Platform Engineering, Kubernetes, Cloud or Infrastructure and trying to understand how AI changes your career, this episode is for you.

Subscribe if you want to follow my exploration of AI DevOps, Agentic DevOps, AI Platform Engineering and Production AI Engineering.

Let me know in the comments what you want me to explore next — AI SRE, Agentic Skills, Hermes, AI infrastructure, GPU scheduling, agentic CI/CD, guardrails or something else.

Resources and my Mindmap:

• My ChatGPT Research

• GitHub Agentic Workflow : https://github.blog/changelog/2026-06-11-github-agentic-workflows-is-now-in-public-preview/?utm_source=chatgpt.com

• DevOps Agent Kit by Cloudbees (https://www.cloudbees.com/blog/how-to-build-devops-agent?utm_source=chatgpt.com)

• Goose Harness/Agent by Block : https://goose-docs.ai/

• Devops Kit by CloudBees : https://cloud.google.com/devops

• AWS Devops Agent : https://docs.aws.amazon.com/devopsagent/

• Azure SRE Agent : https://azure.microsoft.com/en-us/products/sre-agent

You do not need to become a machine learning expert to work with AI agents. You already know much of the hard part: permissions, APIs, retries, logs, failure handling, and safe changes.

The new part is understanding how an agent chooses and repeats actions.

The short version

• An AI agent uses a model to choose tools and repeat steps until it reaches a goal or a limit.

• This week, MCP became easier to run, OpenTelemetry added an end-to-end agent demo, and cloud providers moved toward separate agent identities.

• In a Google Cloud survey, 83% of 1,402 IT leaders said production agents require infrastructure upgrades.

• The START checklist shows how to try one narrow, read-only agent task with traces and hard limits.

• Before adding write access, protect tools against retries that repeat a successful change.

First, what is an AI agent?

A normal chatbot answers a question. An agent can take several steps toward a goal.

For example, you ask: “Why is checkout latency high?”

The agent may:

1. Query metrics.

2. Read recent deployment events.

3. Search logs.

4. Compare the evidence.

5. Suggest the likely cause.

The model decides which tool to call next. The tools do the real work. The loop continues until the agent reaches an answer, hits a limit, or is stopped.

That makes an agent similar to an automation workflow whose next step is chosen at runtime. This flexibility is useful. It also creates new failure modes.

Why your DevOps experience matters

Agentic AI uses new names for familiar problems.

• A tool call is still an API request.

• Agent memory is still state that can become stale or leak data.

• A tool permission is still an access-control decision.

• An agent loop is still a loop that needs a deadline and retry limit.

• A model or prompt change is still a production change that needs testing.

The model is new. Production engineering is not.

Your advantage is knowing that successful execution is not the same as a correct outcome. A command can return zero and still change the wrong resource.

Three signals from this week

1. MCP is becoming easier to operate

The Model Context Protocol, or MCP, is a common way for AI applications to discover and call tools. Its 2026-07-28 revision makes the core protocol stateless, which means servers fit normal HTTP load balancers and gateways more easily.

For a beginner, the idea is simple: MCP is an adapter between an agent and a tool. It does not make the tool safe. Authentication, approval, retries, and permissions still matter.

MCP release notes and AWS compatibility guide.

2. OpenTelemetry now has an agent demo

OpenTelemetry Demo 3.0 includes an agent, an MCP server, and end-to-end traces. One trace follows a request through the model, tool calls, and downstream services.

This is a useful learning environment because you can see what the agent did. You do not need to connect it to your production systems.

Demo walkthrough and GenAI telemetry guide.

3. Agents are getting their own identity

Google Cloud now describes an agent as its own identity. AWS documents a separate authentication pattern for AgentCore.

You do not need these products to understand the lesson. Do not give an agent your personal token. You should be able to identify, limit, audit, and revoke the agent separately from the person using it.

Google Cloud’s agent IAM model and AWS’s JWT pattern.

One number this week: 83%

In Google Cloud’s survey of 1,402 global IT leaders, 83% said their organization needs infrastructure upgrades for production-grade autonomous systems.

This is vendor-sponsored research, not a neutral benchmark. Still, it is a useful signal: many companies can build AI demos, but they are not ready to operate them safely at scale.

Report summary and methodology.

Your first runbook: START read-only

Do not begin with an agent that restarts pods or changes cloud resources. Start with an investigation task.

S: Scope one task

Choose a narrow question such as: “Summarise the likely cause of this alert.”

T: Tools stay read-only

Allow metrics, logs, traces, and deployment history. Do not allow changes.

A: Audit every step

Record the question, model version, tool calls, inputs, outputs, time, and cost.

R: Restrict the loop

Set a deadline, tool-call limit, token or cost budget, and a manual stop control.

T: Test bad conditions

Try missing data, stale data, a failed tool, a timeout, and conflicting evidence. Check whether the agent admits uncertainty.

Your first success is not autonomous remediation. It is a useful answer with a trace you can explain.

Failure mode: a retry repeats the change

Later, when you add write tools, a timeout can become dangerous. The change may succeed while the response is lost. The agent sees a failure and tries again.

The control belongs in the tool, not the prompt. Use an idempotency key, which makes a repeated request return the original result instead of creating another change.

“Please do not deploy twice” is not a control.

Should we add hands-on labs?

I am considering starting weekly lab/project where we take up one Devops/SRE use case and try to implement Agentic AI for it, or maybe take up something on MLOps/LLMOps side and show you how to build it.

Reply to this issue and tell me:

• What would make sense to you ?

• Which DevOps and AI problems should become labs?

• What should this newsletter include more or less of?

• What would make Runbooks & Robots worth opening every week?

Looking forward to read your comments to this article.

Robot walks into an incident review

What to do Monday

1. Write down one investigation task that uses only read access.

2. List the tools and data the agent would need for that task.

3. Run or review the OpenTelemetry agent demo and follow one trace.

Closing thought

You are not starting from zero. Agentic AI needs the same discipline that made cloud systems reliable: narrow access, visible work, bounded execution, and safe failure.

Start read-only. Learn how the agent behaves. Add power slowly.

What was this experiment about

AI coding agents like Claude and Codex are great at writing code. But the moment you point them at real infrastructure, be it a Kubernetes cluster, a production database, or a live AWS account, things get scary. You do not want the agent’s good behaviour to be the only thing stopping it from deleting something it should not touch.

So I set out to build a harness for this. The idea: give an agent its own identity, not yours, scoped down to exactly what it needs, enforced by the system itself and not by a polite instruction in a prompt. And make the skills that agent uses portable. Write a skill once, run it on Claude Code, Codex, or whatever agent runtime comes next, the same way an Ansible role runs on any server.

I called it AOH, an Agentic Ops Harness. And I set myself one more rule at the start, out loud: it had to be simple enough to run from inside whatever coding agent I was already talking to. No separate tool to learn and no additional CLI options to remember.

What I built

More than I expected, and a lot of it actually worked.

Three different agent runtimes could all run the same skill pack. I proved the safety story on a real cluster, not a mock one. The agent tried to delete a pod, and Kubernetes itself said no. Not because the agent chose to be careful. Because its permissions simply did not allow it. That single moment told me the core idea was sound.

I also built a strict review process around every change. Before anything shipped, a second AI would review the design, specifically trying to find what I had missed. And it did find real things. Once it caught a permission rule that accidentally gave read access to every secret in the cluster. Another time it caught a credential design that would have quietly leaked tokens. That process earned its keep every single time.

By the end there was even a full write-a-skill-and-publish loop. You draft a skill in whatever session you are already in, run one command, and it lands as a real commit or a real pull request on GitHub. I watched this happen against a real repository, with three hundred and sixty seven tests behind it. So this was not a toy. It was real, working software.

Why I am still abandoning it

Here is the part that stings a bit. One day I asked myself the simplest question a builder can ask: can I actually use this?

And the honest answer was six different commands, spread across two modes of the same tool, with flags that meant different things depending on which mode you were in. I had built a harness, and I could not operate my own harness without a cheat sheet next to me.

That is not a missing feature. That is a sign the design was wrong from the start. I had said clearly, on day one, that I wanted zero commands to remember. I actually built that, exactly once, for one single feature, the publish step. For everything else, install, fleet management, version locking, I kept adding a new command instead of wrapping it the same simple way. Each of these decisions was fine on its own. Add them all up, and you get a tool that only its own builder could drive.

So I am abandoning this build. Not the idea behind it, the idea is still good, and I proved enough of it live to trust it. What I am abandoning is this particular attempt. I do not think you fix a tool that has drifted this far from its own starting point by patching it here and there. You go back to that starting point and build toward it directly, this time holding the one rule that mattered most as non-negotiable from day one, not bolted on later for a single feature.

The lessons

1. Working software and the right software are two different tests.Every phase of this project passed its own review. Tests were green. Features worked. And the whole thing still drifted away from what I said I wanted at the start. Passing your own review is not the same as staying on course.

2. Ask “can I use this without notes” after every single milestone, not at the end.I should have run that test every week. I ran it once, forty days in, and by then the answer was already no.

3. Building with AI is fast, and fast cuts both ways.You can go from idea to tested, working, reviewed code in hours. But it is just as easy to build your way away from your original intent as it is to build toward it, because every small step still feels justified, still passes review, still works. Nothing stops you automatically and asks if you are still headed where you meant to go.

4. Complexity built with AI is easy to add, but surprisingly hard to unwind. Once a system gets complicated enough, with enough small decisions that each made sense in the moment, it becomes genuinely hard to undo. Not impossible, just hard in a way that quietly pushes you toward one more patch instead of a rethink. And when an AI can produce that next patch in minutes, the cost of “just one more fix” drops so low that you stop noticing how many fixes you have already made.

5. Adversarial review is worth every bit of the extra time. The bugs that mattered most were never caught by a quick look. They showed up only when someone, or something, traced the whole path across files with the specific job of trying to break it.

What a DevOps engineer, SRE, or platform practitioner should take from this

If you are exploring AI agents for operations work, three things worth carrying forward.

One, put the safety boundary in the real system, not in the prompt.

RBAC, scoped credentials, whatever your platform gives you. Never trust an agent’s good behaviour as your only line of defence.

Two, if you are building any kind of tool or workflow around agents, decide your usability bar before you write the first line, and check against it constantly, not once at the end. It is far easier to hold a line from the start than to pull a system back to it later.

And three, do not be afraid to restart. An abandoned build is not proof the idea was wrong. Sometimes it is proof you learned enough to try it again, better. That is exactly what I am doing next, same idea, different shape, and I will tell you how it goes.

The code for this attempt is public, MIT licensed, at agenticdevops/aoh

with docs at agenticdevops.github.io/aoh. If any part of it is useful to you, take it further than I did.

You do not see the sorting desk, the address check, the delivery route, or the worker handling each step.

ChatGPT feels like one box too. But the answer comes from a chain of small steps, not from a single magic reply button.

A chat box makes an AI system look simple.

You type a question. Words come back. It feels like a person read your message, thought for a moment, and wrote a reply.

That picture is useful, but it hides the real system. Behind the box is a request pipeline with rules, text splitting, model work, memory-like cache, one-token-at-a-time generation, and streaming.

The simplest mental model is this:

Your prompt is a letter. The AI service is the post office. The answer is delivered piece by piece.

That does not mean an LLM works like a human clerk. It does not. The analogy only helps us see the flow.

Think of the chat app like a post office.

Everyday pictureAI systemYou drop a letter in the boxYou press Enter with a promptThe post office prepares and sorts itThe app assembles context and tokenizes textA reply is handled through many small stepsThe model generates one token at a time

Where the analogy stops:

A post office moves physical letters. An LLM does math over token IDs. It does not understand, remember, or verify information the way a person does.

The analogy is only for the flow: package, sort, process, deliver.

When you press Enter, several things happen before the first word appears on screen:

• the app prepares the request,

• safety and product rules may be attached,

• your text is split into small units,

• the model processes the full context,

• the model starts choosing the next unit of output,

• the app streams the reply as units arrive.

The important shift is this: an LLM chat response is not a finished document fetched from a shelf. It is generated step by step.

Step 1 — The app prepares the package

The model usually does not receive only the sentence you typed.

A chat product can add hidden instructions, conversation history, tool descriptions, retrieved notes, file excerpts, safety metadata, and your latest message. The exact package depends on the product.

Plain-English version: before the letter goes to the worker, the service attaches the envelope, address label, rules, and previous conversation.

Technical name: this is request assembly or prompt assembly.

Step 2 — The text is split into tokens

The model does not read text exactly like people do.

It reads and writes tokens. A token is a small piece of text. It might be a whole word, part of a word, a space, punctuation, or a common text pattern.

So the model does not see one smooth sentence. It sees a list of token IDs, which are numbers that represent those small text pieces.

Plain-English version: the post office sorts a letter into small handling units. The model sorts text into small reading units.

Where the analogy stops: postal workers still understand addresses and letters as people. A model computes over numbers that stand for text pieces.

Step 3 — The model reads the context first

Before the answer begins, the model processes the tokens it has been given.

This first pass is often called prefill. It is the setup work before output starts.

During this step, the system also builds internal state used for attention. In many serving systems, that state is stored in a KV cache. You can think of it as a scratchpad the system uses so it does not have to redo all the same work for previous tokens at every step.

This helps explain why a long prompt can feel slow before the answer starts. The system has more material to read first.

Step 4 — The model chooses one token

After the setup, the model begins the part we experience as “typing.”

It scores many possible next tokens. Then the runtime chooses one of them. That chosen token is added to the growing answer.

Then the process repeats.

The loop is:

read current context → score next tokens → choose one token → append it → repeat

This is why the answer can change direction. Each token becomes part of the context for the next choice.

Step 5 — The sampler controls how strict the choice is

At each step, the model produces scores for candidate tokens. These raw scores are often called logits.

The runtime then applies a selection rule. Settings such as temperature and top-p change how strictly the system follows the highest-scoring options.

Plain-English version: if several next words are possible, the system can always pick the strongest guess, or it can allow some other reasonable guesses into the draw.

This does not make the model more truthful. It only changes how output is selected.

Step 6 — The answer streams back before it is finished

The app does not need to wait for the whole answer.

As tokens are created, they can be sent back to the browser or app. That is why you see the response appear gradually.

Plain-English version: the delivery person does not wait for the whole stack of mail to finish before showing movement. You see the route in progress.

Technical name: this is streaming.

Operational trade-offs and failure modes

Long input changes the first wait

There are two waits in an LLM response.

The first is the wait before output begins. This is often called time to first token.

The second is how fast the remaining output arrives. This is often measured as tokens per second.

A long prompt or long conversation can increase the first wait because the system must process more context before it starts generating.

Context is helpful, but it costs capacity

More context can help the model answer with more information.

But context is not free. More tokens require more compute and memory. More irrelevant context can also confuse the answer.

A good prompt is not always the longest prompt. It is the right information in the right order.

A fluent answer is not always a checked answer

The model is built to continue text.

That does not mean it checked a database, inspected your production logs, or verified a command. Unless the app connects the model to tools or trusted sources, the answer is generated from the provided context and learned patterns.

This is why a model can sound confident and still be wrong.

Safety and permissions are part of the product

A production AI chat system is more than the model.

There may be input filters, output checks, policy prompts, classifiers, rate limits, logging, tool permissions, and human approval steps.

If an assistant refuses, answers strangely, or cannot access something, the cause might be the model. It might also be the application around the model.

Cost follows tokens

Many hosted LLM systems charge for input and output tokens.

Long prompts, long histories, and long answers all matter. Token count is both a pricing concern and a capacity concern.

For teams building AI features, token usage is not trivia. It affects latency, memory, throughput, and cost.

Practitioner takeaway

For DevOps, platform, SRE, and AI engineering readers, the useful translation is:

• Prompt length affects prefill. Longer context can increase time to first token.

• Output length affects decode. More generated tokens means more loop iterations.

• Streaming hides some latency, but not compute. The user sees progress while the backend still works.

• Token accounting is capacity planning. Tokens drive cost, memory pressure, queueing, and throughput.

• Policy sits around the model. Refusals, tool limits, and access failures may come from the harness, not only from the model weights.

Key takeaways

• ChatGPT is a request pipeline. The text box hides assembly, tokenization, inference, selection, and streaming.

• Tokens are the working unit. The model reads and writes small text pieces, not whole thoughts.

• The answer is built in a loop. One token is chosen, appended, and used to choose the next.

• Streaming is partial delivery. The answer can appear before generation is complete.

• Fluency is not verification. A generated answer still needs source-of-truth checks for real operational work.

Sources

• Vaswani et al., “Attention Is All You Need”: https://arxiv.org/abs/1706.03762

• Hugging Face Transformers documentation, generation strategies: https://huggingface.co/docs/transformers/en/generation_strategies

• Hugging Face Tokenizers documentation: https://huggingface.co/docs/tokenizers/main/en/index

• OpenAI API documentation, text generation: https://platform.openai.com/docs/guides/text-generation

• OpenAI API documentation, streaming responses: https://platform.openai.com/docs/guides/streaming-responses

• OpenAI tiktoken tokenizer repository: https://github.com/openai/tiktoken

In the same room, barely an hour later, another engineer said something I now hear everywhere: “AI is moving fast, and I feel like I am standing outside the room.”

Two different complaints. One answer. You do not go back to VMs, and you are not outside the AI room either. You go one level down, to the open standard underneath Docker. And that same standard, it turns out, is your on-ramp into AI. Let me show you why.

The three roads to AI Native

The way I see it, there are three roads a DevOps engineer can take into AI. I call this the AI Trinity for DevOps:

1. AI-Augmented DevOps. You use AI to do your existing job better. Copilots for your pipelines, agents that triage your alerts, AI that writes your Terraform.

2. AI Infrastructure. That is MLOps, LLMOps, and AgentOps. You build and run the platforms that AI workloads live on. Someone has to serve the models, wire the vector databases, package the artifacts, and keep the whole thing secure. That someone looks a lot like you.

3. Agentic DevOps. You build agents and agentic systems that do engineering work themselves.

This series walks the second road. And here is the important point: the second road does not start with mathematics or model training. It starts with something you already know very well. Containers.

You already have the skills. Really.

Ask yourself what an AI stack actually needs in production. It needs a model served behind an API. It needs a database. It needs an app. It needs images built, versioned, pushed to a registry, scanned, signed, and shipped through CI.

Sound familiar? It should. This is your day job with one new workload type on top.

So who is best placed to run AI infrastructure? Not the data scientist who trained the model. The engineer who has been packaging, serving, isolating, and shipping software for years. That is you.

But there is one decision to get right before anything else, and it is the subject of the rest of this article.

Container-native, not Docker-native

Imagine you are standing at a shipping yard. A crane lifts a steel box off a ship and sets it down on a waiting truck. Stop here for a second and notice what nobody at the yard is doing. Nobody opens the box to check what is inside. Nobody asks which company owns the truck. The crane operator would handle ten thousand of these boxes a year without knowing what a single one contains. Why? Because the box follows a standard. Every crane, every ship, and every truck in the world is built for the same box.

Your application is the cargo. The container spec is the box. And Docker, Colima, OrbStack, Rancher Desktop, and Podman? They are just the carriers. The yard does not belong to any one of them.

Now, why does this matter suddenly? Because Docker Desktop is now paid for organisations with more than 250 employees or 10 million dollars in revenue. That is the email my workshop participant got. That one pricing change broke a lazy assumption the whole industry had been carrying: that “container” means “Docker Desktop”.

But here is the good news. The standard underneath Docker is fully open. It has two parts:

• The OCI image format, that is the standard for how container images are built and stored.

• The Compose Spec, that is the standard for describing a multi-service stack in one file.

Every serious runtime implements both. So the same compose.yaml would run without any changes on Colima, OrbStack, Rancher Desktop, or Podman.

One file. Four runtimes. The same result. So the rule for this whole series is simple: we learn container-native, not Docker-native. Which carrier you pick is your business.

What do containers actually buy an AI stack?

Think of a container as a sealed shipment. The model server, the embedding pipeline, the vector database, and the agent each travel in their own sealed box. Any machine can open them.

Concretely, containers give an AI system four things:

Role What it means in practice Package Lock the Python version, the CUDA driver, and every library version. “Works on my machine” becomes “works on every machine”. Serve Run the embedding service, the vector DB, and the gateway behind predictable ports. Nothing pollutes the host. Isolate Two AI frameworks with clashing dependencies? Each lives in its own container. No virtualenv juggling. Ship Push to any OCI registry, that is GHCR, Docker Hub, or Quay. Pull and run anywhere.

Package, serve, isolate, ship. Keep these four words in mind. Every article in this series is about one or more of them, applied to AI.

The one wrinkle: your laptop’s GPU

Now here is the catch, and I want you to know it exists before you run anything.

On Apple Silicon Macs, a container cannot see the GPU. macOS gives the container’s virtual machine CPUs and memory, but no graphics device. So a model running inside a container on a Mac would fall back to CPU, and what takes one second natively would take three to six in the container. You cannot fix this with configuration. It is a hard platform boundary.

The pattern that solves it: run the model server natively on the Mac, where it can use the Metal GPU, and containerise everything else. The containers reach the model at one magic address:

http://host.docker.internal:11434

.

The next article is entirely about this. For now, just keep the picture in mind: the model stays native, everything else is a container.

Prove it yourself in a few minutes

Enough reading. Let’s prove the wiring works. We assume you have a container runtime installed (any of the four above) and Ollama installed natively.

Pull a small model. This one is under 1 GB:

ollama pull qwen2.5:1.5b

Check it is being served natively:

curl -s http://localhost:11434/api/tags

Now the real move. Start a throwaway container and call the model from inside it:

docker run --rm curlimages/curl:latest -s \
  http://host.docker.internal:11434/api/generate \
  -d '{"model":"qwen2.5:1.5b","prompt":"Say hi in 5 words.","stream":false}'

You should get back JSON with a "response" field and a short reply from the model. Stop here for a second and notice what just happened. A container, which is your world, just called a locally served LLM, which is the new world, over a standard HTTP API. That is the whole bridge. Everything we build in this series stands on it.

Where this series goes

Over the next thirteen weeks we will build one realistic system, one step at a time, and every step runs on a normal 16 GB laptop:

1. Serve a local model behind an OpenAI-compatible API, and swap engines without touching app code

2. Scale it with vLLM and understand batching and quantization

3. Package and version models as OCI artifacts, the same way you ship images

4. Build a RAG assistant over real runbooks, and see exactly where naive RAG breaks

5. Define an agent in plain Markdown files, with tools and guardrails

6. Grow it into a multi-agent incident crew

7. Secure the whole thing: guardrails, SBOM, scan, sign, eval

8. Ship it end to end, and prove portability by swapping the runtime

No GPU cluster and no cloud bill. Just the open container standard, open-source AI tools, and the skills you already have.

If you understand this well, the rest will be easy to figure out. See you in the next one.

This series is adapted from my 2-day hands-on workshop, Containers for GenAI & Agentic AI — The Open-Source Way, where we build this entire stack step by step on a regular 16 GB laptop. If your team wants to run it live, reach out. And if you want the next article, subscribe — it’s free.

Uber’s method is simple. Pair an AI-proficient engineer with a domain expert. Shadow the real work. Find high-impact opportunities. Build an agent. Validate it with users. Then ship.

The strongest lesson is this: the workflow is the unit of automation, not the individual task.

That fits operations well. Our work is full of repeated checks, manual investigation and context gathering. But operations also has a larger blast radius. So the method needs stronger boundaries, better testing and gradual automation.

Start with the work humans should not be doing

The first agentic operations team should look more like an L1 SRE or associate DevOps team than a group of autonomous senior engineers.

This is where agents can create immediate value. They can take on boring, repeatable and manual work. Work that is necessary, but not a good use of human attention.

• Search 100,000 log lines for the one abnormal pattern.

• Collect incident context from alerts, metrics, logs and recent changes.

• Enrich and deduplicate alerts.

• Check why a Kubernetes workload is pending.

• Verify a deployment and compare before-and-after signals.

• Investigate a failed CI/CD or ML pipeline.

• Draft an incident timeline or handover.

• Run safe, well-defined parts of a runbook.

• Prepare capacity, reliability and cost reports.

These are not trivial problems. They often consume hours because the evidence is spread across many tools. Agents are good at searching, collecting, correlating and summarizing that evidence.

Build agents the way we build teams

A profile should represent a role. A skill should represent a capability. A workflow should combine skills to achieve a goal.

Profile = Role • Skill = Capability • Workflow = Work • Goal = Outcome

Profiles map to real roles. Examples include SRE for Team X, PlatformOps Engineer, Deployment Engineer, Kubernetes Operations Engineer or MLOps Engineer.

The profile defines the agent’s scope, services, tools, permissions, memory, policies and escalation path. An SRE for Team X should know Team X’s services, dashboards, SLOs, repositories, clusters and on-call process.

Skills map to functions and capabilities. Examples include querying logs, collecting incident evidence, inspecting Kubernetes workloads, checking recent deployments, detecting configuration drift, calculating SLO impact, preparing a rollback plan or drafting an incident report.

The same skill can be reused by many profiles. A Kubernetes inspection skill may be used by an SRE, a PlatformOps engineer and an MLOps engineer. Each profile uses it with different context and permissions.

A goal combines several skills. To investigate a rise in checkout-service errors, the SRE profile may collect alert context, search logs, inspect Kubernetes, check recent deployments, correlate metrics, explain the likely cause and prepare the next action.

A compact way to adapt Uber’s method

Pick one high-toil, low-risk workflow. Read-only investigation is usually the best first target.

Pair the operator with an agent builder. The operator knows the real work. The builder knows models, tools and automation.

Shadow the workflow. Capture the hidden checks and judgement that are missing from the runbook.

Create a profile for the role. Add only the context, tools and permissions that role needs.

Convert the runbook into small skills. Each skill should have clear inputs, tools, expected evidence, constraints and an escalation path.

Test in shadow mode. Compare the agent’s output with real incidents and experienced engineers.

Move up the safety ladder slowly. Add approvals, audit logs and bounded execution only after the workflow proves reliable.

Use Hermes to experiment, not to skip governance

Hermes can be a useful starting harness because it supports separate profiles, memory, sessions, tools and skills. That maps well to role-based operational agents.

For example, you could create profiles such as sre-team-x, platform-ops, deployment-engineer, mlops-team-y.

Start with a few shared skills: collect incident context, search logs, inspect Kubernetes, check deployments and draft a report. Then test them against real cases.

Do not measure success only by autonomy

Most teams can get strong value before an agent is allowed to change production.

Observe: gather evidence.

Explain: summarize what is happening.

Recommend: suggest the next step.

Prepare: draft commands, patches or change plans.

Execute with approval: act only after a human reviews the exact action.

Levels one to four can remove a large amount of toil. Full autonomy is not the goal. Reliable help is.

The practical recommendation

Do not begin by building an autonomous SRE platform. Begin by building a small agentic operations team for L1 work.

Create profiles that match real roles. Give them reusable skills. Combine those skills into clear workflows. Start with read-only access. Validate on real incidents. Keep humans responsible for important decisions.

Let agents handle the noise. Let engineers handle the impact.

Uber’s Agentic Pods method gives us a useful operating model. The operations adaptation adds the pieces our field needs: role boundaries, reusable capabilities, deterministic tools, shadow testing, approvals and auditability.

Start small. Automate the boring. Build trust. Then expand.

Sources and further reading

Uber Engineering: Agentic Pods overview

Google SRE: Eliminating Toil

Hermes Agent: Profiles documentation

Hermes Agent: Skills system

The newest AI employee apparently costs $20, $100 or perhaps $200 a month.

It writes code. It analyses logs. It creates Terraform. It investigates incidents. It never sleeps, never asks for a promotion and never complains about being on call.

Place that next to the salary of a senior SRE, DevOps engineer or platform engineer, and the conclusion appears obvious:

The engineer is an expensive legacy interface. The agent is the future.

There is only one problem.

The comparison is economically meaningless.

A flat-rate AI subscription is the price of giving one person access to a model under usage controls. It is not the cost of operating an autonomous production worker that continuously reads telemetry, retrieves context, invokes tools, retries failed actions, validates results, maintains state, produces audit evidence and remains available during an incident.

Replacing an engineer means replacing more than the visible actions they perform.

You must replace the work, the context, the judgement, the coordination and the accountability.

Once we calculate that system honestly, the $200 AI employee starts looking less like an employee and more like a heavily discounted demonstration.

Welcome to Signal Over Hype

This is the first edition of Signal Over Hype, a new Agentic Ops Dispatch series for DevOps, SRE, platform, infrastructure and AI engineers.

The purpose of this series is not to dismiss AI.

I use these systems every day. I believe AI will automate a substantial portion of operational work. Engineers who learn to work effectively with agents will have a serious advantage over those who ignore them.

But engineers are increasingly being asked to make career, architecture and workforce decisions in an environment filled with vendor incentives, executive predictions, benchmark theatre and social-media fear and FOMO.

In each edition, we will take one popular claim and examine it using:

Facts.

First principles.

Production experience.

Economic incentives.

And ordinary engineering common sense.

The recurring question will be simple:

What remains true after the keynote ends and the system has to run in production?

The job-displacement claim is not imaginary

In May 2025, Anthropic CEO Dario Amodei warned that AI could eliminate half of entry-level white-collar jobs within one to five years and push unemployment significantly higher. The statement was framed as a warning rather than a celebration, but it contributed to a powerful narrative: organisations should prepare for rapid labour replacement because the technology is moving faster than workers or governments realise.

It would be foolish to dismiss this simply because the prediction came from the CEO of an AI company.

The models are improving quickly.

METR’s research has found that the difficulty of software tasks frontier agents can complete with 50% reliability has increased rapidly over several years. Its researchers originally estimated a doubling in the measured task horizon roughly every seven months. METR also stresses an important qualification: this measures the difficulty of a task based on how long a human would take to complete it. It does not mean that an AI agent can safely run unattended in production for that many hours.

There will be displacement.

Some teams will become smaller. Some entry-level activities will disappear. Some companies will use AI to reduce hiring. Others will use AI as the explanation for cuts they already wanted to make.

But we should also notice that the industry narrative is not uniform.

OpenAI’s own 2025 employment paper said that the evidence available at the time pointed towards AI helping developers do more rather than simply replacing them. Anthropic’s 2026 labour-market research similarly reported limited evidence of broad employment effects so far, although it found suggestive evidence that hiring of younger workers may have slowed in highly exposed occupations.

That is the signal:

AI can perform more engineering work, and the labour market is beginning to adjust.

The hype begins when we jump from that observation to:

A $200 subscription can replace an SRE.

The first trick: compare a seat licence with a salary

Claude Max currently offers an individual plan priced at $200 per month with higher usage capacity than its standard plans.

That is real.

It is also almost entirely irrelevant to the cost of replacing an engineer.

Anthropic describes the Max plan as an individual-user product. It separately states that Claude subscriptions and Claude API usage are different products with different billing. Additional API or usage-credit consumption is charged independently of the subscription.

This distinction matters.

A person using Claude Code interactively is not the same workload as an autonomous operations platform.

A person chooses a task, provides context, waits for the result, inspects the answer and stops when the work is complete.

A production agent may:

Wake up for every alert.

Read logs, traces, dashboards, manifests, tickets and runbooks.

Send much of that context back to the model at every reasoning step.

Launch parallel subagents.

Retry failed tool calls.

Ask another model to review the first model.

Retain state across a long-running incident.

Create an evidence trail for every action.

Operate when nobody is watching.

And escalate to a more capable model whenever the cheaper one becomes confused—which, as production engineers know, is usually shortly before the dangerous part.

The $200 plan is not deceptive.

It is simply the wrong unit for the comparison.

Comparing an AI seat licence with the salary of an engineer is like comparing the price of Microsoft 365 with the cost of running the finance department.

The viral 320-million-token receipt

The image that inspired this article compares a $200 Claude subscription with an $8,000 API bill for 320 million tokens.

The arithmetic can be correct under a particular assumption.

Anthropic currently lists Claude Opus 4.8 at $5 per million input tokens and $25 per million output tokens. At those rates, 320 million output tokens would indeed cost $8,000. The same number of input tokens would cost $1,600.

But “320 million tokens” by itself is not a workload description.

You need to know:

Which model was used?

How many tokens were input versus output?

How much repeated context was cached?

How many tool results were added back into the conversation?

How many retries occurred?

How many agents operated in parallel?

Was the workload interactive, batched or long-running?

Was an expensive model used for everything, including tasks a smaller model or a twenty-line script could have handled?

Using the published Opus rates, a workload containing 90% input and 10% output would cost roughly $2,240 for 320 million total tokens—not $8,000.

A cheaper model could reduce it further. Prompt caching and batching could reduce some costs again. Poor context management, excessive retries and uncontrolled subagents could send them in the opposite direction.

So the honest conclusion is not:

Your $200 subscription secretly costs $8,000.

The honest conclusion is:

A subscription price tells you almost nothing about the cost of a production agent until you define the model, input-output ratio, context, caching, concurrency, retry behaviour and workload.

That is less viral.

It is also true.

Anthropic’s own Claude Code documentation says enterprise deployments average around $150–$250 per active developer per month, although costs vary significantly with model choice, codebase size, automation and concurrent usage.

That is important counterevidence.

Useful AI assistance can be economically attractive.

But notice the unit again:

Per developer.

That is the cost of augmenting a human.

It is not the cost of replacing the organisational function that the human performs.

An SRE is not a command-typing machine

The original SRE concept was never based on paying humans to repeat shell commands forever.

Google defines toil as repetitive, predictable and automatable operational work. Its SRE organisation has historically aimed to keep operational toil below half of an engineer’s time, leaving at least 50% for engineering work that reduces future toil or improves reliability, performance and utilisation.

In other words:

The role includes:

Designing systems that fail safely.

Defining SLOs and error budgets.

Building delivery and rollback mechanisms.

Reviewing production readiness.

Managing capacity and cost.

Improving observability.

Coordinating incidents.

Understanding dependencies.

Negotiating risk with product and business teams.

And creating systems that need less human intervention over time.

A model can generate a kubectl command.

That is not the same as knowing whether the command should be run.

It does not automatically know whether the evidence is complete, whether the operation is reversible, what the blast radius is, which customer commitment is at risk, whether the database can tolerate the failover or who has authority to accept the consequences.

The pager is not requesting text generation.

It is requesting ownership.

The five real costs of replacing an SRE

1. Model access

This is the cost everyone talks about:

Input tokens.

Output tokens.

Reasoning tokens.

Cache reads and writes.

Tool-call context.

Web searches.

Multimodal input.

Retries.

Parallel agents.

Background jobs.

And escalation to premium models.

It may still be much cheaper than an engineer’s salary.

That is entirely possible.

But it is only the first line of the bill.

2. The execution harness

A production agent requires an operating environment around the model.

You need identity and access control.

Secrets management.

Tool or MCP gateways.

Policy enforcement.

Approval workflows.

State and memory.

Scheduling and event triggers.

Sandboxed execution.

Model routing.

Evaluations.

Observability.

Audit logs.

Rate and cost limits.

Rollback mechanisms.

Provider failover.

And a way to stop the agent when it begins confidently fixing the wrong problem.

Google’s SRE guidance describes automation as a force multiplier, not a panacea. Thoughtless automation can create as many problems as it solves.

LLMs make this principle more important, not less.

Traditional automation is generally wrong in repeatable ways.

Probabilistic automation can be wrong creatively.

3. Human review

Automation does not eliminate labour whenever someone must inspect, approve or correct its output.

A more useful formula is:

Net automation value = labour avoided − review labour − correction labour − platform cost − expected failure cost

Suppose an agent saves an engineer forty minutes.

It then requires twenty minutes of review.

Once every ten runs, it creates an hour of correction work.

The productivity benefit is not forty minutes.

And that is before accounting for infrastructure, token usage, security controls and context switching.

Research on AI productivity is not one-directional.

DORA’s 2024 findings associated AI adoption with improvements in individual productivity, flow and job satisfaction, while also finding negative relationships with software delivery stability and throughput. Its 2025 work described AI more positively as an amplifier of the surrounding organisational system: strong teams benefit more, while weak processes become faster at producing downstream chaos.

METR’s 2025 study of experienced open-source developers found that participants took 19% longer when permitted to use AI tools on mature projects they understood well. Strikingly, the developers believed they had become faster.

Both results can be true.

AI can accelerate well-scoped work while slowing experts in high-context environments where prompting, reviewing, correcting and steering the tool becomes expensive.

Production operations is one of the highest-context environments we have.

4. Failure and blast radius

A wrong answer in a document creates editing work.

A wrong production action can:

Delete data.

Expose credentials.

Break routing.

Corrupt state.

Increase cloud spending.

Violate a regulatory control.

Extend an outage.

Or create a second incident while attempting to resolve the first.

Uptime Institute’s 2026 analysis found that 57% of respondents said their most recent major outage cost more than $100,000. One in five said it exceeded $1 million.

You do not need many autonomous-agent mistakes at that price before the salary comparison starts looking childish.

The expected failure cost is not:

Probability of a model error × cost of an API request.

It is:

Probability of an unsafe decision escaping your controls × business impact of the action.

That is why dry runs, bounded permissions, staged execution, canaries, approvals and rollback are not bureaucratic additions.

They are part of the product.

5. Knowledge and accountability

Experienced engineers hold knowledge that is only partly documented.

Why does the architecture look unnecessarily complicated?

Because the simpler version failed during a peak event four years ago.

Why has nobody removed that temporary workaround?

Because it quietly became load-bearing.

Why is one alert routinely ignored?

Because it lies.

Why is another apparently harmless warning taken seriously?

Because it appears fifteen minutes before the system collapses.

Which customer cannot tolerate a five-minute disruption?

Which deployment order is dangerous?

Which dashboard stays green during the precise failure you fear?

An AI system can retrieve recorded knowledge.

It cannot reliably retrieve what the organisation never recorded.

If a company removes engineers before converting that knowledge into runbooks, executable skills, tests, policies, dependency maps and incident history, it is not eliminating cost.

It is deleting state.

Accountability is harder still.

NIST’s generative-AI risk guidance recommends defining human-AI responsibilities, applying suitable human oversight, and maintaining review, tracking and documentation appropriate to the risk.

When an agent recommends deleting a cluster, who approves the decision?

When it is wrong, who owns the incident?

When the regulator asks who accepted the risk, do we provide the model ID?

Follow the money—but do not invent a conspiracy

It is tempting to tell a neat story:

AI companies are subsidising subscriptions now.

They want organisations to become dependent.

Once companies fire their employees, token prices will rise.

The vendors will collect the difference forever.

Parts of this are plausible.

The full story is not yet proven.

Technology companies frequently use flat-rate plans, free tiers, credits and generous usage limits to accelerate adoption. Production workloads then move towards metered infrastructure, enterprise contracts and consumption-based billing.

But model prices can also fall as hardware, inference software and competition improve. Customers can route workloads to cheaper models. Caching and batching can reduce costs. Open models and local inference create alternatives.

The stronger, evidence-based argument is this:

Subscription access and API consumption are different products.

Production automation tends to expose real usage costs.

Operational integration creates switching costs.

And once your company’s runbooks, workflows, memory, tooling and approvals are built around one provider, that provider gains commercial leverage—even when the cost per token falls.

The largest risk is not necessarily that a token becomes more expensive.

The risk is that your organisation becomes unable to operate without a particular model, memory format, proprietary agent runtime or vendor-controlled tool ecosystem.

We have seen this movie before.

This time the lock-in comes with a friendly chatbot.

What will actually be automated

A large portion of today’s operational toil is highly automatable:

Alert enrichment and deduplication.

Log and trace summarisation.

Incident timeline construction.

Runbook drafting.

Routine script generation.

Configuration and policy checks.

Change preparation.

Documentation.

Dependency lookup.

Ticket triage.

Known-issue diagnosis.

Post-incident summaries.

Capacity analysis.

And execution of constrained, reversible procedures.

Some of this should use frontier models.

Some should use smaller or local models.

Some should use conventional code.

Calling the most expensive model available to parse a known JSON document is not AI-native engineering.

It is a refusal to write twenty lines of Python.

The goal is not maximum AI usage.

The goal is the minimum reliable cost for the required outcome.

What should remain human-led—for now

Most serious organisations should continue to keep human ownership around:

Incident command.

Architecture decisions.

Reliability and cost trade-offs.

Risk acceptance.

Security-sensitive changes.

Ambiguous diagnosis.

Cross-team coordination.

Customer communication.

Novel failure modes.

Regulatory accountability.

And any action whose blast radius exceeds the system’s ability to recover safely.

The boundary will move.

Agents will become more capable. Evaluations will improve. Organisations will gather successful production history. Permissions and policies will become more granular.

But the correct way to move this boundary is through evidence:

Tested historical incidents.

Measured success rates.

Controlled production deployments.

Bounded permissions.

Demonstrated rollback.

And clear escalation paths.

Not because an executive said the future was arriving next quarter.

Can AI reduce the size of an SRE team?

Yes.

That is an entirely reasonable outcome.

AI may reduce the number of engineers needed for some kinds of systems.

But this becomes responsible only when:

The service has clear SLOs.

Operational knowledge is documented.

Common incidents have tested runbooks.

Deterministic actions use deterministic code.

Agent permissions are narrowly scoped.

High-risk actions require approval.

Changes are staged and reversible.

Telemetry is trustworthy.

The agent is evaluated against historical incidents.

Costs are measured end to end.

Humans can take control.

And the remaining team has enough capacity to improve the system instead of spending every day supervising bots.

It is not responsible when the organisation has:

Weak observability.

Undocumented systems.

Shared administrator credentials.

Unreliable tests.

No rollback process.

No incident-command discipline.

Unclear ownership.

And a management strategy that can be summarised as:

Claude will figure it out.

The more chaotic the environment, the more impressive an AI demonstration can appear.

It is also the environment in which autonomous execution is most dangerous.

The architecture that makes economic sense

The sensible Agentic DevOps architecture is not:

Send every operational event to the largest frontier model available.

It is layered.

Deterministic code for deterministic work

Use scripts, APIs, controllers, workflow systems and policy engines when the rules are known.

A model does not need to decide how to restart a service when the procedure is already deterministic, tested and safe.

Skills for reusable operational knowledge

Convert runbooks and procedures into versioned skills containing:

Instructions.

Scripts.

Expected evidence.

Preconditions.

Tests.

Permissions.

Escalation rules.

And rollback steps.

This turns institutional knowledge into a portable operational asset rather than a collection of documents an agent may or may not interpret correctly.

Smaller or local models for predictable cognition

Use efficient models for:

Classification.

Extraction.

Routing.

Summarisation.

Known-pattern diagnosis.

And low-risk decision support.

These workloads do not always need frontier reasoning.

Frontier models for genuinely hard problems

Use the most capable models when ambiguity, novel incidents, architecture analysis or complex synthesis justifies the additional cost.

The expensive model should be an escalation path.

Not the default parser for every log line.

A governed execution harness around everything

Add:

Identity.

Policy.

Approvals.

Observability.

Evaluations.

Budgets.

Audit.

Isolation.

Rollback.

And human escalation.

This architecture reduces dependence on any one model vendor.

It also allows the organisation to improve its operational system while models continue to change.

Models are replaceable components.

Your skills, telemetry, controls and institutional knowledge are the durable assets.

The verdict

AI will change SRE, DevOps and platform engineering profoundly.

It will eliminate toil.

It will compress some roles.

It will allow smaller teams to run larger systems.

It may eventually operate significant production environments with limited human supervision.

But replacing an SRE with a chatbot subscription is not transformation.

It is an accounting illusion.

The real comparison is not:

Engineer salary versus AI subscription.

It is:

The fully loaded cost and risk of a human-led engineering system versus the fully loaded cost and risk of an AI-led engineering system.

Sometimes the AI-led system will win.

Frequently, the best answer will be a smaller and stronger engineering team operating a well-governed automation platform.

And in immature organisations, removing the engineers first may be the most expensive decision of all.

So automate aggressively.

Measure token consumption.

Measure review time.

Measure failed actions.

Build reusable skills.

Use local models where they are sufficient.

Use frontier models where they earn their cost.

Require human approval where the blast radius demands it.

But do not fire the people who understand production because somebody compared their salary with the promotional price of a model subscription.

Fire the toil. Keep the SRE. Modernise the pager.

Because the pager does not care about the keynote.

And when it rings at 3:17 in the morning, somebody still owns the outcome.

About Signal Over Hype

Signal Over Hype is an Agentic Ops Dispatch series, curated by Gourav Shah, examining the claims shaping DevOps, SRE, platform engineering, AI infrastructure and agentic operations.

No reflexive optimism.

No reflexive pessimism.

Just facts, incentives, first principles and production reality.

Subscribe now

Sources and further reading

Anthropic, “Choose a Claude plan” and documentation covering subscription versus API billing.

Anthropic, Claude Platform pricing and Claude Code cost guidance.

Anthropic, “Labor market impacts of AI: A new measure and early evidence,” March 2026.

OpenAI, “Jobs in the Intelligence Age,” September 2025.

METR, research on AI task-completion horizons and experienced developer productivity.

DORA, 2024 and 2025 research on AI-assisted software development.

Google, Site Reliability Engineering: “Eliminating Toil” and “The Evolution of Automation at Google.”

NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.

Uptime Institute, Annual Outage Analysis 2026.

What would an Ansible-like system look like if it were designed for AI-native operations?

Not “AI writes a shell script once.”

Not “chat with a bot and hope it remembers your infra.”

I mean something closer to the way real ops teams work: roles, teams, responsibilities, reusable capabilities, project context, runtime permissions, workflows, and operating models.

That led to today’s build: AOH: Agentic Ops Harness.

Repo: https://github.com/agenticdevops/aoh

AOH is my first cut at a Git-native harness for designing and running agentic DevOps, SRE, Platform Engineering, and MLOps capabilities.

The Thought Process

The first instinct was to model “skills.”

That made sense because tools like Hermes, Goose, Claude Code, Codex, and others are converging on some form of reusable agent skill: instructions, scripts, references, and runtime behavior bundled together.

But very quickly, “skills” alone felt too small.

In real organizations, we do not just have a pile of skills. We have teams.

A platform team has SREs, DevOps engineers, platform engineers, maybe MLOps engineers. Each role has different responsibilities, different tools, different access patterns, and different operating habits.

So the model evolved:

Org / Business Unit / Project
  -> Team
    -> Role
      -> Skills / Capabilities
      -> Workflows
      -> Runtime Requirements
      -> Model Profile

That felt much closer to reality.

An SRE role might get:

• service health reporting

• incident triage

• log analysis

• Kubernetes read-only diagnostics

A DevOps automation role might get:

• deployment automation

• Terraform plan review

• release verification

• rollback workflows

An MLOps role might get:

• training job triage

• GPU utilization checks

• model deployment diagnostics

• checkpoint/retry guidance

Same team. Different roles. Different capabilities.

That became the core abstraction.

What I Built Today

I built the first working MVP of AOH: Agentic Ops Harness.

Right now AOH is a Git repository structure plus a small CLI/compiler.

It contains:

• Packs: source-of-truth bundles for agentic ops capabilities

• Teams: org/project/BU-level groups

• Roles: real-world job functions like SRE, DevOps Engineer, MLOps Engineer

• Skills: reusable agent capabilities written as SKILL.md

• Workflows: repeatable ops flows

• Runtime requirements: tools and capabilities needed by a role/workflow

• Model profiles: model/provider intent

• Adapters: compilers into agent runtimes

The first runtime adapter is for Hermes Agent.

For Hermes, AOH compiles:

AOH Team -> multiple Hermes profiles
AOH Role -> one Hermes profile
AOH Skills -> profile-local Hermes skills
AOH Role instructions -> SOUL.md
AOH launch -> launch.sh

So a team definition can generate multiple runnable Hermes agents.

The First Real Example

I created an example pack:

examples/acme-platform-ops

It models an imaginary Acme Platform Ops Team with three roles:

platform-ops team
  -> sre-platform
  -> devops-automation
  -> mlops-training

Each role has a different capability set.

The SRE role gets:

service-health-report
docker-disk-cleanup

The DevOps role gets:

deployment-automation
terraform-plan-review
service-health-report

The MLOps role gets:

ml-training-job-triage
service-health-report

That is the important part: the agent is not just “an AI assistant.” It is a scoped operational role with a curated capability set.

Why This Matters

Most AI tooling today is still very individual-agent centric.

You launch a coding agent. You give it a task. It works in a repo.

That is useful, but DevOps and SRE work is team-shaped.

We need AI-native ops systems that understand:

• who the agent is acting as

• what team it belongs to

• what project or business unit it serves

• what capabilities it should have

• what workflows it can run

• what runtime should execute it

• what tools it needs

• what model profile makes sense for the job

AOH is trying to make that structure explicit and version-controlled.

How It Works

At the source level, AOH is just files in Git.

A simplified pack looks like this:

acme-platform-ops/
  AOH.yaml
  teams/
    platform-ops.yaml
  agents/
    sre-platform.yaml
    devops-automation.yaml
    mlops-training.yaml
  skills/
    service-health-report/
      SKILL.md
    deployment-automation/
      SKILL.md
    ml-training-job-triage/
      SKILL.md
  workflows/
  models/
  runtime-requirements/
  evals/

Then the CLI validates and compiles that pack into runtime-native artifacts.

For Hermes:

uv run aoh install-hermes-team examples/acme-platform-ops \
  --profiles-dir ~/.hermes/profiles \
  --team platform-ops \
  --profile-prefix acme-platform \
  --provider openai-codex \
  --model gpt-5.4 \
  --cwd "$PWD"

That creates Hermes profiles like:

acme-platform-sre-platform
acme-platform-devops-automation
acme-platform-mlops-training

Each profile has its own role instructions, local skills, config, and launch script.

Where We Are Now

The MVP is working.

I tested the generated Hermes profile for the DevOps role:

~/.hermes/profiles/acme-platform-devops-automation/launch.sh \
  -q "Answer in one sentence: what AOH role are you, and which AOH skills are associated with you?" \
  --max-turns 2 --quiet

The generated agent responded:

I’m the AOH DevOps Engineer for Acme Platform in the devops-automation role,
and my associated AOH skills are deployment-automation, terraform-plan-review,
and service-health-report.

That is a small test, but it proves the shape:

• team defined in Git

• role defined in Git

• skills associated with role

• Hermes profile generated

• role-specific agent launched

• associated skills preloaded at runtime

That is the harness.

What This Is Not Yet

This is not a finished platform.

It is not yet a full registry, package manager, or policy engine.

It does not yet have adapters for Goose, Codex, Claude Code, or OpenCode.

It does not yet have a complete eval runner.

It does not yet model enterprise-grade approvals, secrets, RBAC, or audit trails deeply.

But the foundation is now there.

The source-of-truth model is becoming clear.

What Comes Next

The next steps I’m thinking about:

1. Goose adapter
   Map AOH skills and workflows into Goose skills, recipes, sub-recipes, and extensions.

2. Codex and Claude Code adapters
   Compile AOH roles into project instructions, skills, and launch conventions.

3. Eval runner
   Validate whether a generated role-agent can actually perform the workflow it claims to support.

4. Registry
   Share AOH packs across teams and organizations.

5. Runtime requirement negotiation
   Let a runtime say: “I can satisfy these requirements, but not these.”

6. Policy metadata
   Keep AOH engine-neutral but allow adapters to map risk, approvals, and tool access into runtime-native guardrails.

Why I’m Excited About This

This feels like a useful bridge between traditional automation and AI-native operations.

The old world had:

roles
playbooks
inventories
modules
runbooks

The new world needs:

teams
agent roles
skills
workflows
runtime adapters
evals

AOH is an attempt to make that transition concrete.

Not as a slide.

As a repo you can clone and run.

Try it here:

https://github.com/agenticdevops/aoh

If you’re a DevOps Engineer today, you already have half the foundation built — cloud, automation, and operational excellence. But the other half? That’s a whole new skill set in MLOps, LLM orchestration, GenAI agents, and AI governance.

Here’s a practical, month-by-month roadmap to take you from DevOps to LLMOps in under a year.

Stage 1 – Strengthen Your DevOps & Cloud Core (Months 1–2)

Before you tackle AI workloads, you need airtight operational skills.

• Cloud Platforms: Deepen AWS/GCP/Azure expertise. Focus on IAM, networking, cost control.

• Containers & Orchestration: Docker best practices, Kubernetes deployments, RBAC.

• CI/CD Pipelines: Automate deployments with GitHub Actions, GitLab CI, or Jenkins.

• Observability: Master Prometheus, Grafana, and distributed logging.

Mini Project: Deploy a multi-service app with CI/CD, monitoring, and RBAC on Kubernetes.

Stage 2 – Enter the MLOps Arena (Months 3–4)

Understand how models are trained, deployed, and maintained.

• MLOps Tools: MLflow for tracking, DVC for versioning.

• Serving Models: FastAPI + Docker, Seldon Core, BentoML.

• Pipelines: Automate training, evaluation, deployment.

• Monitoring: Detect drift, track inference performance.

Mini Project: Deploy an ML model with MLflow registry, serve it via FastAPI, monitor it in Grafana.

Stage 3 – Specialize in LLMOps & Agent Workflows (Months 5–6)

Now it gets exciting — orchestrating GenAI at scale.

• LLM APIs: OpenAI, Anthropic, Cohere, Azure OpenAI.

• Agent Frameworks: LangChain, LlamaIndex, LangGraph.

• Vector Databases & RAG: FAISS, Pinecone, Weaviate.

• Optimization: LoRA fine-tuning, quantization, caching.

• Hybrid Orchestration: Multi-LLM routing and fallback strategies.

Mini Project: Build a multi-agent RAG system with LangGraph + Pinecone that uses two different LLM providers for redundancy.

Stage 4 – Governance, Security & Responsible AI (Month 7)

The part most engineers overlook — and the one employers value most.

• Governance: NIST AI RMF, GDPR, EU AI Act basics.

• Responsible AI: Fairness, explainability, bias detection.

• Security: Secure API gateways, data encryption, sandboxing.

• Model Safety Monitoring: Detect hallucinations, toxicity, performance drops.

Mini Project: Build a governed LLMOps pipeline with IAM, encryption, hallucination detection, and audit logging.

Capstone Project – Your Portfolio Booster

“Secure, Governed, Multi-LLM Agent Platform”

• Multi-agent orchestration with LangGraph

• Hybrid LLM integration (OpenAI + Anthropic)

• RAG with Pinecone

• MLflow-tracked LoRA models

• Governance and monitoring baked in

This is the kind of end-to-end, security-conscious AI platform employers are hiring for right now.

Why This Works

• You build on what you already know (DevOps skills)

• You stack MLOps before LLMOps (no skipping steps)

• You end with governance (where real enterprise adoption happens)

Start your MLOps/LLMOps Journey

Guest: Malcolm Matalka, Co-founder Terrateam
Host: Gourav Shah, Founder of School of DevOps & AI

GitOps has transformed the way we manage infrastructure and applications, but often, it’s seen as synonymous with Kubernetes and tools like ArgoCD and FluxCD. In this episode of the RealOps Podcast, we expand beyond Kubernetes and explore GitOps for Infrastructure as Code (IaC) with Terraform and OpenTofu. Joining us is Malcolm, co-founder of Terrateam, a company dedicated to bringing GitOps principles to Terraform and beyond.

The Story Behind Terrateam

Malcolm and his co-founder, Josh, saw a critical gap in the industry: existing Terraform automation tools were either rigid enterprise solutions that slowed teams down or hobbyist open-source projects that lacked robustness. They built Terrateam, an open-source GitOps-based infrastructure automation tool, to bridge this gap. Their goal? To create a flexible yet powerful workflow that integrates seamlessly with Git-based operations, ensuring teams can move fast without compromising security or stability.

What Makes Terra Team Unique?

1. Full GitOps Workflow for Terraform and OpenTofu

   • Unlike traditional CI/CD tools, Terra Team integrates deeply into your Git-based workflows, ensuring infrastructure changes are proposed, reviewed, tested, and applied all within pull requests.

2. Fine-Grained Access Control and Workflow Customization

   • Enterprises often struggle with Terraform automation because of coarse access control and rigid workflows. Terra Team provides workspace-level access control, meaning different environments (dev, prod) can have distinct policies.

3. Cost Visibility with OpenInfraQuote (earlier InfraCosts) Integration

   • One of the most exciting features is cost estimation built into pull requests. Before applying a change, teams can see the projected cost impact, helping them make informed decisions.

   • In fact, Terrateam is replacing the earstwhile InfraCost integration with their own brand new open source project OpenInfraQuote, which is 100% open-source and requires no server (https://github.com/terrateamio/openinfraquote).

4. Scalability and Security

   • Unlike tools like Atlantis that run everything on a single server, Terrateam offloads compute to GitHub Actions and other CI/CD systems, ensuring scalability while keeping sensitive credentials within your environment.

The Rise of OpenTofu

The conversation also delved into OpenTofu, a community-driven fork of Terraform that emerged after HashiCorp changed Terraform’s licensing model. Malcolm, a founding member of OpenTofu, explained how it remains fully compatible with Terraform while introducing enhancements like state encryption and dynamic provider configuration.

For those wondering whether to switch from Terraform to OpenTofu, Malcolm’s advice was simple: stick with Terraform 1.5.7 if it meets your needs, but keep an eye on OpenTofu as it evolves. The transition between the two is seamless, ensuring no lock-in for users.

Why GitOps for Infrastructure Matters

GitOps isn't just about Kubernetes—it’s about bringing clarity, control, and automation to infrastructure changes. With Terrateam, Git becomes the single source of truth for Terraform, OpenTofu, and Pulumi deployments. This means:

• No manual SSH-ing into machines to apply Terraform changes.

• No risk of drift between actual infrastructure and what’s in Git.

• Clear audit trails and approval workflows.

Learn More about OpenTofu, Terrateam, and GitOps

To continue your journey into GitOps for Infrastructure as Code, be sure to explore the official resources of the tools we discussed:

• Terrateam: https://github.com/terrateamio/terrateam

• OpenTofu: https://opentofu.org/

These websites provide documentation, tutorials, and community guides that offer deeper insights into how GitOps principles can be applied to Terraform workflows and infrastructure management.

Also, don't miss the fact that this episode includes a live demo of Terrateam's PR-based GitOps workflow in action. It's worth watching to see firsthand how a pull request triggers and manages Terraform changes, giving you a practical understanding of why this approach is so powerful in real-world scenarios.

Finally, we encourage you to engage with the OpenTofu and Terrateam communities. By connecting with these communities (via forums, GitHub, or social channels), you can stay updated on the latest best practices and innovations in the IaC and GitOps space – and continue learning as these tools and techniques evolve.

Getting Started with GitOps for Terraform

If you’re an IT professional, DevOps engineer, or cloud infrastructure specialist, now is the time to level up your skills in GitOps and Terraform. The School of DevOps offers hands-on training programs to help you master Terraform, GitOps, and cloud-native infrastructure.

🎯 Start your journey with our expert-led courses today! 👉 Join the School of DevOps

🔊 Listen to the full episode on the RealOps Podcast to dive deeper into GitOps for Infrastructure as Code!

Guest: Vanita Mohite, Principal Production Engineer, Yahoo
Host: Gourav Shah, Founder of School of DevOps

🔹 Episode Summary:

In this insightful episode of DevOps Career Talks, we sit down with Vanita Mohite, Principal Production Engineer at Yahoo, to explore what it takes to transition from DevOps to SRE and Production Engineering. Vanita shares her 15+ years of experience at Yahoo, highlighting the evolution of infrastructure from on-premise data centers to AWS and Kubernetes (EKS), the critical role of observability, and the challenges of managing large-scale production systems.

Yahoo, despite no longer being at its peak, remains a powerhouse in Site Reliability Engineering (SRE) and Production Engineering. Vanita walks us through her career journey, from working with databases at IBM to becoming a Principal Production Engineer, leading major cloud migrations, automation projects, and resiliency strategies.

🔹 Key Takeaways from the Episode:

1️⃣ The Path from DevOps to SRE/Production Engineering

• Vanita's early career in IBM’s Tivoli Backup team, scripting in Perl and Java for test automation.

• Moving to Yahoo and starting with database administration, evolving into automation and infrastructure scaling.

• SRE vs. DevOps: How production engineering at Yahoo involves designing, building, scaling, and maintaining critical systems.

2️⃣ Building & Managing Large-Scale Infrastructure at Yahoo

• Yahoo’s hybrid infrastructure: On-prem data centers, AWS, GCP, Kubernetes (EKS).

• Managing 1,000+ hosts, handling millions of user requests per hour.

• Scaling Yahoo Help and internal applications with high availability and performance tuning.

3️⃣ Observability & Incident Management in Production Engineering

• Role of monitoring & alerting: Custom Nagios-based monitoring, Splunk for logs, Datadog & New Relic for APM.

• The importance of incident response & on-call: 30% of time spent on observability, monitoring, and post-mortem analysis.

• SLA, SLI, SLO deep dive: Understanding availability contracts and how they affect business.

4️⃣ The Transition to Cloud & Automation

• Migrating legacy applications to AWS & Kubernetes while maintaining custom internal stacks.

• Automating CI/CD pipelines with Jenkins, Maven, and Ant.

• Evolution of DevOps practices at Yahoo and lessons from building an enterprise-scale CI/CD system.

5️⃣ Avoiding Production Outages: Lessons from Mistakes

• Vanita shares a memorable production outage due to server restarts in the wrong environment. 😨

• How color-coding terminals for Dev, Staging, and Prod became a game-changer in preventing future issues.

• Why GitOps and modern DevOps tools reduce human errors.

6️⃣ The Rise of FinOps: Cost Optimization in Production Engineering

• Understanding FinOps: Tracking AWS cost projections for infrastructure.

• How Yahoo’s custom dashboards help teams optimize resources in real-time.

• The analogy of FinOps as a fitness tracker – what you measure, you optimize.

🎧 Why Listen to This Episode?

✔ Are you a DevOps Engineer exploring SRE or Production Engineering? This episode breaks down what you need to know.
✔ Curious about managing large-scale production systems? Get behind-the-scenes insights into Yahoo’s global infrastructure.
✔ Want to learn about career growth in DevOps, SRE, and Production Engineering? Vanita shares practical lessons, mistakes, and strategies.

👉 Tune in now and take your DevOps career to the next level!

📢 Follow the RealOps Podcast on YouTube & Substack!

Taking Devops to Next Level

If you are a Devops Professional and want to take it to the next level, check out our programs on DevSecOps, Advanced Devops, MLOps and more at campus.schoolofdevops.com

Take your Devops Career to Next Level

Chapters

• 00:00 Vanita's Journey into Production Engineering / SRE

• 01:49 Transition to Yahoo and SRE / ProdOps Role

• 05:25 The Role of a Production Engineer

• 17:02 Infrastructure Management and Scaling

• 22:26 Monitoring and Availability Strategies

• 24:02 Monitoring and Incident Management in DevOps

• 28:01 Understanding SLAs, SLIs, and SLOs

• 32:55 Learning from Human Errors in Production

• 37:30 The Evolution of DevOps and Collaboration

• 42:40 Cost Management and Optimization in Cloud

• 47:34 The Impact of Cloud Transformation on Startups

• 48:39 The Importance of Documentation in Tech Projects

• 50:47 Learning from Incidents and Continuous Improvement

• 53:13 The Role of Automation in Production Engineering

• 56:19 Balancing Work and Family as a Woman in Tech

• 59:09 Navigating a Career Path in SRE and Production Engineering

• 01:04:26 Essential Skills and Tools for Aspiring SREs

• 01:09:53 Future Trends: MLOps and AIOps

• 01:12:07 Outro

From Pune to California, Pranav's path is packed with lessons on mastering cloud tools, CI/CD pipelines, and platform engineering. Whether you're just starting or looking to level up, this episode has something for everyone.

🎧 Episode Highlights

In this episode, Pranav and I discuss:

• His start in DevOps and how it all began

• Key skills he acquired with AWS, Jenkins, Bitbucket, Docker Swarm, and Kubernetes

• The shift from traditional CI/CD setups to container orchestration with Helm and Kubernetes

• How platform engineering differs from DevOps engineering

• Insights into DevSecOps and MLOps

• Career tips for aspiring DevOps engineers

Listen in to gain an insider’s perspective on what it takes to thrive in DevOps today!

🚀 Interested in launching your DevOps career?
Check out the DevOps Career Launchpad Program from the School of DevOps! This program is designed to help IT professionals build essential skills in Linux, CI/CD, Docker, Kubernetes, and more, with hands-on labs, expert coaching, and live workshops. If you’re ready to jumpstart your journey to becoming a DevOps Engineer, click here to learn more and enroll.