Kubernetes Is Being Rebuilt for AI Agents. Your Infrastructure Team Needs to Know.

The theme was hard to miss. At KubeCon North America last week in Salt Lake City, session after session circled back to the same point: Kubernetes clusters are being redesigned to run AI agents as a primary workload, not as an add-on to existing application infrastructure.

The most significant contribution came from Google, which donated a framework called Agent Substrate to the Cloud Native Computing Foundation (CNCF), where it entered Sandbox status. The problem it solves is specific. AI agents spend a lot of time idle, waiting on tool calls, database responses, or input from other agents in the same workflow. Even when doing nothing, those agents still occupy resources on the cluster. Agent Substrate separates an agent’s identity and saved state from its active compute, allowing idle agents to be paused and restored in under 500 milliseconds. A cluster can then pack far more agents into the same infrastructure without the control plane collapsing under thousands of parked, resource-holding workloads.

Security got equal attention. Google also contributed gVisor and OpenRL to restrict what agents can access inside a cluster. NVIDIA contributed OpenShell, which uses default-deny policies to limit what code an agent can execute. When an AI agent generates code as part of its workflow and then runs it, which is a standard pattern in modern AI pipelines, those controls make it significantly harder for the agent to reach infrastructure it was never authorized to touch.

New Relic took a different approach with a Kubernetes Agent it launched in public preview. Rather than restricting what agents do, it watches what happens across a cluster using more than 90 diagnostic detectors, then generates remediation code for an engineer to review and approve. The agent diagnoses. The human decides. That pattern is showing up across multiple tools right now and represents the emerging standard for responsible agentic operations.

The shift here is bigger than any single tool release. Running AI on Kubernetes used to mean standing up inference servers and treating the model as a service endpoint. What is happening now is that AI agents are becoming the application, and infrastructure teams need to manage their lifecycle, isolate their permissions, checkpoint their state between turns, and keep the control plane stable under constant workload churn. That is a different engineering problem than what most platform teams are currently staffed and tooled to handle.

For any organization running Kubernetes in production today, the question isn’t whether this shift will affect your infrastructure. It already is. The platform engineering decisions you make in the next twelve months about agent isolation, state management, and security controls will determine how difficult or smooth that transition turns out to be.

Want to explore how modern DevOps infrastructure could support your AI initiatives? Let’s talk.

Kubernetes Is Being Rebuilt for AI Agents. Your Infrastructure Team Needs to Know.

Leave a Reply

Your email address will not be published. Required fields are marked *