Operational Overhead of Running Hundreds of Concurrent Agent Micro-VMs on Bare Infrastructure

Firecracker micro-VM isolation for AI agents gives each sandbox its own kernel, using hardware virtualization. That boundary suits untrusted agent code, but running hundreds of these VMs on bare infrastructure means owning image pipelines, scheduling, networking, state, recovery, and security maintenance. The operational overhead can outweigh the raw compute bill.

How do you give each AI agent its own isolated kernel instead of a shared container kernel?

Containers share the host’s Linux kernel. Namespaces and cgroups separate workloads and control resources, but they do not provide an independent kernel boundary. A reachable kernel vulnerability can therefore expose the host and other tenants. For an agent executing code no human reviewed, that exposure includes credentials, other tenants’ data, and the network.

The 2024–2025 CVEs involving runc, BuildKit, and NVIDIA Container Toolkit included paths to host filesystem access, although some required build-time exploitation. Docker Desktop’s exposure came through its bundled runc and BuildKit components. These differences affect how an attack works, while containing untrusted execution remains necessary.

A micro-VM puts a separate guest kernel behind hardware virtualization using Intel VT-x or AMD-V. Compromising that guest does not itself compromise the host kernel. Escaping requires a flaw in the virtualization boundary, a rarer and more expensive class of attack. For this workload, micro-VMs are a minimum isolation layer.

Why bare metal or nested virtualization is the entry price, and what each path costs

Firecracker requires read/write access to /dev/kvm. Ordinary cloud VM configurations do not necessarily expose it because they already run under a provider’s hypervisor. Bare metal exposes hardware virtualization directly, while selected cloud instance types support access through nested virtualization.

Bare metal offers meaningfully cheaper raw compute per core than equivalent cloud VMs at on-demand rates. Direct PCIe passthrough is also possible, which is relevant to GPU-accelerated sandboxes. But the infrastructure commitment is less elastic. Monthly billing and slower provisioning come with responsibility for the host operating system, patches, and recovery.

Nested virtualization trades that ownership profile for another cost. Selected instance families at AWS, Google Cloud, and Azure support it, but each VM exit adds work through the nested virtualization layers. Google warns of reduced CPU-bound performance for nested workloads. Concurrent guests that cause many exits are especially exposed, while PCIe passthrough is frequently unavailable. Neither path makes the underlying constraints disappear.

How image management and boot pipelines become a continuous engineering surface

Each micro-VM needs a root filesystem image paired with a kernel. There is no shared container-layer cache to absorb every customization. Packages, tools, and agent dependencies must be built into an artifact or mounted at boot. The fleet therefore depends on a maintained build and distribution pipeline.

Updates make that pipeline a continuing obligation. Guest kernel fixes require new builds distributed to hosts, while userland fixes require image rebuilds and redeployment. Fast boot adds another artifact to manage. Pre-warmed snapshots must be versioned and invalidated when their base image changes, or a newly restored VM can start with stale software.

Storage management grows with that artifact inventory. Snapshots consume space, copy-on-write pages accumulate, and old versions need an explicit lifecycle that limits retention. A practical test is whether the team can identify the kernel version in its oldest live sandbox. Without that visibility, successful builds say little about what the fleet actually runs.

Scheduling hundreds of short-lived VMs is a different problem from scheduling long-lived containers

Agent traffic comes in bursts, and an agent’s next action can create more work with no way to predict when it will stop. A scheduler must account for VM boot latency as well as available compute. It must also preserve mid-task state. Killing an agent sandbox can discard accumulated tool calls that cannot simply be replayed.

That means the team must make big changes to how Kubernetes schedules work. A long-lived service can often be replaced, but an active agent session may need to remain attached to its state. Memory complicates placement further because each idle VM consumes memory before useful workload begins. Dense packing is not just a CPU allocation problem.

Suspension reduces idle memory use but adds lifecycle decisions. The scheduler must distinguish running and suspended sessions, route requests correctly, and place resumed VMs where their snapshots are available or pay to transfer them. Sessions that persist also need host affinity or migration that keeps their state intact. The team owns this software whether it builds the logic into a custom scheduler or uses operators around a general-purpose orchestrator.

Networking hundreds of isolated VMs without leaking state between tenants

Firecracker connects each VM through a tap device. Hundreds of concurrent sandboxes therefore require hundreds of interfaces, address allocations, and associated iptables or nftables rules. Creation and teardown must follow VM lifecycle changes cleanly. Leftover configuration becomes an operational problem, while growing rule counts can introduce traversal latency that reduces sandbox throughput.

Network access also changes the credential threat. An agent that has outbound access and API keys in its environment can leak those keys even if the VM boundary holds. Browser Use dealt with this by putting only a session token, control-plane URL, and session ID in the guest environment. Cloud keys, database credentials, and API tokens were kept outside the sandbox.

That separation creates work outside the guest. A fleet that mediates credential use needs the corresponding proxy or credential-handling layer to keep secrets outside agent code. Workloads that should not reach the internet also need host-enforced egress restrictions. Those restrictions must keep working as VMs appear and disappear. Hardware isolation does not manage any of this configuration.

Snapshot state and checkpoint/resume at scale: the mechanics and the failure modes

Snapshots make fast boot, persistent sessions, and rollout fan-out practical. Firecracker checkpointing pauses execution and saves the state needed for restoration, with memory and filesystem state managed as part of the checkpoint. Without this path, each fresh VM cold-boots and repeats session setup. How much checkpointing saves depends on which step does the work.

DeltaBox, a 2026 paper on Firecracker-based checkpoint/restore, describes that trade-off precisely. Firecracker Diff captures pages dirtied since the previous snapshot, but shifts merging work to restoration. Restore must materialize a sparse copy of the base and overlay changes along the ancestor chain. As a search tree deepens, that chain grows.

Forking introduces a less visible correctness risk. VMGenID causes the Linux guest to reseed its kernel random-number generator on resume, but userspace generators still need explicit reseeding after a fork. Otherwise, rollouts copied from the same snapshot can sample identical actions. Nothing has to crash for the results to be wrong.

State can also span more than the VM. DeepSeek’s DSec work separates rollout execution into an agent sandbox and a worker container that jointly retain complete rollout state. A preempted GPU job can reconnect and continue because that handoff is part of the architecture. Reproducing the behavior requires coordination between scheduling and sandbox state.

Failure recovery when the stateful VM is the product

A stateless service can often retry work elsewhere. An agent host that dies, though, may hold the only copy of a session that is still in progress. Replacing the machine restores capacity. Recovery depends on what state survived outside the failed host and whether it describes where the agent stopped.

Checkpoint frequency therefore affects the recovery boundary. If too much work happens between saved states, a failure can force the task to restart from scratch. Moving a still-running VM off an overloaded host has a different requirement. Migration needs shared storage or a fast transfer of memory state, and the client must be able to bear the wait. It cannot bring back state that was lost when a host died.

If one host fails, it can interrupt hundreds of suspended user sessions, so the team needs monitoring, alerting, and automatic rescheduling to deliver the product. DeepSeek’s state handoff illustrates why recovery belongs in the original design. It is not a property that appears automatically once the fleet supports resume.

Security patching and the ongoing CVE surface of a fleet you own

Isolation and maintenance solve different security problems. The host kernel, virtual machine monitor, guest kernels, and control plane each have their own patch obligations. Firecracker and Cloud Hypervisor have release and CVE cycles independent of the guest software. Pinning a version for stability can leave known vulnerabilities unresolved.

The control channel needs equal scrutiny. CVE-2026-25725, published in February 2026 and associated with Claude Code, demonstrated how sandboxed code could create a malicious settings hook that executed commands on the host. The guest-to-host interface must therefore be treated as adversarial.

Applying patches also consumes operating capacity. Host kernel updates require draining VMs, patching, and rebooting in coordination with the scheduler, while guest updates depend on the image pipeline. Healthcare, lending, and insurance deployments can add HIPAA and DPA obligations to this work. Audits also require records of patches and how long it took to deploy them after disclosure. Security operations continue for as long as the fleet runs.

What the cumulative overhead actually looks like when the layers are added together

Each layer is manageable on its own. The cost grows because changes cross those layers while all of them remain in service. A guest update requires image distribution and snapshot invalidation. To update a host, the team must coordinate scheduling and recovery. Suspension saves memory but makes it more important to store state reliably and route requests to the right place.

Image engineering maintains reproducible artifacts, scheduler development manages lifecycle and placement, and network operations maintain isolation during constant churn. State engineering must handle restore performance and silent fork-correctness bugs. Recovery design keeps progress safe when hosts fail, while security operations keeps the whole stack up to date.

These responsibilities cannot be priced as a one-time installation. Custom operators, lifecycle tooling, recovery paths, and patch automation need maintenance as dependencies and workloads change. There is no supported universal headcount or dollar figure for that commitment. The honest accounting includes the engineering work required to keep these systems correct together, rather than treating the bare-metal invoice as the cost of the service.

What builders who have shipped production micro-VM agent fleets chose instead, and why

Browser Use needed full Chrome environments with strong isolation. It tried Vercel Functions, Kubernetes, Docker, and third-party browser providers, but those options did not meet its required cost and performance for hundreds of concurrent instances. Its published architecture placed the entire agent inside a sandboxed micro-VM. This made the sandbox the unit of execution, rather than a separate destination for occasional code calls.

With managed infrastructure, someone outside the application team runs the fleet. AWS Lambda MicroVMs, announced June 22, 2026, formalized that division by taking a developer-supplied Dockerfile, building an image, and capturing a launch snapshot. AWS manages virtualization, networking, suspension, and termination. The application owner manages what runs inside.

That division still comes with constraints. The configured lifecycle is bounded, including suspended time, and AWS notes that the first request after suspension waits for memory and disk restoration. Resume latency depends on saved state size and the duration of the /resume hook.

Managed compute costs more per vCPU-second than bare metal, but it can take on the work of running the fleet described here. The comparison is between that premium and the cost of building and maintaining the fleet at current and projected scale. Cheap cores are only one side of that calculation.