Cold Start Latency Benchmarks for Agent VM Runtimes
Agent cold starts stack three delays, and most benchmarks measure only one.

A cold-start number pulled from a standard serverless benchmark table tells a builder almost nothing about how long a real user will wait for an agent to respond. Standard FaaS benchmarks measure one thing: the time from invocation to first handler execution, inside one container, one runtime, one process. An agent is not one process executing once.
The scale of the problem becomes unmistakable once GPU inference enters the picture. Lambda's Firecracker implementation does not support PCIe passthrough, ruling it out for local GPU inference regardless of any other performance characteristic. Even an agent that only calls out to a hosted API, with no local model weights involved at all, pays for SDK initialization and a TLS handshake before it does anything useful, and that overhead alone can push a cold start to several seconds against a warm invocation that runs in the hundreds of milliseconds.
What makes this different from a slow Lambda cold start is structural: the delay comes from three stages that compound: sandbox initialization (image pull, filesystem setup, namespace config), runtime handshake (SDK init, TLS, connection pools), and stateful context restoration (memory, credentials, prior session state), each a standalone cold-start problem that benchmark suites like FaaSdom were not built to capture together, and most published benchmarks measure only one of them. Because of that ambiguity, agent cold-start benchmarks need their own reading discipline, distinct from the one builders learned from serverless functions.
The three layers that stack inside every agent cold start

Every agent cold start is, in practice, three separate latency problems running one after another, and most benchmarks that get published and cited only measure one of them, so treating the three as a single undifferentiated "cold start" number is the single most common misreading in this space, and each layer deserves naming on its own terms.
Sandbox initialization is the cost of getting an isolated execution environment alive in the first place. It covers image pull, filesystem preparation, and namespace configuration, and it is made worse by the fact that LLM serving containers routinely weigh 8 to 15 GB, a size class that ordinary web-handler containers never approach. Isolation technology controls this layer most directly, and the choice of substrate has the largest effect on the headline number a benchmark reports.
Runtime handshake is the cost of getting the agent's language runtime and SDK ready to execute once the sandbox itself is alive. SDK client initialization, whether through Anthropic's client, OpenAI's, or boto3, sets up HTTP connection pools and validates credentials before any application logic runs. A function whose only workload is importing heavyweight LLM client libraries can add several seconds of delay before a single line of the handler executes. Framework choice amplifies this, since swarms-rs, written in Rust, starts in 6 milliseconds, while Swarms and other Python frameworks take seconds to reach the same point, a gap that spans more than two orders of magnitude from framework selection alone. Most agent framework benchmarks report only this layer in isolation, with no accounting for sandbox cost or context restoration sitting on top of it.
Stateful context restoration is the cost of getting the agent back to where it left off. A stateless function has no equivalent to this layer, since it has no memory of anything that came before, but an agent in a multi-turn workflow often finds this the dominant cost of the entire exchange. It covers reloading files, credentials, memory, and whatever session state needs to survive from one invocation to the next; snapshot-restore technology exists specifically to address this layer, discussed in detail below.
The practical consequence of this three-layer structure is that no single benchmark answers the question a builder actually has. A benchmark that measures sandbox boot time says nothing about runtime handshake cost. A framework benchmark says nothing about sandbox overhead. Neither says anything about context restoration. Reading an agent cold-start number requires first identifying which of the three layers it describes, starting with the isolation substrate underneath it.
How isolation technology determines benchmark tier
The isolation mechanism a runtime is built on sets the ceiling and the floor for its Layer 1 latency, and it also determines which other runtimes its published numbers can honestly be compared against. Reading across substrates without accounting for this produces comparisons that look precise and are not.
Firecracker microVMs isolate at the kernel level through KVM hardware virtualization, so a kernel exploit inside one VM cannot reach the host machine or any neighboring VM. That isolation guarantee comes with a hard constraint: Lambda's implementation of Firecracker does not support PCIe passthrough, which rules it out for local GPU inference no matter how favorable its other performance numbers look. Firecracker's cold-boot headline is not, in fact, the number that matters most for production use, a point taken up in full in the next section.
WebAssembly and V8 isolates take a different approach entirely. Rivet's agentOS, launched April 4, 2026, isolates in-process using the same technology that separates tabs inside a Chrome browser. The trade-off is explicit: without full kernel isolation, a kernel-level exploit path exists that simply does not exist inside a Firecracker VM. The benchmark figure attached to this approach compares it against what its source calls "the fastest mainstream sandbox" as of March 30, 2026, reporting a p95 of 950 milliseconds on that comparator. A number like that is only meaningful once the reader knows it is being measured against a specific named comparator rather than against the field broadly.
SmolVM, launched April 17, 2026, is a single-executable microVM, and it is the clearest teaching case for why a better headline number can mask a worse operational profile. At its best-case lower bound, SmolVM's cold start matches or beats Firecracker's headline boot time, but its typical quoted cold start, around 200 milliseconds, runs slower than Firecracker's roughly 125-millisecond headline. SmolVM's range only comes in faster than Firecracker's cold-boot figure in its best case. Reading SmolVM's number as categorically faster than Firecracker's, based on a single favorable data point, is exactly the kind of error this piece is warning against. gVisor applies a user-space kernel to intercept syscalls and sits between full microVM isolation and in-process isolation on the security-versus-overhead spectrum.
Confidential VMs, built on Intel TDX or AMD SEV-SNP, add a different variable entirely: full guest-memory encryption for hardware-backed confidentiality. That encryption disables cross-VM page deduplication and reduces how much memory can be reclaimed, which shrinks the number of warm containers a fixed memory budget can support. Workloads that generate frequent VMEXITs, idle transitions being the most common case, suffer substantial slowdowns that get amplified by the vCPU-to-memory coupling typical of serverless platforms. A confidential VM's numbers are not comparable to a standard VM's numbers even when the nominal configuration, vCPU count and memory allocation, looks identical on paper.
Kubernetes-orchestrated containers introduce yet another tier above raw VM boot time, since scheduler overhead sits on top of whatever the underlying VM or container runtime already costs. Across the full range of substrates and workload tiers discussed here, cold-start latency spans more than two orders of magnitude between the fastest and slowest options. None of this produces a single ranking of "best" isolation technology. It produces a taxonomy: each substrate belongs to its own benchmark tier, trading security guarantees against latency and memory efficiency in a different place, and a builder's job is to identify which tier a given number was measured in before deciding what it implies for a comparable workload.
What snapshot-restore measures for persistent agents
If a runtime is built to support it, snapshot-restore changes which number actually matters. Cold-boot latency on a snapshot-capable runtime is a one-time initialization cost, paid once when a template is first created. The number that governs what a user experiences in practice is restore latency, and that number is frequently absent from benchmark tables that only report cold-boot time.
Snapshot-restore works by having an incoming request restore from a previously stored VM state rather than booting a kernel from nothing, which bypasses the entire Layer 1 cost on every invocation after the first one. Firecracker's restore path achieves latency that is categorically faster than anything available to a substrate lacking the mechanism at all, regardless of how that substrate's cold-boot headline compares on paper. A runtime can take several seconds to cold-boot from scratch but still restore to production-ready latency in well under a second, because restore sidesteps the kernel-boot cost instead of optimizing it.
On a Firecracker-based runtime, this mechanism can become an operating strategy rather than a performance footnote. Only the first spawn of a fresh template pays the roughly three-second cold boot; every create after that pays the much cheaper restore cost instead. Once restore is that cheap, the economically sound move is to stop maintaining a warm pool altogether: destroy the VM when a session ends, free the host for other work, and recreate the agent in well under a second the next time a request lands. Copy-on-write memory makes that pattern cheaper still. Memory from the snapshot is mapped copy-on-write, so a hundred VMs restored from the same template share the base image's pages in memory and each guest only pays for the pages it has individually modified.
AWS AgentCore V2 reached general availability on September 18, 2026, and it applies the identical principle at managed-service scale. V1's latency rises with image size: it grows from a few seconds at the low end to nearly 30 seconds across that range. V2 restores a prepared snapshot rather than pulling and initializing a fresh image on every create, the same snapshot-restore mechanism, now offered as a managed opt-in through the platformVersion: V2 setting. A major managed cloud provider adopted the same pattern independently, and that is strong confirmation that snapshot-restore is becoming the standard answer to Layer 1 cost, not an idiosyncratic optimization specific to one infrastructure vendor.
A cold-start figure published for a runtime that supports snapshot-restore needs to state which path it measured, because a number that blends cold-boot and restore latency together is not usable for production capacity planning. For a runtime that does not support the mechanism at all, such as SmolVM as of its April 2026 launch, the cold-boot number simply is the recurring latency, with no cheaper restore path underneath it to fall back on. Checking whether a runtime supports snapshot-restore at all is the first filter to apply to any cold-start benchmark before comparing it to anything else.

How idle-cost economics shape latency benchmark optimization
Latency architecture matters in the context of how a fleet of agents actually gets used, and utilization profile decides which cold-start number is worth optimizing against. A fleet running at low average utilization has a different cost driver than a fleet running at high, sustained utilization, and the two call for different engineering priorities even on identical infrastructure.
Most agent fleets run at low average utilization. A fleet can carry many concurrently open sessions while only a small fraction of them do active work at any given instant, so most always-on VM-hours go to keeping idle agents alive rather than serving requests. Snapshot-and-delete converts that idle compute cost into storage cost instead, because a snapshot sitting on disk costs far less to hold than a VM sitting idle in memory, and that conversion is why per-user agent economics can work at all when always-on billing would make them financially unworkable. Maritime, a Y Combinator-backed cloud platform that runs each AI agent in its own persistent Firecracker micro-VM and charges for storage rather than idle compute when an agent sleeps, is built around exactly this scheduling model.
Managed-service pricing shows that this trade-off does not resolve itself automatically. AWS AgentCore V2 raised its per-unit rates for both CPU and memory in the us-east-1 region at general availability. Its memory-reclaim feature reclaims unused memory after 120 seconds of idle time, so it only lowers a bill when a session idles long enough after a memory spike for reclaim to fire, and short chat-style turns never run long enough to reach that 120-second window. A builder whose workload consists of brief, frequent exchanges gets none of the benefit that the reclaim mechanism is designed to provide, even while paying the higher per-unit rate that came with V2.
Google's Vertex AI Agent Runtime offers a different shape of pricing for the same underlying problem, billing competitive per-vCPU-hour and per-GiB-hour rates with a monthly free tier, and billing no charge at all for idle time between turns. Set beside AgentCore V2, this comparison shows that the right pricing structure depends on a fleet's session pattern. A fleet made of long sessions with real idle gaps benefits from a platform that reclaims memory during those gaps. A fleet made of short, frequent turns benefits more from a platform that never bills for idle time.
The conclusion that follows from all four layers examined here is that no single cold-start benchmark can tell a builder what to optimize for in isolation. Every agent cold start is actually three separate latency problems running in sequence, and most published benchmarks measure only one of them.
Sources
- Performance Study of Serverless Workloads in Confidential Virtual Machines
- FaaSdom: A Benchmark Suite for Serverless Computing
- Let's Trace It: Fine-Grained Serverless Benchmarking using Synchronous and Asynchronous Orchestrated Applications
- AgentCore Runtime V2 GA (Sep 2026): 1.9s Starts
- Swarms Rust Benchmarks: 6 ms Cold Starts, 3.7 MB of Memory, and 100 Parallel Agents in 0.52 Seconds


