Agent VM Root Filesystem Design With Overlayfs

Firecracker micro-VM isolation gives each AI agent its own guest kernel rather than relying only on namespaces in a shared host kernel. Inside that boundary, an OverlayFS root filesystem separates an immutable base image from private agent changes. Together, these layers provide the right foundation for untrusted agents in production, pairing VM isolation with reversible filesystem state and economical per-agent storage.

Why the root filesystem decision is not an implementation detail

The YoloFS paper documents agents that wiped entire drives, destroyed irreplaceable documents, and silently leaked credentials to attackers. Afterward, an agent’s response could offer remorse but not recovery, including “I am absolutely devastated” and “recovering the data will require whatever backups you have”. The missing protection was below the conversation layer. Without a filesystem recovery primitive, an apology could not undo the operations the agent had already performed.

This risk follows from the way coding agents work. They execute hundreds of file operations and shell commands autonomously with user-level privileges, rather than asking a user to approve each individual mutation. The design of the root filesystem thus sets what a session can change, which changes last, and whether an earlier state is still available. It also determines the storage cost of preserving those states across sessions. Observability, reversibility, and isolation must be built into the operating environment instead of left to how well the model works or follows a prompt.

What OverlayFS is and why its layered model fits agent workloads

OverlayFS presents a merged view of a read-only lower directory and a writable upper directory. The lower layer contains the base image, while modifications accumulate in the upper layer without changing that base. The agent sees a mutable environment, but its changes remain separate from the system image and can be discarded or committed as a unit.

flintlock’s overlay boot design illustrates this arrangement inside a VM. It uses a read-only EROFS base built from an OCI image and a small per-VM writable ext4 disk for the upper layer. The guest assembles the overlay. In this model, rollback to the base means discarding the upper layer, a filesystem snapshot captures that layer, and separate upper layers allow agents to diverge from the same starting image. Capturing the upper layer preserves only filesystem changes.

The YoloFS paper identifies three primitives for an agent-native filesystem, namely introspect effects, undo mutations, and gate accesses. Introspection exposes what a command will do or has done, while undo restores a staged state without requiring the agent to stop. Gating lets users refine access rules progressively. The layer model provides a substrate for these operations. Staging keeps changes in the upper layer, snapshots save the states along the way, and gating controls access across the filesystem boundary.

The isolation gap that the overlay layer alone cannot close

An overlay mounted on a working directory controls mutations only within that mount. An agent that can write elsewhere can bypass the staged environment. The mount’s protection requires user namespaces, which separate the process from the host’s user identifier and privilege mapping, and mount namespaces, which constrain the filesystem view.

AgentFS’s Linux implementation demonstrates that combination with a FUSE overlay implemented in userspace. It creates a mount namespace with unshare(), bind-mounts the overlay onto the working directory, and leaves only explicitly allowed paths writable while remounting other paths read-only with MS_RDONLY. System paths including /proc, /sys, /dev, and /tmp stay writable so the system can work. The isolation design must therefore choose which paths stay writable, since the overlay does not do this on its own.

An agent can ignore instructions, prompt injection can override them, and a filter targeting rm can miss an equivalent operation through Python’s os.remove. The operation’s spelling is not the protection boundary. The kernel enforces path restrictions no matter which command asks for access, so they cover what agent-harness rules alone can miss.

How do you give each AI agent its own isolated kernel with a micro-VM?

A Firecracker microVM places the agent and its root filesystem inside a VM running its own guest kernel. By contrast, namespace-isolated processes still depend on the shared host kernel. That distinction defines the outer isolation boundary. A kernel exploit in a namespace-isolated process targets the host kernel, whereas compromising a guest kernel does not by itself cross the hardware boundary protecting the host and other microVMs.

For multi-tenant agents, this separates two different responsibilities. OverlayFS controls how filesystem state changes inside the guest, while the VM boundary separates that guest’s execution environment from other tenants. A private upper layer does not provide an isolated kernel, and an isolated kernel does not make filesystem mutations reversible.

Firecracker’s practical characteristics make this separation suitable for short-lived and intermittently active sessions. It has sub-second boot and minimal memory overhead per VM, so thousands of VMs can fit on a single server. Memory can also grow during a session through hotplugging, up to a limit configured at startup. It is hard to know ahead of time how much agents will need.

How YoloFS extends the overlay model to give agents structured self-correction

YoloFS turns staging into an explicit control mechanism for agents and users. It holds mutations for review, preserves the original filesystem until commit, and retains intermediate states that agents can use for self-correction. On 11 tasks with hidden side effects, the system enabled agents to self-correct in 8 and staged all mutations for user review. On 112 routine tasks, it needed less input from users while matching the baseline success rate.

Those results support a specific architectural conclusion. Safety need not depend entirely on interrupting the agent more often when the environment can preserve changes for later inspection and reversal. The mechanism also has to remain efficient. To avoid the rename amplification associated with copying a lower-layer file into an upper layer, YoloFS uses a flat content store and an in-memory path-to-backing override tree. Renaming a large file becomes a pointer update.

The same design must prevent access around the staged filesystem. YoloFS’s CLI runs commands in a mount namespace and uses pivot_root to make its mount the root filesystem. The mount confines the agent’s access to the base filesystem, connecting its review and recovery mechanisms to the namespace boundary.

How DeltaBox pushes the overlay model toward high-frequency checkpoint workloads

Tree-search and reinforcement-learning agents place a different demand on the filesystem. They need to checkpoint and roll back repeatedly while exploring alternative states, making remount-based transitions impractical. In standard Linux OverlayFS, the layer stack is fixed when it is mounted. Reconfiguring it requires an unmount/mount cycle, which cannot accommodate the agent’s open files and the checkpoint frequency required by Monte Carlo Tree Search.

DeltaBox, from Shanghai Jiao Tong University / IPADS, addresses this in its May 2026 system with DeltaFS, a custom kernel extension based on Linux 6.8 OverlayFS. DeltaFS exposes an ioctl for hot layer-stack reconfiguration. At a checkpoint, it freezes the existing writable layer and inserts a new writable layer above it. No file duplication or unmount cycle is needed. With reflink-enabled XFS, vfs_clone_file_range clones extents by reference, so an upper file shares physical blocks with its source until they are overwritten.

DeltaCR supplies the complementary process-state mechanism through a userspace daemon. It performs incremental CRIU dumps within the LLM I/O window and keeps a frozen template process from which rollback forks directly. DeltaBox’s checkpoint latency is 14 ms, and its rollback latency is 5 ms. These speeds let agents explore more deeply in the time they have. A StateManager coordinates DeltaFS and DeltaCR inside Firecracker microVMs, although the same optimizations also apply to container- or other VM-based sandboxes.

The integration connects these OS operations to agent state. A Python class implements LangGraph’s BaseCheckpointSaver, with put() calling StateManager.checkpoint() and get() calling StateManager.restore(). Filesystem, process, and graph state can therefore be saved and restored together in a checkpoint-and-restore operation.

Per-agent and per-tenant fleet design: which layers are shared, which are private

Across the fleet, tenants share the read-only lower layer, and each tenant has its own private writable upper layer. AgentFS makes it concrete by using the host filesystem as a read-only base and an embedded relational database as the delta layer for an agent session. Writes copy files into the delta and keep the base intact, while agentfs run mounts the overlay without anyone having to configure it by hand.

Deletions also remain private to the session. A whiteout table records deleted paths, and reads consult that table to treat those paths as absent while leaving base files intact. Multiple agents can join one session through a session ID or use isolated sessions over the same base. The session ID can match a git branch name.

The economic benefit follows from storing the shared image once and retaining only each tenant’s delta. A snapshot-and-restore scheduler suspends idle agents to snapshot storage and wakes them in under a second, making per-user deployment viable without keeping every VM active. When a tenant is idle, most of its storage cost comes from the small delta image. This depends on scheduling and restoration as well as layering, but the filesystem supplies the shared/private decomposition those mechanisms need.

Practical rootfs design decisions and their trade-offs

For an implementation team, the first choice is the base image format. EROFS and SquashFS are both compressed, read-only lower-layer options, with the choice depending on compression ratio and random-access read performance. The upper backing store answers a different question. tmpfs is the fastest option for ephemeral sessions that discard their changes, with zero persistence cost but zero durability. A per-VM ext4 disk preserves changes across reboots and supports snapshots, at the cost of provisioning a small block device per tenant.

SQLite-backed FUSE offers portability and inspection instead. AgentFS’s entire delta is a single file that can be copied, diffed, or attached to a bug report, but FUSE adds overhead on the hot path. For checkpoint-heavy workloads, reflink-enabled XFS is the appropriate underlying filesystem because copy-up can clone extents. Rename behavior also deserves explicit planning. YoloFS’s flat-store design avoids large-file rename amplification, while task designs that avoid renaming large lower-layer files offer another mitigation.

Snapshot depth introduces a separate implementation constraint. Stock OverlayFS has a 50-snapshot kernel limit, so production systems without a custom kernel must plan to run compaction from time to time or limit how deep the search can go. DeltaBox’s hot layer-stack reconfiguration removes that limit at the cost of a custom kernel module. Permission changes require their own design choice. Progressive gating needs a permission model that can be updated without remounting, rather than an assumption that OverlayFS itself supplies adaptable access policy.

What the overlay model still does not solve, and where the frontier sits

Filesystem layering does not resolve the scheduling trade-off between persistent sandbox reservations and lazy creation. Long-lived reservations consume memory, while on-demand instantiation introduces cold-start penalties in multi-tenant, multi-turn workloads. SpecBox, a 2026 research system, addresses this through speculative sandbox preallocation tuned to dynamic LLM agent execution pipelines. Scheduling determines when an environment exists, while layering determines how its filesystem state is represented.

A filesystem snapshot also omits the heap, stack, and open file descriptors. Full checkpoints must couple filesystem and process state atomically to avoid restoring inconsistent versions, as the process-checkpointing work in DeltaCR illustrates. Base-image integrity remains another independent concern. A malicious dependency introduced at build time can already be present in the lower layer, and preserving that layer from agent writes does not verify its integrity at mount time.

OverlayFS-based rootfs design is therefore the best foundation. Agent-native access controls, incremental process checkpoints, and speculative scheduling each build on it in a different way. Firecracker gives that work an isolated guest kernel to run in.