[spectrum-devel] [PATCH www] design: state subdirectories, not block devices

Michael Raskin
2019-10-16T21:25:52Z

> Each virtual machine will be generated by
> a Nix derivation, and will have a
> completely immutable root file system. Persistent storage will be
>-provided by virtual block devices, that arbitrary paths on the system
>-can be mapped to from the host. There may be other writable mount
>-points inside the virtual machine, but these will not persist between
>-reboots of the VM. Using Nix to generate virtual machines allows them
>-to be reproducibly built, rolled back, edited, and migrated as source
>-code, rather than large, opaque virtual machine images.
>+provided by mounting subdirectories of the global state directory into
>+virtual machines. There may be other writable mount points inside the

I guess in the eventual design we want VM-in-jail, i.e. the VM process
itself (and the corresponding FS daemon, if separate from the VM) should
not have any access to the resources not intended for the application.

>+State directories and applications will be m:n.

Note (probably not affecting the design right now): this can also be
a source of annoyance with running everything under unique single-use
UIDs: to have multiple UIDs access the same resource, setting ACLs is
a natural idea. It works well, but might require some cleanup of
obsolete ACLs for a workflow with a large churn of instances ??? the
maximum length of an ACL is limited.

>+Some virtual machines may have no persistent storage, or even write
>+access to a disk, at all. In other cases, it might be desirable for
>+multiple applications to be able to access the same device, such as a
>+local mail store being shared by two mail clients. Other resources
>+and permissions, such as network cards and USB controllers, will
>+similarly be defined in Nix. There are three logical sections for the
>+Nix configuration -- applications, which are just packages, resources
> (virtual or physical devices), and application instances, which
> are mappings between applications and accessible resources. This
> structure allows users to have multiple instances of the same

Given that it is sometimes convenient to spawn multiple VMs with almost
identical settings on the fly (e.g. Firefox instances but with different
state directories for downloads/uploads), should we commit to define
a CLI layer for starting VMs (probably similar to the underlying VMM
software CLI, but adapted to the notions we want to use), with the
Nix-based management of VM fleet built on top of this CLI?

I suspect that building things the other way round is not much simpler,
but building a single VM image usable with slightly different settings
will have smaller overhead than evaluating a Nix expression for each
extra VM.

I guess it didn't make any difference in the previous design version
where the maximum goal was a VM per application, not per application
instance. It does matter now.