[PATCH v2 0/6] Introduce a shared base for application VMs

Alyssa Ross
2022-10-10T23:33:21Z

This series was originally developed for some work I'm finishing up
now for NLnet, for letting Spectrum users build VMs on the system with
Nix, so it's time for it to get another round.

Changes since v1:

• make-vm.nix only generates the VM's configuration directory,
not a whole data/$name hierarchy that needs to be merged.

• vm-lib/make-vm.nix and vm/make-vm.nix are separated, so
system-provided VMs can deduplicate against the base image, while
user-defined VMs can't so they're independently upgradeable.

v1: https://spectrum-os.org/lists/archives/spectrum-devel/20220919073659.1703271-1-hi@alyssa.is/

The idea here is to reduce duplication between application VMs, both
in terms of source code size and output size. After this change,
creating a new VM just requires writing a very small Nix file like
this:

{ config ? import ../../../nix/eval-config.nix {} }:

import ../../vm-lib/make-vm.nix { inherit config; } {
name = "appvm-lynx";
providers.net = [ "netvm" ];
run = config.pkgs.pkgsStatic.callPackage (
{ writeScript, lynx }:
writeScript "run-lynx" ''
#!/bin/execlineb -P
${lynx}/bin/lynx https://spectrum-os.org
''
) { };
}

Rather than a whole big source tree as before, most of which was
duplicated with every other application VM.

When a VM generated this way is started, it gets two disk images. One
is the shared base image, which is part of the Spectrum base system,
and the other contains only the application-specific stuff: the run
script, and any store path dependencies that are not already present
in the base image. This means that the amount of storage required for
each new application VM is substantially reduced.

Of course, this isn't the only way to generate VMs. Monolithic VMs
like we had before would still work, with some small adjustments for
the new disk layout.

I also see this fitting well into making it possible to configure
extra VMs at build time. It doesn't directly help with that, but
making it so that each VM doesn't need to provide everything itself
will make creating external VMs easier when it does happen.

In future we might want to apply a similar mechanism to service VMs,
like netvm, but since we only have one of those so far, it's not clear
which parts exactly would be duplicated, so I'm leaving it for now.

Other future work is considering the impacts of the shared base image
on guest isolation. Can guests observe whether reads of the shared
base image hit the host page cache, or even an internal disk cache?
At the moment I suspect that the base image doesn't have enough
specialised code in it that there would be any interesting results,
but it's worth thinking about if the shared image grows new
functionality, whether it would be interesting to another guest to
be able to observe whether those resources have previously been loaded
or not. If this _does_ turn out to be a concern, it could be
mitigated by simply copying the base image to temporary storage before
booting a VM, and then booting the VM from the copy.

Alyssa Ross (6):
host/start-vm: support multiple block devices
scripts/make-gpt.sh: add support for labels
vm: build GPT images
host/start-vm: boot using partition label
release: rename from "img"
img/app: extract from appvm-{lynx,catgirl}

Documentation/creating-vms.adoc | 8 +-
Documentation/getting-spectrum.adoc | 2 +-
host/initramfs/extfs.nix | 19 +--
host/rootfs/default.nix | 11 +-
host/start-vm/lib.rs | 38 +++++-
host/start-vm/tests/vm_command-basic.rs | 6 +-
{vm/app/lynx => img/app}/Makefile | 57 ++++----
{vm/app/catgirl => img/app}/bin | 0
{vm/app/lynx => img/app}/default.nix | 22 ++--
img/app/etc/fstab | 8 ++
{vm/app/catgirl => img/app}/etc/init | 0
{vm/app/catgirl => img/app}/etc/mdev.conf | 0
{vm/app/lynx => img/app}/etc/mdev/iface | 2 +-
{vm/app/catgirl => img/app}/etc/passwd | 0
.../catgirl => img/app}/etc/passwd.license | 0
{vm/app/catgirl => img/app}/etc/resolv.conf | 0
.../app}/etc/s6-linux-init/scripts/rc.init | 1 +
.../s6-rc/lynx => img/app/etc/s6-rc/app}/run | 3 +-
.../catgirl => img/app/etc/s6-rc/app}/type | 0
.../app/etc/s6-rc/app}/type.license | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
.../app}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
.../app}/etc/s6-rc/mdevd-coldplug/up | 0
.../app}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
.../catgirl => img/app}/etc/s6-rc/mdevd/run | 0
.../catgirl => img/app}/etc/s6-rc/mdevd/type | 0
.../app}/etc/s6-rc/mdevd/type.license | 0
.../app}/etc/s6-rc/ok-all/contents | 0
.../catgirl => img/app}/etc/s6-rc/ok-all/type | 0
.../app}/etc/s6-rc/ok-all/type.license | 0
.../app}/etc/ssl/certs/ca-certificates.crt | 0
{vm/app/lynx => img/app}/shell.nix | 11 +-
release.nix | 2 +-
{img => release}/combined/default.nix | 0
{img => release}/combined/eosimages.nix | 0
{img => release}/combined/grub.cfg.in | 0
{img => release}/combined/run-vm.nix | 0
...ble-gpt-partition-attribute-55-check.patch | 0
...pt-disable-partition-table-CRC-check.patch | 0
.../0003-install-remove-Endless-OS-ad.patch | 0
...4-finished-don-t-run-eos-diagnostics.patch | 0
...omote-spectrum-not-the-Endless-forum.patch | 0
{img => release}/installer/app/default.nix | 0
.../installer/app/vendor-customer-support.ini | 0
{img => release}/installer/configuration.nix | 0
{img => release}/installer/default.nix | 0
{img => release}/installer/run-vm.nix | 0
{img => release}/installer/seat.rules | 0
{img => release}/live/Makefile | 0
{img => release}/live/default.nix | 0
{img => release}/live/shell.nix | 0
scripts/make-gpt.sh | 4 +-
scripts/sfdisk-field.awk | 2 +-
vm-lib/make-vm.nix | 51 ++++++++
vm/app/catgirl.nix | 17 +++
vm/app/catgirl/Makefile | 123 ------------------
vm/app/catgirl/default.nix | 92 -------------
vm/app/catgirl/etc/fstab | 6 -
vm/app/catgirl/etc/mdev/iface | 36 -----
.../catgirl/etc/s6-linux-init/scripts/rc.init | 10 --
vm/app/catgirl/etc/s6-rc/catgirl/run | 31 -----
.../data/appvm-catgirl/providers/net/netvm | 0
vm/app/catgirl/shell.nix | 17 ---
vm/app/lynx.nix | 15 +++
vm/app/lynx/bin | 1 -
vm/app/lynx/etc/fstab | 6 -
vm/app/lynx/etc/init | 5 -
vm/app/lynx/etc/mdev.conf | 5 -
vm/app/lynx/etc/passwd | 1 -
vm/app/lynx/etc/passwd.license | 2 -
vm/app/lynx/etc/resolv.conf | 4 -
vm/app/lynx/etc/s6-rc/lynx/type | 1 -
vm/app/lynx/etc/s6-rc/lynx/type.license | 2 -
.../etc/s6-rc/mdevd-coldplug/dependencies | 4 -
vm/app/lynx/etc/s6-rc/mdevd-coldplug/type | 1 -
.../etc/s6-rc/mdevd-coldplug/type.license | 2 -
vm/app/lynx/etc/s6-rc/mdevd-coldplug/up | 4 -
vm/app/lynx/etc/s6-rc/mdevd/notification-fd | 1 -
.../etc/s6-rc/mdevd/notification-fd.license | 2 -
vm/app/lynx/etc/s6-rc/mdevd/run | 5 -
vm/app/lynx/etc/s6-rc/mdevd/type | 1 -
vm/app/lynx/etc/s6-rc/mdevd/type.license | 2 -
vm/app/lynx/etc/s6-rc/ok-all/contents | 4 -
vm/app/lynx/etc/s6-rc/ok-all/type | 1 -
vm/app/lynx/etc/s6-rc/ok-all/type.license | 2 -
vm/app/lynx/etc/ssl/certs/ca-certificates.crt | 1 -
.../host/data/appvm-lynx/providers/net/netvm | 0
vm/make-vm.nix | 9 ++
vm/sys/net/Makefile | 23 ++--
vm/sys/net/default.nix | 10 +-
92 files changed, 236 insertions(+), 457 deletions(-)
rename {vm/app/lynx => img/app}/Makefile (66%)
rename {vm/app/catgirl => img/app}/bin (100%)
rename {vm/app/lynx => img/app}/default.nix (77%)
create mode 100644 img/app/etc/fstab
rename {vm/app/catgirl => img/app}/etc/init (100%)
rename {vm/app/catgirl => img/app}/etc/mdev.conf (100%)
rename {vm/app/lynx => img/app}/etc/mdev/iface (98%)
rename {vm/app/catgirl => img/app}/etc/passwd (100%)
rename {vm/app/catgirl => img/app}/etc/passwd.license (100%)
rename {vm/app/catgirl => img/app}/etc/resolv.conf (100%)
rename {vm/app/lynx => img/app}/etc/s6-linux-init/scripts/rc.init (90%)
rename {vm/app/lynx/etc/s6-rc/lynx => img/app/etc/s6-rc/app}/run (80%)
rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type (100%)
rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type.license (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/dependencies (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type.license (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/up (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd.license (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/run (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type.license (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/contents (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type (100%)
rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type.license (100%)
rename {vm/app/catgirl => img/app}/etc/ssl/certs/ca-certificates.crt (100%)
rename {vm/app/lynx => img/app}/shell.nix (51%)
rename {img => release}/combined/default.nix (100%)
rename {img => release}/combined/eosimages.nix (100%)
rename {img => release}/combined/grub.cfg.in (100%)
rename {img => release}/combined/run-vm.nix (100%)
rename {img => release}/installer/app/0001-gpt-disable-gpt-partition-attribute-55-check.patch (100%)
rename {img => release}/installer/app/0002-gpt-disable-partition-table-CRC-check.patch (100%)
rename {img => release}/installer/app/0003-install-remove-Endless-OS-ad.patch (100%)
rename {img => release}/installer/app/0004-finished-don-t-run-eos-diagnostics.patch (100%)
rename {img => release}/installer/app/0005-finished-promote-spectrum-not-the-Endless-forum.patch (100%)
rename {img => release}/installer/app/default.nix (100%)
rename {img => release}/installer/app/vendor-customer-support.ini (100%)
rename {img => release}/installer/configuration.nix (100%)
rename {img => release}/installer/default.nix (100%)
rename {img => release}/installer/run-vm.nix (100%)
rename {img => release}/installer/seat.rules (100%)
rename {img => release}/live/Makefile (100%)
rename {img => release}/live/default.nix (100%)
rename {img => release}/live/shell.nix (100%)
create mode 100644 vm-lib/make-vm.nix
create mode 100644 vm/app/catgirl.nix
delete mode 100644 vm/app/catgirl/Makefile
delete mode 100644 vm/app/catgirl/default.nix
delete mode 100644 vm/app/catgirl/etc/fstab
delete mode 100755 vm/app/catgirl/etc/mdev/iface
delete mode 100755 vm/app/catgirl/etc/s6-linux-init/scripts/rc.init
delete mode 100755 vm/app/catgirl/etc/s6-rc/catgirl/run
delete mode 100644 vm/app/catgirl/host/data/appvm-catgirl/providers/net/netvm
delete mode 100644 vm/app/catgirl/shell.nix
create mode 100644 vm/app/lynx.nix
delete mode 120000 vm/app/lynx/bin
delete mode 100644 vm/app/lynx/etc/fstab
delete mode 100755 vm/app/lynx/etc/init
delete mode 100644 vm/app/lynx/etc/mdev.conf
delete mode 100644 vm/app/lynx/etc/passwd
delete mode 100644 vm/app/lynx/etc/passwd.license
delete mode 100644 vm/app/lynx/etc/resolv.conf
delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type
delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type.license
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/dependencies
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type.license
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/up
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd.license
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/run
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type
delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type.license
delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/contents
delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type
delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type.license
delete mode 120000 vm/app/lynx/etc/ssl/certs/ca-certificates.crt
delete mode 100644 vm/app/lynx/host/data/appvm-lynx/providers/net/netvm
create mode 100644 vm/make-vm.nix

base-commit: 7a6d44e24ddcc9cba73deed25fb85038b7c3d823
--
2.37.1