// SPDX-License-Identifier: GFDL-1.3-or-later OR CC-BY-SA-4.0

= Spectrum Developers’ Guide
Alyssa Ross <hi@alyssa.is>
:toc: left
:nofooter:

== Introduction

The purpose of this book is to assist you in contributing to the
Spectrum compartmentalized operating system.  If, while using this
book, you find information that is outdated, incorrect, incomplete or
difficult to understand, *I implore you to improve it*.  Contributing
to documentation means that your work will form part of the foundation
of the continued development of the project.

Without good documentation, new contributors would not be able to get
involved with the project, existing contributors would not be able to
keep up, and development would slow to a halt.  Spectrum is a
complicated system, and nobody can keep everything they need to know
to work on every part of the system in their head at one time.

== Principles

* Trust as little as possible.  We trust KVM out of necessity, but we
  don't necessarily trust the rest of the kernel, especially security
  features like namespaces that have to be implemented througout --
  there's always the chance somewhere has been missed.

* It should be possible to use Spectrum without using any proprietary
  (non-free) software.  Proprietary software is a security issue,
  because we can't necessarily know what it does, or fix it if it's
  behaving in a way that is incorrect or user-hostile.

== Concepts

=== virtio_wl

virtio_wl, also sometimes called virtwl or virtio-wayland, is a
transport for the https://wayland.freedesktop.org/[Wayland] protocol
over
https://www.ozlabs.org/~rusty/virtio-spec/virtio-paper.pdf[virtio],
which is the standard protocol for KVM VMs to communicate with the
outside world.  It was developed for Chromium OS to allow graphical
applications to be run in VMs, but still be part of the standard
Chromium OS desktop environment.  Currently, this is the only
implementation of virtio_wl.

With virtio_wl, it is possible for a graphical application running in
a VM to interact with a Wayland compositor running outside it.  This
allows Spectrum to present multiple applications, running in seperate
VMs, as part of a unified windowing system, while preventing the
applications from being able to interact with each other beyond what
is permitted by the compositor on behalf of the user.

virtio_wl requires three components to work together to achieve this:

- A *kernel driver*, to handle the virtio communication from the VM to
  the outside world.  In Spectrum, this is the
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-4.19/drivers/virtio/virtio_wl.c[virtio_wl]
  module from the Chromium OS kernel tree.

- A *nested Wayland compositor*, which handles communication between
  Wayland clients (applications) running inside a VM, and the
  virtio_wl device exposed by the host kernel.  As far as the clients
  know, they're talking to a normal Wayland server, rather than
  virtio_wl.
+  
This could also have been implemented as part of the kernel module,
but the authors at Google found it easier to implement this way.
+
In Spectrum, the program that does this is <<_sommelier>>.

- The *Virtual Machine Manager (VMM)*, which connects a Wayland socket
  to the other end of the virtio queue created by the kernel driver.
  The Spectrum VMM, which supports virtio_wl, is <<_crosvm>>.

== Software

=== crosvm

https://chromium.googlesource.com/chromiumos/platform/crosvm/[crosvm]
is the Virtual Machine Monitor (VMM) from Chromium OS.  It is also the
origin of Amazon's
https://firecracker-microvm.github.io/[Firecracker], another
popular VMM.

crosvm is used to run Spectrum's VMs.  There are several reasons for
choosing crosvm over other options like Firecracker or QEMU:

- virtio_wl is unique to crosvm.

- crosvm is written in Rust, a memory-safe language, and so should be
  much less susceptible to memory safety exploits than a VMM written
  in a non-memory-safe language like C.

- crosvm is very focused on running Linux VMs.  This means it is very
  lightweight compared to more general purpose VMs, both in code
  volume and runtime efficiency.

crosvm is somewhat documented in its
https://chromium.googlesource.com/chromiumos/platform/crosvm/+/refs/heads/master/README.md[README].
The generated Rust documentation for crosvm's internals is also a very
useful tool for working with crosvm's source code.  For easy access,
the https://spectrum-os.org/doc/crosvm/crosvm/[crosvm documentation]
for the version currently used by Spectrum is hosted on the Spectrum
website.

=== Sommelier

https://chromium.googlesource.com/chromiumos/platform2/+/master/vm_tools/sommelier[Sommelier]
is a nested Wayland compositor developed for Chromium OS that
implements the user-space side of <<_virtio_wl>>.  It interacts with
the device exposed by the kernel virtio_wl driver (usually
`/dev/wl0`), and exposes a Wayland compositor interface to Wayland
clients running in the VM.

It also supports
https://wayland.freedesktop.org/xserver.html[XWayland], so
applications using the legacy X11 protocol can still be used with
virtio_wl.

Sommelier is somewhat documented in its
https://chromium.googlesource.com/chromiumos/platform2/+/master/vm_tools/sommelier/README.md[README].

The version of Sommelier in Spectrum's nixpkgs has been modified to
support the modern, stable xdg-shell protocol, rather than the
obsolete xdg-shell-unstable-v6 still used by Chromium OS at the time
of writing.  Spectrum's modifications to Sommelier are managed as
patch files applied to the Sommelier package.  These are located in
the
https://spectrum-os.org/git/nixpkgs/tree/pkgs/os-specific/linux/chromium-os/sommelier["sommelier"]
directory in Spectrum's Nixpkgs tree.

=== mktuntap

mktuntap is a program that was written for Spectrum to facilitate
creating virtual (https://en.wikipedia.org/wiki/TUN/TAP[TAP]) network
devices for VMs.  crosvm expects to be given these devices as file
descriptors, rather than as file paths.  This means that Spectrum can
use non-persistent TAP devices, and have the kernel automatically
clean up the device when the last file handle for the device is
closed.  This saves implementing TAP device lifecycle in Spectrum
code.

You can find the development source for mktuntap in its
https://spectrum-os.org/git/mktuntap/[Git repository].  If you would
like to make bug report, suggest an improvement, or send a patch,
please use the
https://spectrum-os.org/participating.html#spectrum-devel[Spectrum
development] mailing list.  mktuntap aims to be a general purpose
utility program, and improvements are therefore welcome even if not
directly required by Spectrum's use case for the program.

[appendix]
== Copyright and License

Copyright 2020 Alyssa Ross.

Permission is granted to copy, distribute and/or modify this document
under either the terms of the
http://creativecommons.org/licenses/by-sa/4.0/[Creative Commons
Attribution-ShareAlike 4.0 International License], or the
https://www.gnu.org/licenses/fdl-1.3.html[GNU Free Documentation
License, Version 1.3] or any later version published by the Free
Software Foundation; with no Invariant Sections, no Front-Cover Texts,
and no Back-Cover Texts.
