Verified boot and filesystem choices

Verified boot and filesystem choices

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 1112 bytes --]

Bcachefs is not very stable right now, and BTRFS is not a good choice
from a verified boot perspective.  f2fs is what is used in Android
and ext4 is used in Chromebooks, so they at least have the backing of
Google's security team when it comes to vulnerabilities involving
maliciously crafted filesystem images. BTRFS doesn't.

The reason this matters for Spectrum is that verified boot aims to
prevent system compromise from persisting across reboots, and an
attacker who has compromised a Spectrum system can craft whatever
image they want on the writable volume.

Would it make sense to use f2fs or ext4?  That means no reflinks
and no snapshots, which would be annoying at least.  Another option
might be to use FUSE for the writable volume, with kernel filesystems
only used for the (signed and dm-verity protected) root volume.
This is the only option supported by Linux's upstream maintainers,
who (with the notable exception of Kent Overstreet) appear to have
no interest in hardening filesystems against maliciously crafted
images.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Verified boot and filesystem choices

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]

Demi Marie Obenour <demiobenour@gmail.com> writes:

> Bcachefs is not very stable right now,

Neither is Spectrum!  Given that changing filesystem later if it doesn't
work out will be a very easy change to make (up to a point), we can
afford to wait.  It's an approach that has served us well so far —
sometimes focusing on other things means that by the time we have to
look at something, the problem has been solved by somebody else.

Filesystems are always going to have bugs, so in my opinion the most
important thing is to make having good backups easy, so that recovery is
possible when something goes wrong, regardless of choice of filesystem.
I am very keen for Spectrum to have an integrated backup solution,
ideally as easy to use as Time Machine.

> and BTRFS is not a good choice
> from a verified boot perspective.  f2fs is what is used in Android
> and ext4 is used in Chromebooks, so they at least have the backing of
> Google's security team when it comes to vulnerabilities involving
> maliciously crafted filesystem images. BTRFS doesn't.
>
> The reason this matters for Spectrum is that verified boot aims to
> prevent system compromise from persisting across reboots, and an
> attacker who has compromised a Spectrum system can craft whatever
> image they want on the writable volume.
>
> Would it make sense to use f2fs or ext4?  That means no reflinks
> and no snapshots, which would be annoying at least.  Another option
> might be to use FUSE for the writable volume, with kernel filesystems
> only used for the (signed and dm-verity protected) root volume.
> This is the only option supported by Linux's upstream maintainers,
> who (with the notable exception of Kent Overstreet) appear to have
> no interest in hardening filesystems against maliciously crafted
> images.

I think snapshots are going to be very important for us to do things
like the aforementioned integrated backups, and it would be very
unfortunate to have to limit ourselves to out of date filesystems that
lack modern features like that.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Verified boot and filesystem choices

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 2594 bytes --]

On 6/14/25 04:23, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
> 
>> Bcachefs is not very stable right now,
> 
> Neither is Spectrum!  Given that changing filesystem later if it doesn't
> work out will be a very easy change to make (up to a point), we can
> afford to wait.  It's an approach that has served us well so far —
> sometimes focusing on other things means that by the time we have to
> look at something, the problem has been solved by somebody else.
> 
> Filesystems are always going to have bugs, so in my opinion the most
> important thing is to make having good backups easy, so that recovery is
> possible when something goes wrong, regardless of choice of filesystem.
> I am very keen for Spectrum to have an integrated backup solution,
> ideally as easy to use as Time Machine.

To clarify, I'm not referring to bugs that cause data loss, but to bugs
that allow kernel code execution when a maliciously crafted filesystem
is mounted.  Backups don't protect against this.  This attack is mostly
relevant for kiosks, mobile devices, and other cases where being able
to restore trust after a device compromise is critical.

>> and BTRFS is not a good choice
>> from a verified boot perspective.  f2fs is what is used in Android
>> and ext4 is used in Chromebooks, so they at least have the backing of
>> Google's security team when it comes to vulnerabilities involving
>> maliciously crafted filesystem images. BTRFS doesn't.
>>
>> The reason this matters for Spectrum is that verified boot aims to
>> prevent system compromise from persisting across reboots, and an
>> attacker who has compromised a Spectrum system can craft whatever
>> image they want on the writable volume.
>>
>> Would it make sense to use f2fs or ext4?  That means no reflinks
>> and no snapshots, which would be annoying at least.  Another option
>> might be to use FUSE for the writable volume, with kernel filesystems
>> only used for the (signed and dm-verity protected) root volume.
>> This is the only option supported by Linux's upstream maintainers,
>> who (with the notable exception of Kent Overstreet) appear to have
>> no interest in hardening filesystems against maliciously crafted
>> images.
> 
> I think snapshots are going to be very important for us to do things
> like the aforementioned integrated backups, and it would be very
> unfortunate to have to limit ourselves to out of date filesystems that
> lack modern features like that.

That makes sense.  
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Verified boot and filesystem choices

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 1485 bytes --]

Demi Marie Obenour <demiobenour@gmail.com> writes:

> On 6/14/25 04:23, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>> 
>>> Bcachefs is not very stable right now,
>> 
>> Neither is Spectrum!  Given that changing filesystem later if it doesn't
>> work out will be a very easy change to make (up to a point), we can
>> afford to wait.  It's an approach that has served us well so far —
>> sometimes focusing on other things means that by the time we have to
>> look at something, the problem has been solved by somebody else.
>> 
>> Filesystems are always going to have bugs, so in my opinion the most
>> important thing is to make having good backups easy, so that recovery is
>> possible when something goes wrong, regardless of choice of filesystem.
>> I am very keen for Spectrum to have an integrated backup solution,
>> ideally as easy to use as Time Machine.
>
> To clarify, I'm not referring to bugs that cause data loss, but to bugs
> that allow kernel code execution when a maliciously crafted filesystem
> is mounted.  Backups don't protect against this.  This attack is mostly
> relevant for kiosks, mobile devices, and other cases where being able
> to restore trust after a device compromise is critical.

So are you saying that bcachefs's lack of stability means that it's
uniquely vulnerable to this sort of vulnerability?  I'd be surprised,
given that as you say Kent is actually interested in preventing them.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Verified boot and filesystem choices

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 2288 bytes --]

On 6/15/25 05:13, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
> 
>> On 6/14/25 04:23, Alyssa Ross wrote:
>>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>>
>>>> Bcachefs is not very stable right now,
>>>
>>> Neither is Spectrum!  Given that changing filesystem later if it doesn't
>>> work out will be a very easy change to make (up to a point), we can
>>> afford to wait.  It's an approach that has served us well so far —
>>> sometimes focusing on other things means that by the time we have to
>>> look at something, the problem has been solved by somebody else.
>>>
>>> Filesystems are always going to have bugs, so in my opinion the most
>>> important thing is to make having good backups easy, so that recovery is
>>> possible when something goes wrong, regardless of choice of filesystem.
>>> I am very keen for Spectrum to have an integrated backup solution,
>>> ideally as easy to use as Time Machine.
>>
>> To clarify, I'm not referring to bugs that cause data loss, but to bugs
>> that allow kernel code execution when a maliciously crafted filesystem
>> is mounted.  Backups don't protect against this.  This attack is mostly
>> relevant for kiosks, mobile devices, and other cases where being able
>> to restore trust after a device compromise is critical.
> 
> So are you saying that bcachefs's lack of stability means that it's
> uniquely vulnerable to this sort of vulnerability?  I'd be surprised,
> given that as you say Kent is actually interested in preventing them.

bcachefs isn't hardened against these vulnerabilities *yet*, but BTRFS
is probably not hardened at all.  Also, bcachefs will be rewritten in
Rust, hopefully reducing the impact to denial of service (kernel panic).
This post was about BTRFS, not bcachefs, but I should have been clearer
about this.

One area where bcachefs *does* have a problem is its native encryption,
which is vulnerable to catastrophic nonce reuse due to the limited
96-bit nonce size of ChaCha20.  They need to use XChaCha20-Poly1305,
which has a 192-bit nonce.  Also, truncating the authentication tag is
usually a bad idea, though not *always*.  However, this problem with
bcachefs can be fixed.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]