From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 601CC2F99F; Wed, 17 Aug 2022 13:25:40 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 496) id 1AA3D2F984; Wed, 17 Aug 2022 13:25:36 +0000 (UTC) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by atuin.qyliss.net (Postfix) with ESMTPS id 2129F2F964 for ; Wed, 17 Aug 2022 13:25:32 +0000 (UTC) Received: by mail-ej1-x62a.google.com with SMTP id gk3so24513375ejb.8 for ; Wed, 17 Aug 2022 06:25:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unikie.com; s=google; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc; bh=7hzguCbD2Oc53j22QIsWrNnTBr/7nUCYresJyx7lUP4=; b=b/Vo/NcOfBDqxa+Nu4G5VdTYRpB45G+5wm8dyXbm+M9M1SRYqJqxxNBaOmgypoHrOX 9JyjKZguqMEaPC7VpNlHrqKnnjZie6aIQNcwGk6pXCPQaoykazVbMTTu6ixGZuau7dKy dM4KdtruT3hbPIT66JAl0MDW1kuEfrDe0UqEcd4lFWo8VhlzPxoaQZCfL6YYlnJ+eCnz eRUAjXNRZkmHaEDbGFaSe39JRIYdR7be+sHc4Cl1a6BVvAJF0g1NY8rPTeTp8nPcjsuG oL+9skJvX0vbLEVE9fDMmO9QhwuqX+zkTZbED7U8vYZDuQML0Vsfi1c5CfPB5tTk5yvj 26IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc; bh=7hzguCbD2Oc53j22QIsWrNnTBr/7nUCYresJyx7lUP4=; b=jBwtyM40mHUnubLCEfj2buaVY3lldBTQY0Dlu2DBRO422Bcu94ZlBZxVrS22HMJNaB SjY/ppMM/qEqsCnL3dkOa+/HbY885nQTjdyevGyLErFN/7n4BwhG6DvZWQtkIZMlMlmc HgVk2YbE9hDR4h0B1VSp6HSGFj86yurI0R9uoqMaIXgURH5uTfA2S1VV0EhQyg+nZVdl 4hsanxNjXe2BfJeX8kWvNlaSroQprBBihAWNF383BAvEi3rSNY2FFdWxwtIuSY80o2uc A2XbQ+A/KZ7Ag/QwLvasZ7sILQRphjV81kXrdpNBCdHWM+jduClR1cSi/29Di+7twTUY n76Q== X-Gm-Message-State: ACgBeo0sGPkf1kdeTRnL6WG1Bv/ep3EAe48/RRD1nd/f1raxb3GPNA61 vyAobZCtMRx5ArZy3A6ZzxjA/nqzvmZ6A60dIXlLh6Rqb138Kw== X-Google-Smtp-Source: AA6agR7oeu/W64iE61F72M8Z2jzxZnbdPGbEs/FC6Mv+wO5hh2s86g4Uhnm6mHmJrmZli1IEZ8y5ZkOBBJD+5QEVQco= X-Received: by 2002:a17:907:a427:b0:732:ea25:2d38 with SMTP id sg39-20020a170907a42700b00732ea252d38mr16531230ejc.87.1660742731446; Wed, 17 Aug 2022 06:25:31 -0700 (PDT) MIME-Version: 1.0 References: <20220817075255.wjw24mzuyyl3lhgz@eve> In-Reply-To: <20220817075255.wjw24mzuyyl3lhgz@eve> From: Ville Ilvonen Date: Wed, 17 Aug 2022 16:25:20 +0300 Message-ID: Subject: Re: HW identification and configuration on Spectrum To: Alyssa Ross Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Message-ID-Hash: 2XR4NCXVO7ZF2ODZPU6I43ENADBETGTM X-Message-ID-Hash: 2XR4NCXVO7ZF2ODZPU6I43ENADBETGTM X-MailFrom: ville.ilvonen@unikie.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-discuss.spectrum-os.org-0; header-match-discuss.spectrum-os.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: discuss@spectrum-os.org X-Mailman-Version: 3.3.5 Precedence: list List-Id: General high-level discussion about Spectrum Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, Aug 17, 2022 at 10:52 AM Alyssa Ross wrote: > > On Tue, Aug 16, 2022 at 06:50:48PM +0300, Ville Ilvonen wrote: > > Hi, > > > > Now that we've been developing Spectrum ARM (aarch64) support > > with iMX8 boards, I'd like to get back to Spectrum HW configuration des= ign. > > > > On x86 the generic image with kernel supporting most devices as modules= can > > make sense. On ARM, the vendor specific BSP HW quirks are more common. > > What impact will Google's Generic Kernel Image[1] efforts have on this? > As I understand it, going forward Android devices won't be allowed to > make arbitrary kernel changes, and will be restricted to adding extra > modules. Which presumably means that SOCs aiming to be used for Android > devices will have to work with the standard Android kernel, which is > hopefully getting closer to mainline over time[2]. ARM goes beyond Google Android ecosystem - including automotive and increasingly servers and other personal devices than smartphones. GKI is definitely the right direction but I'm afraid Google can't pull the BSPs on other sectors in. Which takes us to your point below. Nothing to add there. > Regardless, I understand that there will always be some cases where a > non-upstream kernel is a necessity (even if the Android situation gets > better, there will always be a new MacBook that doesn't have upstream > drivers yet!) so I am keen to figure out how to support that well in > Spectrum. > > [1]: https://source.android.com/docs/core/architecture/kernel/generic-ker= nel-image > [2]: https://lwn.net/Articles/830979/ > > > As of now, the spectrum fork for aarch64 just adds another config > > after rpi configs > > and replaces the default config to use that to build. With small > > changes this could > > be handled like rpi configs. In addition, cloud-hypervisor accepts > > kernel only in > > EFI format for aarch64[1]. Anyway, this would allow us to build an > > aarch64 Spectrum installer > > - even make it with a more generic kernel. That takes us to ARM > > vendor/device specific HW > > quirks which would need to be handled anyway. I'll intentionally leave > > device specific > > kernel hardening and disabling kernel module loading for security > > reasons for now. > > As of now the vendor/device specifics are not supported unless one buil= ds device > > specific Spectrum image with all configs build-time and skips > > installer altogether. > > > > The other option that I see. We discussed earlier nix-hardware and > > device specific modules. > > That would bring nixos configuration.nix and installation supporting > > scripts to Spectrum, > > though. Those could be called from the Spectrum installer but it would > > change the installer > > logic from writing an image to dynamically configuring the device > > during install based on user > > selections. > > I don't think the full NixOS module system, with rebuilds, etc. belongs > in Spectrum. Being able to treat images as immutable makes it easier to > provide various strong security guarantees. But not wanting to This was and still is one important design decision to build on Spectrum. Regardless, it makes development iterations on target HW more challenging than needed. Conceptually we've had discussions on separating concerns between "development system - writable, easily updatable" and "production system - immutable, updated as image". Latter could have more hardening, security policies etc. enabled which makes development more difficult by design. In practice, some developers have remounted the Spectrum file system as writable to make development iterations easier. In many cases, the development must be done on the target HW which brings us back to the n= eed for the "development system" configuration. Update image iteration cycle is too slow. > integrate the full module system doesn't prevent us taking advantage of > nixos-hardware. It's possible to evaluate NixOS modules standalone in > a Spectrum build, in fact we already do that to reuse NixOS's list of > all redistributable firmware packages[3]. We could do a similar thing > to extract the kernel that nixos-hardware configures for a particular > device, something like this: > > inherit (nixos { > configuration =3D [ = ]; > }.config.boot.kernelPackages) kernel; > > And naturally which device that's pulling from should be configurable =E2= =80=94 > we'll want to have a config file somewhere, just not a full NixOS one. This made me propose nix-hardware usage more and think if we could have what I called "development configuration". In essence, nix-hardware is NixOS channel and we could have a custom channel to support development as well (e.g. dev git repo(s)). > In the medium term, I'd like to decouple nixos-hardware's custom kernel > packages from NixOS configurations. But that would require somebody > finding the time to sit down and make the change, and also convince > other nixos-hardware users that it's the way to go. I don't think it > would be a problem though, especially if it meant nixos-hardware getting > more active maintenance, which it's lacking at the moment because it's > not too well advertised and so not enough people are using it. Ideally yes and I hope we could contribute to that effort. However, we need to focus on getting Spectrum running on aarch64 with imx8 for now. For that I'm reading that nixos-hardware approach is preferred. > I am intrigued by the idea of the installer being able to generate > images, though. Using Nix with a substituter configured on the > installer image would mean that it could download a pre-built image if > one exists for that platform, or fall back to generating one if not. > (And if there was a pre-built image, it would even still be able to > properly Secure Boot with a trusted key once we're in a position to do > that.) So I'm definitely keen on exploring that idea, but it might be > something to do a bit down the road since the work to generate > board-specific Spectrum images wouldn't be contingent on it. Agree, but I'd also move this down the road. > [3]: https://spectrum-os.org/git/spectrum/tree/host/rootfs/default.nix?id= =3Db01594b2c089ce2434dacddccf9a285af7334d24#n64 > > > Any thoughts which would be the preferred way? Maybe some other way? > > In the end, HW specifics are needed also on x86 as we saw with NUCs > > and different > > Lenovo laptops in the spring. I'm not convinced one image to rule them > > all is realistic or secure. > > The issues we saw with Lenovo laptops, etc. wouldn't have been solved by > device specific images =E2=80=94 those devices were broken because of bug= s that > hadn't affected any other systems I'd tried, but in the end the fixes > were applicable everywhere. That itself isn't an indication that we need > device-specific images, just more hardware testing. Fair enough, there were those as well. But there were also device specific quirks. There always will be. HW makers sell HW with their constraints. There will be hacks. > But as I said above, I'm open to having an officially blessed > configuration mechanism to make it possible to build custom images. Sounds good. Would you also please share your thoughts on this "development configuration"? Like some nix tooling enabled for "development" and not inc= luded in the immutable "production" image. > > Finally, this is by no means blocking the hardened iMX8 based Spectrum > > development > > but will keep that work in Spectrum fork until there's an agreed path > > to implement this. > > Integrating this sooner and making it more generic would make Spectrum > > more useful > > for a wider audience. > > Makes sense =E2=80=94 although of course, if any of the work that's been = done so > far is not i.MX8-specific, but is instead just generic stuff to make > Spectrum more ARM-friendly or cross-compilable, I'd be happy to look at > those patches already since they'll be relevant regardless of how we do > device-specific stuff. Thanks, I think it's time to start looking for those with generic stuff in mind so that the board bring-up won't diverge too much from the Spectrum mainline. -Ville