On Wed, Dec 21, 2022 at 01:18:36PM +0200, Ivan Nikolaenko wrote:
> Hello everyone.
>
> Recently I faced an issue with overriding Linux kernel in rootfs recipe:
>
> Rootfs has an attribute kernel in the `let ... in` section, which is used
> for
> setting up correct paths to kernel modules in rootfs derivation. But this
> attribute is also used by the derivation `packagesSysroot` which is used
> by the derivation `packagesTar`, whiсh, in its turn, used by rootfs
> derivation.
>
> All that derivations introduced in the scope of rootfs derivation (the same
> `let ... in` section) and are anonymous.
> What happens when I override kernel attribute of the rootfs derivation:
> 1. Nix evaluates original host/rootfs/default.nix  file and replaces all
> ${kernel} occurences with the actual value of kernel attribute (which is
> linux_latest  at this moment).
> 2. Nix sets linux_latest  as the input for rootfs and packagesSysroot
> derivations.
> 3. Nix evaluates overlay with kernel override and overrides (in fact,
> introduces new) kernel attribute.
> 4. Nothing happens after that.
> Now we have:
> - `kernel` attribute of rootfs derivation pointing to the imx8 kernel.
> - `MODULES_ALIAS` and `MODULES_ORDER` variables of rootfs derivation are
> pointing to linux_latest modules.
> The root of the issue - line `ln -s ${kernel}/lib/modules
> ${firmware}/lib/firmware $out/lib`
> from `packagesSysroot` derivation build command, which creates the /lib link
> in the rootfs
> pointing to linux_latest modules. As I found, kernel attribure is not even
> passed as an attribute to
> the rootfs derivation file. But it is possible to override MODULES_ALIAS and
> MODULES_ORDER with correct kernel modules path instead.
>
> This also touches initramfs
> The possible solution is just to override linux_latest, but in this case all
> VMs will use host kernel version, and we need to avoid this.
>
> I would propose something like this for rootfs' default.nix, because this
> approach solves all kernel issues in an easy and clear way:
>
> diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
> index ad6ea1f..7bf16d2 100644
> --- a/host/rootfs/default.nix
> +++ b/host/rootfs/default.nix
> @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage (
>
>  { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc,
> tar2ext4
>  , busybox, cloud-hypervisor, cryptsetup, execline, jq, kmod
> -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg
> +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, kernel ?
> linux_latest
>  }:
>
>  let
> @@ -73,8 +73,6 @@ let
>      imports = [ (modulesPath + "/profiles/all-hardware.nix") ];
>    });
>
> -  kernel = pkgs.linux_latest;
> -
>    appvm = import ../../img/app {
>      inherit config;
>      inherit (foot) terminfo;
>
> I played a bit with this, and seems that it worked.
>
> Thanks.

Seems reasonable!  Happy to apply if you resend with a Signed-off-by.

Although I think it needs to still be pkgs.linux_latest, as
linux_latest isn't defined.  (We don't want to take it as a callPackage
input because we don't need pkgsStatic.linux_latest.)  And it might be a
good idea to put the "kernel" option on a new line, so that overridable
options are different from package inputs.