From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id D38D24E967; Sun, 5 Jun 2022 15:29:31 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 496) id 953BB4E93D; Sun, 5 Jun 2022 15:29:28 +0000 (UTC) Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by atuin.qyliss.net (Postfix) with ESMTPS id 8194C4E9B2 for ; Sun, 5 Jun 2022 15:29:25 +0000 (UTC) Received: by mail-oi1-x234.google.com with SMTP id y69so2938970oia.7 for ; Sun, 05 Jun 2022 08:29:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=yhsKC4OX4NaJ7YFYsjcrwBaNBTts7Q+H+bTbgrEjKOY=; b=KNmS9Z7i2+HzyDXzMgsHf+5ZFpsq2h0uWaXTGYpPEbw0eDk20pfNGmT6vsKIB6LiOi 6oVNPU/lqlQTwqIpIzb+v4TqX4nFt7xMDkdIBPNhKHgOH5vr8iVUDahr8hsKBjiBspcG NfGLbHrvdb+qzkDDyK+Eidn9BhLfUSwfQ1/8RtZWq0Rs/XWMXG8qH10KI9wfl0HAB+OK LZ2EZu9VT8b+jrMjUOvvNOS9bPyZR4+f823dxJfIeqyY5j9JIgbNIt0s9tx6x8nEl1aV ZhhNk5a6zjhneyRVXrMxNMoQ85YZwiqDzgqWiCtTBmzwf7/y5ErBwt1+HddaQHjcHA2n sVNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yhsKC4OX4NaJ7YFYsjcrwBaNBTts7Q+H+bTbgrEjKOY=; b=6p5g6gWT7XiOiL1NRUrm6R0x+3O8Nvx8bw1hmyuxTJ7m+p44eZjc1VdwaMog49PXo0 W9N38coQ64+wsueJXs1/lkRW3wAWLeJZ889cRZRgbKOWdMjGw2+Y9V6LMBUqGxxnRals eBobeEy7jW4bqJOizQfVvHF8N1rHRHsmfb1i7Zaxp4bEPERXSalj7JSK5Q0QifyEmPeg vlRx8i8YQ9wdaXOdUnat2ROZkzP+gu5k9Lq4+TdppaADBUsqRZv4SNncfvIR6WGY3mPZ kTLN63/6xVRQrJvvSiqbyuA5N039SwYcP1OJmqTSCLg+fVJA6vRdEcZUDlNaKppZxQnr 0zJA== X-Gm-Message-State: AOAM533Ns6kOz8VZo+NPCMyP6YI5cIy8hFFjyIn6Vn1H7GZTYxneXKn4 Xc0w+m/WIKFyauL53DlQ0jSzKl/BdyWkW8+TED+Hcy/15go= X-Google-Smtp-Source: ABdhPJy3LfQGRUWMmW25G0sJa1iLlNztdWKLFbLszDxpX3GHFWteXpFrd7GrqaMBY9GohYbVP/sHD9hL7UuSiSn0o2o= X-Received: by 2002:a05:6808:2002:b0:32e:9379:51a with SMTP id q2-20020a056808200200b0032e9379051amr530734oiw.77.1654442963313; Sun, 05 Jun 2022 08:29:23 -0700 (PDT) MIME-Version: 1.0 References: <20220321160534.w4oifq3cdygrobrm@eve> <20220322111639.pvr5gxz6zmeu5oua@eve> <20220414135737.lvfbrch34kjojn3s@eve> <20220516115527.2a3wnqqgzuo7gt3x@eve> In-Reply-To: From: Thomas Leonard Date: Sun, 5 Jun 2022 16:29:10 +0000 Message-ID: Subject: Re: Using virtio-gpu instead of virtwl To: Alyssa Ross Content-Type: text/plain; charset="UTF-8" Message-ID-Hash: 5DUALOS3EPT7GYIOP77LMIHMD2GL2YW4 X-Message-ID-Hash: 5DUALOS3EPT7GYIOP77LMIHMD2GL2YW4 X-MailFrom: talex5@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-discuss.spectrum-os.org-0; header-match-discuss.spectrum-os.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: discuss@spectrum-os.org X-Mailman-Version: 3.3.5 Precedence: list List-Id: General high-level discussion about Spectrum Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 18 May 2022 at 09:55, Thomas Leonard wrote: > > On Mon, 16 May 2022 at 11:55, Alyssa Ross wrote: [...] > > Do you have a Nix expression somewhere for crosvm with all this stuff > > fixed? I'd like to integrate it into my draft Nixpkgs PR. (If you > > didn't do it with Nix, I can make the changes myself.) > > No, sorry. I've just been hacking on it in nix-shell. I've now collected things together and pushed all my scripts here: https://gitlab.com/talex5/qubes-lite I've got quite a few patches to crosvm now (and NixOS 22.05 brought some new problems), but they're hacks and not suitable for upstreaming. They might be a useful starting point for someone else though. They're in the crosvm subdirectory: fix-gui-jail: Adds some Nix paths to the jail to allow virtio-gpu to start. Otherwise, the mesa libraries can't be loaded. Note that mesa libraries are loaded dynamically by glvnd using dlopen. If that returns NULL to indicate an error, glvnd just ignores the error and doesn't load any drivers, causing crosvm to segfault later on a NULL pointer. That also hit me when I was accidentally compiling crosvm with a different version of glibc. fix-keymaps: Sway/wlroots now sends read-only keymaps, but crosvm wants to map them read/write, which fails. This hack gets crosvm to try a read-only mmap if a read/write one fails. A better solution would be to remember that the resource is a keymap and map it read-only first. fix-suspend: After the host resumes from suspend, the disk driver gets EINTR from uring and stops handling disk requests. My previous trick of setting the locked memory limit to zero no longer works for some reason, so this stops uring in the source. A better fix would be to get crosvm to retry on EINTR. no-gpu-window: When using virtio-gpu, crosvm opens a pointless console window. This patch forces the use of the Stub backend, which hides it. Possibly this still wastes resources though; maybe disabling the display completely would be better. share-as-user: The shared filesystem driver runs in a jail where everything appears to be owned by root, and so this is how it appears in the Linux guest. This patch makes all files appear to be owned by user 1000, and performs all operations as user 0. This allows the regular user in the VM to access shared files. A better fix might be to stop using user namespaces here, but I'm not sure how to make that work. slow-fences: While there are GPU operations outstanding, crosvm wakes up 1000 times a second to check whether they're done, which is very wasteful. This patch makes it wake only once per second, which doesn't seem to have any ill effects. I'm not sure why there are outstanding requests all the time - possibly "wait for a user action" counts as an in-progress request? -- talex5 (GitHub/Twitter) http://roscidus.com/blog/