From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id CF39D26645; Tue, 17 Jun 2025 19:46:23 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 025DC265C0; Tue, 17 Jun 2025 19:46:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DMARC_PASS,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by atuin.qyliss.net (Postfix) with ESMTPS id 6648226641 for ; Tue, 17 Jun 2025 19:46:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750189578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=icFJhskVkKQqZu2iPajxPrgD6zNA+fOLsrBnVy7h8EQ=; b=KolbeqBM045lyF/59JXOINi5gCoVLmCLU51fkXfEnnNbFbl9DWm/OaXX6DS77Lqx+PP+fW jhrcGlN5vFV8Ychc6JrGpWBehquaT/mlpU2112tFyYs96I3A4jIgSXTu5tfk3LlnbzIOHZ lUPQcZyMhHO9qU0I5jYkqNuTnUgjCWI= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-635-QNI29lQ2MXiI9Gtxbrs5xA-1; Tue, 17 Jun 2025 15:46:17 -0400 X-MC-Unique: QNI29lQ2MXiI9Gtxbrs5xA-1 X-Mimecast-MFC-AGG-ID: QNI29lQ2MXiI9Gtxbrs5xA_1750189576 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3a50816cc58so2068965f8f.3 for ; Tue, 17 Jun 2025 12:46:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750189576; x=1750794376; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=icFJhskVkKQqZu2iPajxPrgD6zNA+fOLsrBnVy7h8EQ=; b=kAJI3MEGjy7ePkpexBcwR+OIov6p9CO1yckgxno9Nwdfgy7Id0bU98CliD0hidXXyL /9KKJ1/xR8w3Kzo97KcGtfVB56V9TbGiaxjrPD5VBCkIIgT7m5c2H9Bkz4cDqtC1YNGj IKeNUH7E5vx53cKpBwMPWT1MjmFwPQStMEvN9yDahH/eNwR6Y3ZszYTHKtiSVYo7cnXo AwcSHURbmVXg4YlWmpuEYLyVtt0avRuoUWAz3McGDjCiJ2SycHfNEpV/8sR4RYGVuoYy kO7TKiITgexnzcg9+j3g463pryiZviGM01mQ1QpVDL5r7QEjMBsj+7aKjsI9r0lbM6vV htQg== X-Forwarded-Encrypted: i=1; AJvYcCWTIpfsIibKgPZfda75fg5LoUm08E0paoxXkfJVgyzM0BuuQw7dkNU6uL8/IRLYRFpEUmupsw==@spectrum-os.org X-Gm-Message-State: AOJu0Yww8pylP+BDplPtfnRIGA/vs3FlO8FpSQaLfIdA7oCtbX3J+KTW C9jGZden3kanLu7zIXSk0F5r8HaTwM+qLnX/o1r5D8muS/zdF0ki7XrFfbn82+R5Nj0SbeJtSuU OquQz92e9Ib5POUEwNLAbI+Hr9uznwJV5kwj8G0to6tneEVIWBVdeYUz8 X-Gm-Gg: ASbGncuw0RaJour6DjbqqCWDzN1Mhmz9yDXNrRPQ7QibqE6iomjSitcdkv0pgHqCAcO 1Uubu1BwKPkjMk1eFk6M9D7nblqi+7XrjeVyl5ZfD/T9Ea+EJChLp+Swm2erUb63d4JR9g/WguT gGo5oOZh2UEoWuq06njkIct4tBiBvzan/qXXDwC6tiqaO6nnHHJ11ZuUha/Mt9JQchzqyzZwX3I TQ3XVMhNm0vSvQvyz5Ru6Pv8o3WS8d+D6U3B2rRNsC9XPeHc5fBpsLSYAdylrDIhQ7RB3WjaLvV JqRa1yX2xUCRk7q7 X-Received: by 2002:a05:6000:2007:b0:3a5:25e0:1851 with SMTP id ffacd0b85a97d-3a5723660b5mr12195273f8f.7.1750189576210; Tue, 17 Jun 2025 12:46:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFO9JZf66g6UmsF+FsTytz4QymacwGU4iFb1DNv63Dd/5GxeH4PDvqwykdFRQ9tUf5lrXqgjg== X-Received: by 2002:a05:6000:2007:b0:3a5:25e0:1851 with SMTP id ffacd0b85a97d-3a5723660b5mr12195258f8f.7.1750189575837; Tue, 17 Jun 2025 12:46:15 -0700 (PDT) Received: from redhat.com ([2a06:c701:73ea:4300:f7cc:3f8:48e8:2142]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a568a543d9sm14603319f8f.5.2025.06.17.12.46.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jun 2025 12:46:15 -0700 (PDT) Date: Tue, 17 Jun 2025 15:46:12 -0400 From: "Michael S. Tsirkin" To: Demi Marie Obenour Subject: Re: Virtio-IOMMU interrupt remapping design Message-ID: <20250617154524-mutt-send-email-mst@kernel.org> References: <> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 4Vxj8nMfDZqkcE-yfIiE0PugnghBS-M5CstYlhBa7ME_1750189576 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Message-ID-Hash: C3NXOKYIC2EPHRCB3KGRTSL7BRNDO3E3 X-Message-ID-Hash: C3NXOKYIC2EPHRCB3KGRTSL7BRNDO3E3 X-MailFrom: mst@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Jason Wang , Xuan Zhuo , Eugenio =?iso-8859-1?Q?P=E9rez?= , "Rafael J. Wysocki" , Len Brown , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Joerg Roedel , Will Deacon , Robin Murphy , Alyssa Ross , virtualization@lists.linux.dev, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, iommu@lists.linux.dev, x86@kernel.org, Spectrum OS Development X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Sun, Jun 15, 2025 at 02:47:15PM -0400, Demi Marie Obenour wrote: > Virtio-IOMMU interrupt remapping turned out to be much harder than I > realized. The main problem is that interrupt remapping is set up > very early in boot. In fact, Linux calls the interrupt remapping probe > function from the APIC initialization code: x86_64_probe_apic -> > enable_IR_x2apic -> irq_remapping_prepare(). This is almost certainly > much before PCI has been initialized. Also, the order in which devices > will be initialized is not something Linux guarantees at all, which is a > problem because interrupt remapping must be initialized before drivers > start setting up interrupts. Otherwise, the interrupt remapping table > won't include entries for already-existing interrupts, and things will > either break badly, not get the benefit of interrupt remapping > security-wise, or both. > > The reason I expect this doesn't cause problems for address translation > is that the IOMMU probably starts in bypass mode by default, meaning > that all DMA is permitted. If the IOMMU is only used by VFIO or > IOMMUFD, it will not be needed until userspace starts up, which is after > the IOMMU has been initialized. This isn't ideal, though, as it means > that kernel drivers operate without DMA protection. > > Is a paravirtualized IOMMU with interrupt remapping something that makes > sense? Absolutely! However, the IOMMU should be considered a platform > device that must be initialized very early in boot. Using virtio-IOMMU > with MMIO transport as the interface might be a reasonable option, but > the IOMMU needs to be enumerated via ACPI, device tree, or kernel > command line argument. This allows it to be brought up before anything > capable of DMA is initialized. > > Is this the right path to go down? What do others think about this? > -- > Sincerely, > Demi Marie Obenour (she/her/hers) The project for this discussion is also virtio-comment, this ML is for driver work.