[PATCH] tools/xdp-forwarder: Fix build

[PATCH] tools/xdp-forwarder: Fix build

[Demi Marie Obenour]

Clang complains that it can't fine gnu-stubs32.h.  Drop the ICMP parsing
code to avoid this error.  No functional change as the code is not
actually used.

Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
 tools/xdp-forwarder/parsing_helpers.h | 57 -----------------------------------
 1 file changed, 57 deletions(-)

diff --git a/tools/xdp-forwarder/parsing_helpers.h b/tools/xdp-forwarder/parsing_helpers.h
index da099346008bd58485af8308feb4d3391ceef8f5..1ea822100fdb9a75c2d28d34d93e6bb2b5d3ae26 100644
--- a/tools/xdp-forwarder/parsing_helpers.h
+++ b/tools/xdp-forwarder/parsing_helpers.h
@@ -26,8 +26,6 @@
 #include <linux/if_packet.h>
 #include <linux/ip.h>
 #include <linux/ipv6.h>
-#include <linux/icmp.h>
-#include <linux/icmpv6.h>
 #include <linux/udp.h>
 #include <linux/tcp.h>
 
@@ -46,16 +44,6 @@ struct vlan_hdr {
 	__be16 h_vlan_encapsulated_proto;
 };
 
-/*
- * Struct icmphdr_common represents the common part of the icmphdr and icmp6hdr
- * structures.
- */
-struct icmphdr_common {
-	__u8 type;
-	__u8 code;
-	__sum16 cksum;
-};
-
 /* Allow users of header file to redefine VLAN max depth */
 #ifndef VLAN_MAX_DEPTH
 #define VLAN_MAX_DEPTH 2
@@ -175,51 +163,6 @@ static __always_inline int parse_iphdr(struct hdr_cursor *nh,
 	return iph->protocol;
 }
 
-static __always_inline int parse_icmp6hdr(struct hdr_cursor *nh,
-                                          void *data_end,
-                                          struct icmp6hdr **icmp6hdr)
-{
-	struct icmp6hdr *icmp6h = nh->pos;
-
-	if (icmp6h + 1 > data_end)
-		return -1;
-
-	nh->pos   = icmp6h + 1;
-	*icmp6hdr = icmp6h;
-
-	return icmp6h->icmp6_type;
-}
-
-static __always_inline int parse_icmphdr(struct hdr_cursor *nh,
-                                         void *data_end,
-                                         struct icmphdr **icmphdr)
-{
-	struct icmphdr *icmph = nh->pos;
-
-	if (icmph + 1 > data_end)
-		return -1;
-
-	nh->pos  = icmph + 1;
-	*icmphdr = icmph;
-
-	return icmph->type;
-}
-
-static __always_inline int parse_icmphdr_common(struct hdr_cursor *nh,
-                                                void *data_end,
-                                                struct icmphdr_common **icmphdr)
-{
-	struct icmphdr_common *h = nh->pos;
-
-	if (h + 1 > data_end)
-		return -1;
-
-	nh->pos  = h + 1;
-	*icmphdr = h;
-
-	return h->type;
-}
-
 /*
  * parse_udphdr: parse the udp header and return the length of the udp payload
  */

---
base-commit: c5d5786d3dc938af0b279c542d1e43bce381b4b9
change-id: 20251003-fix-forwarder-build-2889ea9aec91
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

Re: [PATCH] tools/xdp-forwarder: Fix build

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]

Demi Marie Obenour <demiobenour@gmail.com> writes:

> Clang complains that it can't fine gnu-stubs32.h.  Drop the ICMP parsing
> code to avoid this error.  No functional change as the code is not
> actually used.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
>  tools/xdp-forwarder/parsing_helpers.h | 57 -----------------------------------
>  1 file changed, 57 deletions(-)

What did you do that produced this error?  We're supposed to have a
reproducible environment, so something is going wrong if you're getting
errors others don't.

As previously discussed, we don't want to modify these headers, because
then it'll get harder to sync them with upstream.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] tools/xdp-forwarder: Fix build

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 1064 bytes --]

On 10/4/25 11:47, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
> 
>> Clang complains that it can't fine gnu-stubs32.h.  Drop the ICMP parsing
>> code to avoid this error.  No functional change as the code is not
>> actually used.
>>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>>  tools/xdp-forwarder/parsing_helpers.h | 57 -----------------------------------
>>  1 file changed, 57 deletions(-)
> 
> What did you do that produced this error?  We're supposed to have a
> reproducible environment, so something is going wrong if you're getting
> errors others don't.
> 
> As previously discussed, we don't want to modify these headers, because
> then it'll get harder to sync them with upstream.

For those not on the Matrix discussion, the problem turned out to be that
the BPF code included libc headers for the target CPU architecture.  When
building for arm64 this happened to work, but when building for x86_64 it
reproducibly failed. 
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH RFC DOESNOTBUILD v2] tools/xdp-forwarder: Do not include libc headers in eBPF programs

[Demi Marie Obenour]

The build happened to work on arm64 because the glibc arm64 headers
don't support multilib.  On x86_64, glibc headers assume that BPF is a
32-bit platform (because __x86_64__ isn't defined) and fail to find the
32-bit headers.  This is not a glibc bug.  Rather, BPF programs should
not be including glibc headers.

Most Linux headers are not trivial to include in BPF programs.  The
version of the headers meant for userspace use do include glibc headers,
and that isn't supported in BPF.  The version meant for building
kernel modules does not, but using it requires much more complicated
build system.

Solve this problem by only including headers intended for use in BPF
programs.  These headers include declarations explicitly intended for
use in BPF programs, so if they do pull in libc headers that is a bug.
This also allows using an unwrapped clang.  Nix's wrapped clang is not
suitable when a target is explicitly specified, so this is another a
bug fix.

This prevents vendoring the example headers, so replace them with a
header that only includes common functionality common to both eBPF
programs.  A single struct including both Ethernet II and 802.1Q headers
is used, massively simplifying the code.

Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v2:
- EDITME: describe what is new in this series revision.
- EDITME: use bulletpoints and terse descriptions.
- Link to v1: https://lore.kernel.org/r/20251003-fix-forwarder-build-v1-1-856b78ae5656@gmail.com
---
 tools/default.nix                     |   7 +-
 tools/meson.options                   |   3 +
 tools/xdp-forwarder/helpers.h         |  44 ++++++
 tools/xdp-forwarder/meson.build       |   9 +-
 tools/xdp-forwarder/parsing_helpers.h | 274 ----------------------------------
 tools/xdp-forwarder/prog_physical.c   |  43 +++---
 tools/xdp-forwarder/prog_router.c     |  44 ++++--
 tools/xdp-forwarder/rewrite_helpers.h | 146 ------------------
 8 files changed, 109 insertions(+), 461 deletions(-)

diff --git a/tools/default.nix b/tools/default.nix
index 2c6846c80073e7b64fb7a19488103f6cf97a4420..b290cdbee046e362233ef8a0ecf4aa589c194cbb 100644
--- a/tools/default.nix
+++ b/tools/default.nix
@@ -88,7 +88,7 @@ stdenv.mkDerivation (finalAttrs: {
     ++ lib.optionals (appSupport || driverSupport) [ pkg-config ]
     ++ lib.optionals hostSupport [ rustc ]
     ++ lib.optionals driverSupport [ clang_21 ];
-  buildInputs = lib.optionals appSupport [ dbus ] ++ lib.optionals driverSupport [ libbpf ];
+  buildInputs = lib.optionals appSupport [ dbus ] ++ lib.optionals driverSupport [ libbpf linuxHeaders ];
 
   postPatch = lib.optionals hostSupport (lib.concatMapStringsSep "\n" (crate: ''
     mkdir -p subprojects/packagecache
@@ -104,10 +104,7 @@ stdenv.mkDerivation (finalAttrs: {
     "-Dtests=false"
     "-Dunwind=false"
     "-Dwerror=true"
-  ];
-
-  # Not supported for target bpf
-  hardeningDisable = lib.optionals driverSupport [ "zerocallusedregs" ];
+  ] ++ lib.optionals driverSupport [ "-Dclang=${clang_21.cc}/bin/clang" ];
 
   passthru.tests = {
     clang-tidy = finalAttrs.finalPackage.overrideAttrs (
diff --git a/tools/meson.options b/tools/meson.options
index 301efb9f677fdec57c8491fd6a6868f2d35cb076..a804e34811bbedc7336e3d6bb14932c95a353a85 100644
--- a/tools/meson.options
+++ b/tools/meson.options
@@ -13,6 +13,9 @@ option('driver', type : 'boolean', value : false,
 option('hostfsrootdir', type : 'string', value : '/run/host',
   description : 'Path where the virtio-fs provided by the host will be mounted')
 
+option('clang', type : 'string',
+  description : 'Path to clang')
+
 option('tests',
   type : 'boolean',
   description : 'Build tests')
diff --git a/tools/xdp-forwarder/helpers.h b/tools/xdp-forwarder/helpers.h
new file mode 100644
index 0000000000000000000000000000000000000000..2583655a13518ec90de7b0256b2153c9adbdb3bf
--- /dev/null
+++ b/tools/xdp-forwarder/helpers.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+/* SPDX-FileCopyrightText: 2021 The xdp-tutorial Authors */
+// SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+// SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
+/*
+ * This file contains parsing functions that are used in the XDP programs.
+ * They handle the following:
+ *
+ * - Validating VLAN tags.
+ * - Extracting Ethernet and VLAN headers.
+ * - Moving the head of an XDP context by the size of a VLAN header.
+ * - Moving the Ethernet source and destination MAC addresses by the
+ *   size of a VLAN header.
+ */
+
+#ifndef HELPERS_H
+#define HELPERS_H
+
+#include <linux/bpf.h>
+#include <bpf/bpf_endian.h>
+#include <bpf/bpf_helpers.h>
+
+#define VLAN_HDR_SIZE 4
+#define MAC_ADDRESS_COMBINED_SIZE 12
+#define VLAN_VID_MASK           0x0fff /* VLAN Identifier */
+#define VLAN_ETHTYPE (bpf_htons(0x8100))
+
+struct tagged_ethhdr {
+	__u8 mac_addresses[MAC_ADDRESS_COMBINED_SIZE];
+	__be16 h_proto;
+	__be16 h_vlan_TCI;
+	__be16 h_vlan_encapsulated_proto;
+};
+
+// The router doesn't support the PCP and DEI bits
+// and they are not part of the VLAN tag.
+// Therefore, ensure they are unset.
+// Also reject VLAN 0, which is reserved.
+static __always_inline bool vlan_tag_is_valid(__u32 tag)
+{
+	return tag >= 1 && tag <= 0x0FFF;
+}
+
+#endif /* HELPERS_H */
diff --git a/tools/xdp-forwarder/meson.build b/tools/xdp-forwarder/meson.build
index b73130eb27b8000a102b0a8847ecb06b93a955d2..24ba95f4d4f6354d25cae847a6e5ff077c94d03d 100644
--- a/tools/xdp-forwarder/meson.build
+++ b/tools/xdp-forwarder/meson.build
@@ -9,10 +9,13 @@ executable('set-router-iface', 'set_router_iface.c',
   dependencies : libbpf,
   install : true)
 
-clang = find_program('clang', native : true)
+clang_path = get_option('clang')
+if clang_path == ''
+  error('clang must be provided to build XDP forwarder')
+endif
 
 bpf_o_cmd = [
-  clang.full_path(),
+  clang_path,
   '-fno-stack-protector',
   '-fno-strict-aliasing',
   '-fwrapv', '-fwrapv-pointer',
@@ -22,9 +25,9 @@ bpf_o_cmd = [
   '-Wno-sign-compare',
   '-O2',
   '-target', 'bpf',
-  '-I', meson.current_source_dir() + '/include',
   '-g',
   '-c',
+  '-std=gnu23',
   '-o', '@OUTPUT@',
   '-MD',
   '-MQ', '@OUTPUT',
diff --git a/tools/xdp-forwarder/parsing_helpers.h b/tools/xdp-forwarder/parsing_helpers.h
deleted file mode 100644
index da099346008bd58485af8308feb4d3391ceef8f5..0000000000000000000000000000000000000000
--- a/tools/xdp-forwarder/parsing_helpers.h
+++ /dev/null
@@ -1,274 +0,0 @@
-/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
-/* SPDX-FileCopyrightText: 2021 The xdp-tutorial Authors */
-/* Vendored from https://github.com/xdp-project/xdp-tutorial/blob/d3d3eed6ea9a63d1302bfa8b5a8e93862bfe11f0/common/parsing_helpers.h */
-/*
- * This file contains parsing functions that are used in the packetXX XDP
- * programs. The functions are marked as __always_inline, and fully defined in
- * this header file to be included in the BPF program.
- *
- * Each helper parses a packet header, including doing bounds checking, and
- * returns the type of its contents if successful, and -1 otherwise.
- *
- * For Ethernet and IP headers, the content type is the type of the payload
- * (h_proto for Ethernet, nexthdr for IPv6), for ICMP it is the ICMP type field.
- * All return values are in host byte order.
- *
- * The versions of the functions included here are slightly expanded versions of
- * the functions in the packet01 lesson. For instance, the Ethernet header
- * parsing has support for parsing VLAN tags.
- */
-
-#ifndef __PARSING_HELPERS_H
-#define __PARSING_HELPERS_H
-
-#include <stddef.h>
-#include <linux/if_ether.h>
-#include <linux/if_packet.h>
-#include <linux/ip.h>
-#include <linux/ipv6.h>
-#include <linux/icmp.h>
-#include <linux/icmpv6.h>
-#include <linux/udp.h>
-#include <linux/tcp.h>
-
-/* Header cursor to keep track of current parsing position */
-struct hdr_cursor {
-	void *pos;
-};
-
-/*
- *	struct vlan_hdr - vlan header
- *	@h_vlan_TCI: priority and VLAN ID
- *	@h_vlan_encapsulated_proto: packet type ID or len
- */
-struct vlan_hdr {
-	__be16 h_vlan_TCI;
-	__be16 h_vlan_encapsulated_proto;
-};
-
-/*
- * Struct icmphdr_common represents the common part of the icmphdr and icmp6hdr
- * structures.
- */
-struct icmphdr_common {
-	__u8 type;
-	__u8 code;
-	__sum16 cksum;
-};
-
-/* Allow users of header file to redefine VLAN max depth */
-#ifndef VLAN_MAX_DEPTH
-#define VLAN_MAX_DEPTH 2
-#endif
-
-#define VLAN_VID_MASK           0x0fff /* VLAN Identifier */
-/* Struct for collecting VLANs after parsing via parse_ethhdr_vlan */
-struct collect_vlans {
-	__u16 id[VLAN_MAX_DEPTH];
-};
-
-static __always_inline int proto_is_vlan(__u16 h_proto)
-{
-	return !!(h_proto == bpf_htons(ETH_P_8021Q) ||
-	          h_proto == bpf_htons(ETH_P_8021AD));
-}
-
-/* Notice, parse_ethhdr() will skip VLAN tags, by advancing nh->pos and returns
- * next header EtherType, BUT the ethhdr pointer supplied still points to the
- * Ethernet header. Thus, caller can look at eth->h_proto to see if this was a
- * VLAN tagged packet.
- */
-static __always_inline int parse_ethhdr_vlan(struct hdr_cursor *nh,
-                                             void *data_end,
-                                             struct ethhdr **ethhdr,
-                                             struct collect_vlans *vlans)
-{
-	struct ethhdr *eth = nh->pos;
-	int hdrsize = sizeof(*eth);
-	struct vlan_hdr *vlh;
-	__u16 h_proto;
-	int i;
-
-	/* Byte-count bounds check; check if current pointer + size of header
-	 * is after data_end.
-	 */
-	if (nh->pos + hdrsize > data_end)
-		return -1;
-
-	nh->pos += hdrsize;
-	*ethhdr = eth;
-	vlh = nh->pos;
-	h_proto = eth->h_proto;
-
-	/* Use loop unrolling to avoid the verifier restriction on loops;
-	 * support up to VLAN_MAX_DEPTH layers of VLAN encapsulation.
-	 */
-	#pragma unroll
-	for (i = 0; i < VLAN_MAX_DEPTH; i++) {
-		if (!proto_is_vlan(h_proto))
-			break;
-
-		if (vlh + 1 > data_end)
-			break;
-
-		h_proto = vlh->h_vlan_encapsulated_proto;
-		if (vlans) /* collect VLAN ids */
-			vlans->id[i] =
-				(bpf_ntohs(vlh->h_vlan_TCI) & VLAN_VID_MASK);
-
-		vlh++;
-	}
-
-	nh->pos = vlh;
-	return h_proto; /* network-byte-order */
-}
-
-static __always_inline int parse_ethhdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct ethhdr **ethhdr)
-{
-	/* Expect compiler removes the code that collects VLAN ids */
-	return parse_ethhdr_vlan(nh, data_end, ethhdr, NULL);
-}
-
-static __always_inline int parse_ip6hdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct ipv6hdr **ip6hdr)
-{
-	struct ipv6hdr *ip6h = nh->pos;
-
-	/* Pointer-arithmetic bounds check; pointer +1 points to after end of
-	 * thing being pointed to. We will be using this style in the remainder
-	 * of the tutorial.
-	 */
-	if (ip6h + 1 > data_end)
-		return -1;
-
-	nh->pos = ip6h + 1;
-	*ip6hdr = ip6h;
-
-	return ip6h->nexthdr;
-}
-
-static __always_inline int parse_iphdr(struct hdr_cursor *nh,
-                                       void *data_end,
-                                       struct iphdr **iphdr)
-{
-	struct iphdr *iph = nh->pos;
-	int hdrsize;
-
-	if (iph + 1 > data_end)
-		return -1;
-
-	hdrsize = iph->ihl * 4;
-	/* Sanity check packet field is valid */
-	if(hdrsize < sizeof(*iph))
-		return -1;
-
-	/* Variable-length IPv4 header, need to use byte-based arithmetic */
-	if (nh->pos + hdrsize > data_end)
-		return -1;
-
-	nh->pos += hdrsize;
-	*iphdr = iph;
-
-	return iph->protocol;
-}
-
-static __always_inline int parse_icmp6hdr(struct hdr_cursor *nh,
-                                          void *data_end,
-                                          struct icmp6hdr **icmp6hdr)
-{
-	struct icmp6hdr *icmp6h = nh->pos;
-
-	if (icmp6h + 1 > data_end)
-		return -1;
-
-	nh->pos   = icmp6h + 1;
-	*icmp6hdr = icmp6h;
-
-	return icmp6h->icmp6_type;
-}
-
-static __always_inline int parse_icmphdr(struct hdr_cursor *nh,
-                                         void *data_end,
-                                         struct icmphdr **icmphdr)
-{
-	struct icmphdr *icmph = nh->pos;
-
-	if (icmph + 1 > data_end)
-		return -1;
-
-	nh->pos  = icmph + 1;
-	*icmphdr = icmph;
-
-	return icmph->type;
-}
-
-static __always_inline int parse_icmphdr_common(struct hdr_cursor *nh,
-                                                void *data_end,
-                                                struct icmphdr_common **icmphdr)
-{
-	struct icmphdr_common *h = nh->pos;
-
-	if (h + 1 > data_end)
-		return -1;
-
-	nh->pos  = h + 1;
-	*icmphdr = h;
-
-	return h->type;
-}
-
-/*
- * parse_udphdr: parse the udp header and return the length of the udp payload
- */
-static __always_inline int parse_udphdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct udphdr **udphdr)
-{
-	int len;
-	struct udphdr *h = nh->pos;
-
-	if (h + 1 > data_end)
-		return -1;
-
-	nh->pos  = h + 1;
-	*udphdr = h;
-
-	len = bpf_ntohs(h->len) - sizeof(struct udphdr);
-	if (len < 0)
-		return -1;
-
-	return len;
-}
-
-/*
- * parse_tcphdr: parse and return the length of the tcp header
- */
-static __always_inline int parse_tcphdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct tcphdr **tcphdr)
-{
-	int len;
-	struct tcphdr *h = nh->pos;
-
-	if (h + 1 > data_end)
-		return -1;
-
-	len = h->doff * 4;
-	/* Sanity check packet field is valid */
-	if(len < sizeof(*h))
-		return -1;
-
-	/* Variable-length TCP header, need to use byte-based arithmetic */
-	if (nh->pos + len > data_end)
-		return -1;
-
-	nh->pos += len;
-	*tcphdr = h;
-
-	return len;
-}
-
-#endif /* __PARSING_HELPERS_H */
diff --git a/tools/xdp-forwarder/prog_physical.c b/tools/xdp-forwarder/prog_physical.c
index 2b70654ebe21cb2504e6d26ca3b93cb4b62efb0f..c0d459d0bbf3d206f3bd678d0d0e691db46cd2db 100644
--- a/tools/xdp-forwarder/prog_physical.c
+++ b/tools/xdp-forwarder/prog_physical.c
@@ -1,12 +1,8 @@
 // SPDX-License-Identifier: EUPL-1.2+
 // SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+// SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
 
-#define VLAN_MAX_DEPTH 1
-
-#include <linux/bpf.h>
-#include <bpf/bpf_endian.h>
-#include "parsing_helpers.h"
-#include "rewrite_helpers.h"
+#include "helpers.h"
 
 struct {
 	__uint(type, BPF_MAP_TYPE_DEVMAP);
@@ -16,24 +12,37 @@ struct {
 	__uint(pinning, LIBBPF_PIN_BY_NAME);
 } router_iface SEC(".maps");
 
+static __always_inline bool push_vlan_tag(struct xdp_md *ctx, __u16 tag)
+{
+	// Add extra space at the front of the packet.
+	// This avoids reloading pointers or extra bounds checks later.
+	if (bpf_xdp_adjust_head(ctx, -VLAN_HDR_SIZE))
+		return false;
+
+	struct tagged_ethhdr *hdr = (void *)(long)ctx->data;
+	if (hdr + 1 > (void *)(long)ctx->data_end)
+		return false;
+
+	// Move the MAC addresses.
+	__builtin_memmove(hdr, (char *)hdr + VLAN_HDR_SIZE, MAC_ADDRESS_COMBINED_SIZE);
+
+	// Set the VLAN ID and the Ethertype of the frame.
+	hdr->h_vlan_TCI = bpf_htons((__u16)tag);
+	hdr->h_proto = VLAN_ETHTYPE;
+	return true;
+}
+
 SEC("xdp")
 int physical(struct xdp_md *ctx)
 {
-	void *data_end = (void *)(long)ctx->data_end;
-	void *data = (void *)(long)ctx->data;
-
-	struct hdr_cursor nh;
-	nh.pos = data;
-
-	struct ethhdr *eth;
-	if (parse_ethhdr(&nh, data_end, &eth) < 0)
-		return XDP_DROP;
+	__u32 ingress_ifindex = ctx->ingress_ifindex;
 
-	if (ctx->ingress_ifindex < 1 || ctx->ingress_ifindex > VLAN_VID_MASK)
+	if (!vlan_tag_is_valid(ingress_ifindex))
 		return XDP_DROP;
 
-	if (vlan_tag_push(ctx, eth, ctx->ingress_ifindex) < 0)
+	if (!push_vlan_tag(ctx, (__u16)ingress_ifindex))
 		return XDP_DROP;
 
+	// Redirect to the router interface.
 	return bpf_redirect_map(&router_iface, 0, 0);
 }
diff --git a/tools/xdp-forwarder/prog_router.c b/tools/xdp-forwarder/prog_router.c
index 5b0c8fa5c55e365c08fdb1be8f0613bdc67d8649..eb3b277bb035c90c7e23dbdb672404f0d7c0ebfe 100644
--- a/tools/xdp-forwarder/prog_router.c
+++ b/tools/xdp-forwarder/prog_router.c
@@ -1,12 +1,8 @@
 // SPDX-License-Identifier: EUPL-1.2+
 // SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+// SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
 
-#define VLAN_MAX_DEPTH 1
-
-#include <linux/bpf.h>
-#include <bpf/bpf_endian.h>
-#include "parsing_helpers.h"
-#include "rewrite_helpers.h"
+#include "helpers.h"
 
 // The map is actually not used by this program, but just included
 // to keep the reference-counted pin alive before any physical interfaces
@@ -19,23 +15,39 @@ struct {
 	__uint(pinning, LIBBPF_PIN_BY_NAME);
 } router_iface SEC(".maps");
 
+static bool pop_vlan_tag(struct xdp_md *ctx, __u16 *tag)
+{
+	struct tagged_ethhdr *hdr = (void *)(long)ctx->data;
+	if (hdr + 1 > (void *)(long)ctx->data_end)
+		return false;
+
+	// 0x8A88 is also valid but the router does not
+	// use it.  It's meant for service provider VLANs.
+	if (hdr->h_proto != VLAN_ETHTYPE)
+		return false;
+
+	*tag = bpf_ntohs(hdr->h_vlan_TCI);
+
+	// Move the MAC addresses.
+	__builtin_memmove((char *)hdr + VLAN_HDR_SIZE, hdr, MAC_ADDRESS_COMBINED_SIZE);
+
+	// Move the head pointer to the new Ethernet header.
+	// Doing this last avoids needing to reload pointers
+	// or to add extra bounds checks earlier.
+	return !bpf_xdp_adjust_head(ctx, VLAN_HDR_SIZE);
+}
 
 SEC("xdp")
 int router(struct xdp_md *ctx)
 {
-	void *data_end = (void *)(long)ctx->data_end;
-	void *data = (void *)(long)ctx->data;
+	__u16 vlid;
 
-	struct hdr_cursor nh;
-	nh.pos = data;
-
-	struct ethhdr *eth;
-	if (parse_ethhdr(&nh, data_end, &eth) < 0)
-		return XDP_DROP;
+	if (!pop_vlan_tag(ctx, &vlid))
+		return false;
 
-	int vlid = vlan_tag_pop(ctx, eth);
-	if (vlid < 0)
+	if (!vlan_tag_is_valid(vlid))
 		return XDP_DROP;
 
+	// Redirect to the correct physical interface.
 	return bpf_redirect(vlid, 0);
 }
diff --git a/tools/xdp-forwarder/rewrite_helpers.h b/tools/xdp-forwarder/rewrite_helpers.h
deleted file mode 100644
index 71aa23e038cc450087c1db20c00cbdbe1616b0a1..0000000000000000000000000000000000000000
--- a/tools/xdp-forwarder/rewrite_helpers.h
+++ /dev/null
@@ -1,146 +0,0 @@
-/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
-/* SPDX-FileCopyrightText: 2019 The xdp-tutorial Authors */
-/* Vendored from https://github.com/xdp-project/xdp-tutorial/blob/d3d3eed6ea9a63d1302bfa8b5a8e93862bfe11f0/common/rewrite_helpers.h */
-/*
- * This file contains functions that are used in the packetXX XDP programs to
- * manipulate on packets data. The functions are marked as __always_inline, and
- * fully defined in this header file to be included in the BPF program.
- */
-
-#ifndef __REWRITE_HELPERS_H
-#define __REWRITE_HELPERS_H
-
-#include <linux/bpf.h>
-#include <linux/ip.h>
-#include <linux/ipv6.h>
-#include <linux/if_ether.h>
-
-#include <bpf/bpf_helpers.h>
-#include <bpf/bpf_endian.h>
-
-/* Pops the outermost VLAN tag off the packet. Returns the popped VLAN ID on
- * success or negative errno on failure.
- */
-static __always_inline int vlan_tag_pop(struct xdp_md *ctx, struct ethhdr *eth)
-{
-	void *data_end = (void *)(long)ctx->data_end;
-	struct ethhdr eth_cpy;
-	struct vlan_hdr *vlh;
-	__be16 h_proto;
-	int vlid;
-
-	if (!proto_is_vlan(eth->h_proto))
-		return -1;
-
-	/* Careful with the parenthesis here */
-	vlh = (void *)(eth + 1);
-
-	/* Still need to do bounds checking */
-	if (vlh + 1 > data_end)
-		return -1;
-
-	/* Save vlan ID for returning, h_proto for updating Ethernet header */
-	vlid = bpf_ntohs(vlh->h_vlan_TCI);
-	h_proto = vlh->h_vlan_encapsulated_proto;
-
-	/* Make a copy of the outer Ethernet header before we cut it off */
-	__builtin_memcpy(&eth_cpy, eth, sizeof(eth_cpy));
-
-	/* Actually adjust the head pointer */
-	if (bpf_xdp_adjust_head(ctx, (int)sizeof(*vlh)))
-		return -1;
-
-	/* Need to re-evaluate data *and* data_end and do new bounds checking
-	 * after adjusting head
-	 */
-	eth = (void *)(long)ctx->data;
-	data_end = (void *)(long)ctx->data_end;
-	if (eth + 1 > data_end)
-		return -1;
-
-	/* Copy back the old Ethernet header and update the proto type */
-	__builtin_memcpy(eth, &eth_cpy, sizeof(*eth));
-	eth->h_proto = h_proto;
-
-	return vlid;
-}
-
-/* Pushes a new VLAN tag after the Ethernet header. Returns 0 on success,
- * -1 on failure.
- */
-static __always_inline int vlan_tag_push(struct xdp_md *ctx,
-                                         struct ethhdr *eth, int vlid)
-{
-	void *data_end = (void *)(long)ctx->data_end;
-	struct ethhdr eth_cpy;
-	struct vlan_hdr *vlh;
-
-	/* First copy the original Ethernet header */
-	__builtin_memcpy(&eth_cpy, eth, sizeof(eth_cpy));
-
-	/* Then add space in front of the packet */
-	if (bpf_xdp_adjust_head(ctx, 0 - (int)sizeof(*vlh)))
-		return -1;
-
-	/* Need to re-evaluate data_end and data after head adjustment, and
-	 * bounds check, even though we know there is enough space (as we
-	 * increased it).
-	 */
-	data_end = (void *)(long)ctx->data_end;
-	eth = (void *)(long)ctx->data;
-
-	if (eth + 1 > data_end)
-		return -1;
-
-	/* Copy back Ethernet header in the right place, populate VLAN tag with
-	 * ID and proto, and set outer Ethernet header to VLAN type.
-	 */
-	__builtin_memcpy(eth, &eth_cpy, sizeof(*eth));
-
-	vlh = (void *)(eth + 1);
-
-	if (vlh + 1 > data_end)
-		return -1;
-
-	vlh->h_vlan_TCI = bpf_htons(vlid);
-	vlh->h_vlan_encapsulated_proto = eth->h_proto;
-
-	eth->h_proto = bpf_htons(ETH_P_8021Q);
-	return 0;
-}
-
-/*
- * Swaps destination and source MAC addresses inside an Ethernet header
- */
-static __always_inline void swap_src_dst_mac(struct ethhdr *eth)
-{
-	__u8 h_tmp[ETH_ALEN];
-
-	__builtin_memcpy(h_tmp, eth->h_source, ETH_ALEN);
-	__builtin_memcpy(eth->h_source, eth->h_dest, ETH_ALEN);
-	__builtin_memcpy(eth->h_dest, h_tmp, ETH_ALEN);
-}
-
-/*
- * Swaps destination and source IPv6 addresses inside an IPv6 header
- */
-static __always_inline void swap_src_dst_ipv6(struct ipv6hdr *ipv6)
-{
-	struct in6_addr tmp = ipv6->saddr;
-
-	ipv6->saddr = ipv6->daddr;
-	ipv6->daddr = tmp;
-}
-
-/*
- * Swaps destination and source IPv4 addresses inside an IPv4 header
- */
-static __always_inline void swap_src_dst_ipv4(struct iphdr *iphdr)
-{
-	__be32 tmp = iphdr->saddr;
-
-	iphdr->saddr = iphdr->daddr;
-	iphdr->daddr = tmp;
-}
-
-#endif /* __REWRITE_HELPERS_H */

---
base-commit: c5d5786d3dc938af0b279c542d1e43bce381b4b9
change-id: 20251003-fix-forwarder-build-2889ea9aec91
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

Re: [PATCH RFC DOESNOTBUILD v2] tools/xdp-forwarder: Do not include libc headers in eBPF programs

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 2199 bytes --]

On 10/6/25 03:30, Demi Marie Obenour wrote:
> The build happened to work on arm64 because the glibc arm64 headers
> don't support multilib.  On x86_64, glibc headers assume that BPF is a
> 32-bit platform (because __x86_64__ isn't defined) and fail to find the
> 32-bit headers.  This is not a glibc bug.  Rather, BPF programs should
> not be including glibc headers.
> 
> Most Linux headers are not trivial to include in BPF programs.  The
> version of the headers meant for userspace use do include glibc headers,
> and that isn't supported in BPF.  The version meant for building
> kernel modules does not, but using it requires much more complicated
> build system.
> 
> Solve this problem by only including headers intended for use in BPF
> programs.  These headers include declarations explicitly intended for
> use in BPF programs, so if they do pull in libc headers that is a bug.
> This also allows using an unwrapped clang.  Nix's wrapped clang is not
> suitable when a target is explicitly specified, so this is another a
> bug fix.
> 
> This prevents vendoring the example headers, so replace them with a
> header that only includes common functionality common to both eBPF
> programs.  A single struct including both Ethernet II and 802.1Q headers
> is used, massively simplifying the code.
> 
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>

Build failures are because the compiler can't find <linux/bpf.h>.
It appears that libbpf does not provide its own version of that file,
and in any case the include path to find it is wrong.  Using a wrapped
compiler is wrong but might fix this.  Another option, which I think
is probably best, is to move this to vm/sys/net and build it against
the headers from vm/sys/net's kernel.  The build only requires the
UAPI headers, which are stable, so any kernel build that is not truly
ancient can be used.  The correct include directory is

    "${kernel-pkg.dev}/lib/modules/${kernel-pkg.version}/source/include/uapi"

where kernel-pkg is the Nix kernel package derivation.

Also, Yureka, can you look at the (massive) changes to the C code?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3] tools/xdp-forwarder: Do not include libc headers in eBPF programs

[Demi Marie Obenour]

The build happened to work on arm64 because the glibc arm64 headers
don't support multilib.  On x86_64, glibc headers assume that BPF is a
32-bit platform (because __x86_64__ isn't defined) and fail to find the
32-bit headers.  This is not a glibc bug.  Rather, BPF programs should
not be including glibc headers.

Most Linux headers are not trivial to include in BPF programs.  The
version of the headers meant for userspace use do include glibc headers,
and that isn't supported in BPF.  The version meant for building
kernel modules does not, but using it requires much more complicated
build system.

Solve this problem by only including headers intended for use in BPF
programs.  These headers include declarations explicitly intended for
use in BPF programs, so if they do pull in libc headers that is a bug.
This also allows using an unwrapped clang.  Nix's wrapped clang is not
suitable when a target is explicitly specified, so this is another a
bug fix.

This prevents vendoring the example headers, so replace them with a
header that only includes common functionality common to both eBPF
programs.  A single struct including both Ethernet II and 802.1Q headers
is used, massively simplifying the code.

Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v3:
- Fix build.
- Clean up XDP programs.
- Link to v2: https://lore.kernel.org/r/20251006-fix-forwarder-build-v2-1-6fdd7e05cb14@gmail.com

Changes in v2:
- Rewrite the programs to not use the helpers.
- This doesn't build.
- Link to v1: https://lore.kernel.org/r/20251003-fix-forwarder-build-v1-1-856b78ae5656@gmail.com
---
To check that the program does not include libc headers, one can add
#include <stdio.h> (or any other libc header) and check that one gets
an appropriate error.
---
 tools/default.nix                     |  10 +-
 tools/meson.options                   |   7 +
 tools/xdp-forwarder/helpers.h         |  54 +++++++
 tools/xdp-forwarder/meson.build       |  16 +-
 tools/xdp-forwarder/parsing_helpers.h | 274 ----------------------------------
 tools/xdp-forwarder/prog_physical.c   |  49 +++---
 tools/xdp-forwarder/prog_router.c     |  47 ++++--
 tools/xdp-forwarder/rewrite_helpers.h | 146 ------------------
 8 files changed, 142 insertions(+), 461 deletions(-)

diff --git a/tools/default.nix b/tools/default.nix
index 2c6846c80073e7b64fb7a19488103f6cf97a4420..8fd3491010303f981dac80cd6581614d594f993c 100644
--- a/tools/default.nix
+++ b/tools/default.nix
@@ -6,7 +6,7 @@ import ../lib/call-package.nix (
 { src, lib, stdenv, fetchCrate, fetchurl, runCommand, buildPackages
 , meson, ninja, pkg-config, rustc
 , clang-tools, clippy, jq
-, dbus
+, dbus, linuxHeaders
 # clang 19 (current nixpkgs default) is too old to support -fwrapv-pointer
 , clang_21, libbpf
 , buildSupport ? false
@@ -88,7 +88,7 @@ stdenv.mkDerivation (finalAttrs: {
     ++ lib.optionals (appSupport || driverSupport) [ pkg-config ]
     ++ lib.optionals hostSupport [ rustc ]
     ++ lib.optionals driverSupport [ clang_21 ];
-  buildInputs = lib.optionals appSupport [ dbus ] ++ lib.optionals driverSupport [ libbpf ];
+  buildInputs = lib.optionals appSupport [ dbus ] ++ lib.optionals driverSupport [ libbpf linuxHeaders ];
 
   postPatch = lib.optionals hostSupport (lib.concatMapStringsSep "\n" (crate: ''
     mkdir -p subprojects/packagecache
@@ -104,11 +104,11 @@ stdenv.mkDerivation (finalAttrs: {
     "-Dtests=false"
     "-Dunwind=false"
     "-Dwerror=true"
+  ] ++ lib.optionals driverSupport [
+    "-Dclang=${clang_21.cc}/bin/clang"
+    "-Dlinux-headers=${linuxHeaders}/include"
   ];
 
-  # Not supported for target bpf
-  hardeningDisable = lib.optionals driverSupport [ "zerocallusedregs" ];
-
   passthru.tests = {
     clang-tidy = finalAttrs.finalPackage.overrideAttrs (
       { name, src, nativeBuildInputs ? [], ... }:
diff --git a/tools/meson.options b/tools/meson.options
index 301efb9f677fdec57c8491fd6a6868f2d35cb076..2aa82a3188f2912487d075c8f320a63c14878f91 100644
--- a/tools/meson.options
+++ b/tools/meson.options
@@ -13,6 +13,13 @@ option('driver', type : 'boolean', value : false,
 option('hostfsrootdir', type : 'string', value : '/run/host',
   description : 'Path where the virtio-fs provided by the host will be mounted')
 
+option('clang', type : 'string',
+  description : 'Path to clang')
+
+option('linux-headers',
+  type : 'string',
+  description : 'Path to Linux kernel headers')
+
 option('tests',
   type : 'boolean',
   description : 'Build tests')
diff --git a/tools/xdp-forwarder/helpers.h b/tools/xdp-forwarder/helpers.h
new file mode 100644
index 0000000000000000000000000000000000000000..f9bcbf48a230bd374d1035a0f9da8d04dc301505
--- /dev/null
+++ b/tools/xdp-forwarder/helpers.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+/* SPDX-FileCopyrightText: 2021 The xdp-tutorial Authors */
+// SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+// SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
+/*
+ * This file contains a helper for validating a VLAN tag,
+ * as well as structs used by both BPF programs.
+ */
+
+#ifndef HELPERS_H
+#define HELPERS_H
+
+#include <linux/bpf.h>
+#include <bpf/bpf_endian.h>
+#include <bpf/bpf_helpers.h>
+
+#define VLAN_ETHTYPE    0x8100 /* Ethertype for tagged frames */
+
+struct ethhdr {
+	struct {
+		__u8 destination_mac[6];
+		__u8 source_mac[6];
+	} mac_addresses;
+	__be16 h_proto;
+};
+
+struct vlan_hdr {
+	__be16 h_vlan_TCI;
+	__be16 h_vlan_encapsulated_proto;
+};
+
+struct maybe_tagged_ethhdr {
+	union {
+		struct {
+			struct ethhdr eth;
+			struct vlan_hdr vlan;
+		} tagged;
+		struct {
+			struct vlan_hdr pad;
+			struct ethhdr eth;
+		} untagged;
+	};
+};
+
+// The router doesn't support the PCP and DEI bits
+// and they are not part of the VLAN tag.
+// Therefore, ensure they are unset.
+// Also reject VLAN 0, which is reserved.
+static __always_inline bool vlan_tag_is_valid(__u32 tag)
+{
+	return tag >= 1 && tag <= 0x0FFF;
+}
+
+#endif /* HELPERS_H */
diff --git a/tools/xdp-forwarder/meson.build b/tools/xdp-forwarder/meson.build
index b73130eb27b8000a102b0a8847ecb06b93a955d2..dec75c55884c4317af747b9a12e8f791d09d0a19 100644
--- a/tools/xdp-forwarder/meson.build
+++ b/tools/xdp-forwarder/meson.build
@@ -9,10 +9,18 @@ executable('set-router-iface', 'set_router_iface.c',
   dependencies : libbpf,
   install : true)
 
-clang = find_program('clang', native : true)
+clang_path = get_option('clang')
+if clang_path == ''
+  error('clang must be provided to build XDP forwarder')
+endif
+
+linux_headers_path = get_option('linux-headers')
+if linux_headers_path == ''
+  error('Linux header path must be provided to build XDP forwarder')
+endif
 
 bpf_o_cmd = [
-  clang.full_path(),
+  clang_path,
   '-fno-stack-protector',
   '-fno-strict-aliasing',
   '-fwrapv', '-fwrapv-pointer',
@@ -22,10 +30,12 @@ bpf_o_cmd = [
   '-Wno-sign-compare',
   '-O2',
   '-target', 'bpf',
-  '-I', meson.current_source_dir() + '/include',
   '-g',
   '-c',
+  '-std=gnu23',
   '-o', '@OUTPUT@',
+  '-I', libbpf.get_variable('includedir'),
+  '-I', linux_headers_path,
   '-MD',
   '-MQ', '@OUTPUT',
   '-MF', '@DEPFILE@',
diff --git a/tools/xdp-forwarder/parsing_helpers.h b/tools/xdp-forwarder/parsing_helpers.h
deleted file mode 100644
index da099346008bd58485af8308feb4d3391ceef8f5..0000000000000000000000000000000000000000
--- a/tools/xdp-forwarder/parsing_helpers.h
+++ /dev/null
@@ -1,274 +0,0 @@
-/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
-/* SPDX-FileCopyrightText: 2021 The xdp-tutorial Authors */
-/* Vendored from https://github.com/xdp-project/xdp-tutorial/blob/d3d3eed6ea9a63d1302bfa8b5a8e93862bfe11f0/common/parsing_helpers.h */
-/*
- * This file contains parsing functions that are used in the packetXX XDP
- * programs. The functions are marked as __always_inline, and fully defined in
- * this header file to be included in the BPF program.
- *
- * Each helper parses a packet header, including doing bounds checking, and
- * returns the type of its contents if successful, and -1 otherwise.
- *
- * For Ethernet and IP headers, the content type is the type of the payload
- * (h_proto for Ethernet, nexthdr for IPv6), for ICMP it is the ICMP type field.
- * All return values are in host byte order.
- *
- * The versions of the functions included here are slightly expanded versions of
- * the functions in the packet01 lesson. For instance, the Ethernet header
- * parsing has support for parsing VLAN tags.
- */
-
-#ifndef __PARSING_HELPERS_H
-#define __PARSING_HELPERS_H
-
-#include <stddef.h>
-#include <linux/if_ether.h>
-#include <linux/if_packet.h>
-#include <linux/ip.h>
-#include <linux/ipv6.h>
-#include <linux/icmp.h>
-#include <linux/icmpv6.h>
-#include <linux/udp.h>
-#include <linux/tcp.h>
-
-/* Header cursor to keep track of current parsing position */
-struct hdr_cursor {
-	void *pos;
-};
-
-/*
- *	struct vlan_hdr - vlan header
- *	@h_vlan_TCI: priority and VLAN ID
- *	@h_vlan_encapsulated_proto: packet type ID or len
- */
-struct vlan_hdr {
-	__be16 h_vlan_TCI;
-	__be16 h_vlan_encapsulated_proto;
-};
-
-/*
- * Struct icmphdr_common represents the common part of the icmphdr and icmp6hdr
- * structures.
- */
-struct icmphdr_common {
-	__u8 type;
-	__u8 code;
-	__sum16 cksum;
-};
-
-/* Allow users of header file to redefine VLAN max depth */
-#ifndef VLAN_MAX_DEPTH
-#define VLAN_MAX_DEPTH 2
-#endif
-
-#define VLAN_VID_MASK           0x0fff /* VLAN Identifier */
-/* Struct for collecting VLANs after parsing via parse_ethhdr_vlan */
-struct collect_vlans {
-	__u16 id[VLAN_MAX_DEPTH];
-};
-
-static __always_inline int proto_is_vlan(__u16 h_proto)
-{
-	return !!(h_proto == bpf_htons(ETH_P_8021Q) ||
-	          h_proto == bpf_htons(ETH_P_8021AD));
-}
-
-/* Notice, parse_ethhdr() will skip VLAN tags, by advancing nh->pos and returns
- * next header EtherType, BUT the ethhdr pointer supplied still points to the
- * Ethernet header. Thus, caller can look at eth->h_proto to see if this was a
- * VLAN tagged packet.
- */
-static __always_inline int parse_ethhdr_vlan(struct hdr_cursor *nh,
-                                             void *data_end,
-                                             struct ethhdr **ethhdr,
-                                             struct collect_vlans *vlans)
-{
-	struct ethhdr *eth = nh->pos;
-	int hdrsize = sizeof(*eth);
-	struct vlan_hdr *vlh;
-	__u16 h_proto;
-	int i;
-
-	/* Byte-count bounds check; check if current pointer + size of header
-	 * is after data_end.
-	 */
-	if (nh->pos + hdrsize > data_end)
-		return -1;
-
-	nh->pos += hdrsize;
-	*ethhdr = eth;
-	vlh = nh->pos;
-	h_proto = eth->h_proto;
-
-	/* Use loop unrolling to avoid the verifier restriction on loops;
-	 * support up to VLAN_MAX_DEPTH layers of VLAN encapsulation.
-	 */
-	#pragma unroll
-	for (i = 0; i < VLAN_MAX_DEPTH; i++) {
-		if (!proto_is_vlan(h_proto))
-			break;
-
-		if (vlh + 1 > data_end)
-			break;
-
-		h_proto = vlh->h_vlan_encapsulated_proto;
-		if (vlans) /* collect VLAN ids */
-			vlans->id[i] =
-				(bpf_ntohs(vlh->h_vlan_TCI) & VLAN_VID_MASK);
-
-		vlh++;
-	}
-
-	nh->pos = vlh;
-	return h_proto; /* network-byte-order */
-}
-
-static __always_inline int parse_ethhdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct ethhdr **ethhdr)
-{
-	/* Expect compiler removes the code that collects VLAN ids */
-	return parse_ethhdr_vlan(nh, data_end, ethhdr, NULL);
-}
-
-static __always_inline int parse_ip6hdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct ipv6hdr **ip6hdr)
-{
-	struct ipv6hdr *ip6h = nh->pos;
-
-	/* Pointer-arithmetic bounds check; pointer +1 points to after end of
-	 * thing being pointed to. We will be using this style in the remainder
-	 * of the tutorial.
-	 */
-	if (ip6h + 1 > data_end)
-		return -1;
-
-	nh->pos = ip6h + 1;
-	*ip6hdr = ip6h;
-
-	return ip6h->nexthdr;
-}
-
-static __always_inline int parse_iphdr(struct hdr_cursor *nh,
-                                       void *data_end,
-                                       struct iphdr **iphdr)
-{
-	struct iphdr *iph = nh->pos;
-	int hdrsize;
-
-	if (iph + 1 > data_end)
-		return -1;
-
-	hdrsize = iph->ihl * 4;
-	/* Sanity check packet field is valid */
-	if(hdrsize < sizeof(*iph))
-		return -1;
-
-	/* Variable-length IPv4 header, need to use byte-based arithmetic */
-	if (nh->pos + hdrsize > data_end)
-		return -1;
-
-	nh->pos += hdrsize;
-	*iphdr = iph;
-
-	return iph->protocol;
-}
-
-static __always_inline int parse_icmp6hdr(struct hdr_cursor *nh,
-                                          void *data_end,
-                                          struct icmp6hdr **icmp6hdr)
-{
-	struct icmp6hdr *icmp6h = nh->pos;
-
-	if (icmp6h + 1 > data_end)
-		return -1;
-
-	nh->pos   = icmp6h + 1;
-	*icmp6hdr = icmp6h;
-
-	return icmp6h->icmp6_type;
-}
-
-static __always_inline int parse_icmphdr(struct hdr_cursor *nh,
-                                         void *data_end,
-                                         struct icmphdr **icmphdr)
-{
-	struct icmphdr *icmph = nh->pos;
-
-	if (icmph + 1 > data_end)
-		return -1;
-
-	nh->pos  = icmph + 1;
-	*icmphdr = icmph;
-
-	return icmph->type;
-}
-
-static __always_inline int parse_icmphdr_common(struct hdr_cursor *nh,
-                                                void *data_end,
-                                                struct icmphdr_common **icmphdr)
-{
-	struct icmphdr_common *h = nh->pos;
-
-	if (h + 1 > data_end)
-		return -1;
-
-	nh->pos  = h + 1;
-	*icmphdr = h;
-
-	return h->type;
-}
-
-/*
- * parse_udphdr: parse the udp header and return the length of the udp payload
- */
-static __always_inline int parse_udphdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct udphdr **udphdr)
-{
-	int len;
-	struct udphdr *h = nh->pos;
-
-	if (h + 1 > data_end)
-		return -1;
-
-	nh->pos  = h + 1;
-	*udphdr = h;
-
-	len = bpf_ntohs(h->len) - sizeof(struct udphdr);
-	if (len < 0)
-		return -1;
-
-	return len;
-}
-
-/*
- * parse_tcphdr: parse and return the length of the tcp header
- */
-static __always_inline int parse_tcphdr(struct hdr_cursor *nh,
-                                        void *data_end,
-                                        struct tcphdr **tcphdr)
-{
-	int len;
-	struct tcphdr *h = nh->pos;
-
-	if (h + 1 > data_end)
-		return -1;
-
-	len = h->doff * 4;
-	/* Sanity check packet field is valid */
-	if(len < sizeof(*h))
-		return -1;
-
-	/* Variable-length TCP header, need to use byte-based arithmetic */
-	if (nh->pos + len > data_end)
-		return -1;
-
-	nh->pos += len;
-	*tcphdr = h;
-
-	return len;
-}
-
-#endif /* __PARSING_HELPERS_H */
diff --git a/tools/xdp-forwarder/prog_physical.c b/tools/xdp-forwarder/prog_physical.c
index 2b70654ebe21cb2504e6d26ca3b93cb4b62efb0f..4c6744e905132d3cc2fcc21ce936f7893fab29fa 100644
--- a/tools/xdp-forwarder/prog_physical.c
+++ b/tools/xdp-forwarder/prog_physical.c
@@ -1,12 +1,8 @@
 // SPDX-License-Identifier: EUPL-1.2+
 // SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+// SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
 
-#define VLAN_MAX_DEPTH 1
-
-#include <linux/bpf.h>
-#include <bpf/bpf_endian.h>
-#include "parsing_helpers.h"
-#include "rewrite_helpers.h"
+#include "helpers.h"
 
 struct {
 	__uint(type, BPF_MAP_TYPE_DEVMAP);
@@ -16,24 +12,43 @@ struct {
 	__uint(pinning, LIBBPF_PIN_BY_NAME);
 } router_iface SEC(".maps");
 
+static __always_inline bool push_vlan_tag(struct xdp_md *ctx, __u16 tag)
+{
+	struct maybe_tagged_ethhdr *hdr;
+
+	// Add extra space at the front of the packet.
+	// Doing this first avoids reloading pointers and
+	// extra bounds checks later.
+	if (bpf_xdp_adjust_head(ctx, -(int)sizeof(hdr->untagged.pad)))
+		return false;
+
+	hdr = (void *)(long)ctx->data;
+	if (hdr + 1 > (void *)(long)ctx->data_end)
+		return false;
+
+	// Move the MAC addresses.
+	// Ethertype is already in the correct position.
+	__builtin_memmove(&hdr->tagged.eth.mac_addresses,
+	                  &hdr->untagged.eth.mac_addresses,
+	                  sizeof(hdr->tagged.eth.mac_addresses));
+
+	// Set the VLAN ID and the Ethertype of the frame.
+	hdr->tagged.vlan.h_vlan_TCI = bpf_htons((__u16)tag);
+	hdr->tagged.eth.h_proto = bpf_htons(VLAN_ETHTYPE);
+	return true;
+}
+
 SEC("xdp")
 int physical(struct xdp_md *ctx)
 {
-	void *data_end = (void *)(long)ctx->data_end;
-	void *data = (void *)(long)ctx->data;
-
-	struct hdr_cursor nh;
-	nh.pos = data;
-
-	struct ethhdr *eth;
-	if (parse_ethhdr(&nh, data_end, &eth) < 0)
-		return XDP_DROP;
+	__u32 ingress_ifindex = ctx->ingress_ifindex;
 
-	if (ctx->ingress_ifindex < 1 || ctx->ingress_ifindex > VLAN_VID_MASK)
+	if (!vlan_tag_is_valid(ingress_ifindex))
 		return XDP_DROP;
 
-	if (vlan_tag_push(ctx, eth, ctx->ingress_ifindex) < 0)
+	if (!push_vlan_tag(ctx, (__u16)ingress_ifindex))
 		return XDP_DROP;
 
+	// Redirect to the router interface.
 	return bpf_redirect_map(&router_iface, 0, 0);
 }
diff --git a/tools/xdp-forwarder/prog_router.c b/tools/xdp-forwarder/prog_router.c
index 5b0c8fa5c55e365c08fdb1be8f0613bdc67d8649..21595d3c66e596d085d6be7e2057a0aeaaf1c55a 100644
--- a/tools/xdp-forwarder/prog_router.c
+++ b/tools/xdp-forwarder/prog_router.c
@@ -1,12 +1,8 @@
 // SPDX-License-Identifier: EUPL-1.2+
 // SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+// SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
 
-#define VLAN_MAX_DEPTH 1
-
-#include <linux/bpf.h>
-#include <bpf/bpf_endian.h>
-#include "parsing_helpers.h"
-#include "rewrite_helpers.h"
+#include "helpers.h"
 
 // The map is actually not used by this program, but just included
 // to keep the reference-counted pin alive before any physical interfaces
@@ -19,23 +15,42 @@ struct {
 	__uint(pinning, LIBBPF_PIN_BY_NAME);
 } router_iface SEC(".maps");
 
+static __always_inline bool pop_vlan_tag(struct xdp_md *ctx, __u16 *tag)
+{
+	struct maybe_tagged_ethhdr *hdr = (void *)(long)ctx->data;
+	if (hdr + 1 > (void *)(long)ctx->data_end)
+		return false;
+
+	// 0x8A88 is also valid but the router does not
+	// use it.  It's meant for service provider VLANs.
+	if (hdr->tagged.eth.h_proto != bpf_htons(VLAN_ETHTYPE))
+		return false;
+
+	*tag = bpf_ntohs(hdr->tagged.vlan.h_vlan_TCI);
+
+	// Move the MAC addresses.
+	// Ethertype is already in correct position.
+	__builtin_memmove(&hdr->untagged.eth.mac_addresses,
+	                  &hdr->tagged.eth.mac_addresses,
+	                  sizeof(hdr->tagged.eth.mac_addresses));
+
+	// Move the head pointer to the new Ethernet header.
+	// Doing this last avoids needing to reload pointers
+	// and add extra bounds checks earlier.
+	return !bpf_xdp_adjust_head(ctx, (int)sizeof(hdr->untagged.pad));
+}
 
 SEC("xdp")
 int router(struct xdp_md *ctx)
 {
-	void *data_end = (void *)(long)ctx->data_end;
-	void *data = (void *)(long)ctx->data;
+	__u16 vlid;
 
-	struct hdr_cursor nh;
-	nh.pos = data;
-
-	struct ethhdr *eth;
-	if (parse_ethhdr(&nh, data_end, &eth) < 0)
-		return XDP_DROP;
+	if (!pop_vlan_tag(ctx, &vlid))
+		return false;
 
-	int vlid = vlan_tag_pop(ctx, eth);
-	if (vlid < 0)
+	if (!vlan_tag_is_valid(vlid))
 		return XDP_DROP;
 
+	// Redirect to the correct physical interface.
 	return bpf_redirect(vlid, 0);
 }
diff --git a/tools/xdp-forwarder/rewrite_helpers.h b/tools/xdp-forwarder/rewrite_helpers.h
deleted file mode 100644
index 71aa23e038cc450087c1db20c00cbdbe1616b0a1..0000000000000000000000000000000000000000
--- a/tools/xdp-forwarder/rewrite_helpers.h
+++ /dev/null
@@ -1,146 +0,0 @@
-/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
-/* SPDX-FileCopyrightText: 2019 The xdp-tutorial Authors */
-/* Vendored from https://github.com/xdp-project/xdp-tutorial/blob/d3d3eed6ea9a63d1302bfa8b5a8e93862bfe11f0/common/rewrite_helpers.h */
-/*
- * This file contains functions that are used in the packetXX XDP programs to
- * manipulate on packets data. The functions are marked as __always_inline, and
- * fully defined in this header file to be included in the BPF program.
- */
-
-#ifndef __REWRITE_HELPERS_H
-#define __REWRITE_HELPERS_H
-
-#include <linux/bpf.h>
-#include <linux/ip.h>
-#include <linux/ipv6.h>
-#include <linux/if_ether.h>
-
-#include <bpf/bpf_helpers.h>
-#include <bpf/bpf_endian.h>
-
-/* Pops the outermost VLAN tag off the packet. Returns the popped VLAN ID on
- * success or negative errno on failure.
- */
-static __always_inline int vlan_tag_pop(struct xdp_md *ctx, struct ethhdr *eth)
-{
-	void *data_end = (void *)(long)ctx->data_end;
-	struct ethhdr eth_cpy;
-	struct vlan_hdr *vlh;
-	__be16 h_proto;
-	int vlid;
-
-	if (!proto_is_vlan(eth->h_proto))
-		return -1;
-
-	/* Careful with the parenthesis here */
-	vlh = (void *)(eth + 1);
-
-	/* Still need to do bounds checking */
-	if (vlh + 1 > data_end)
-		return -1;
-
-	/* Save vlan ID for returning, h_proto for updating Ethernet header */
-	vlid = bpf_ntohs(vlh->h_vlan_TCI);
-	h_proto = vlh->h_vlan_encapsulated_proto;
-
-	/* Make a copy of the outer Ethernet header before we cut it off */
-	__builtin_memcpy(&eth_cpy, eth, sizeof(eth_cpy));
-
-	/* Actually adjust the head pointer */
-	if (bpf_xdp_adjust_head(ctx, (int)sizeof(*vlh)))
-		return -1;
-
-	/* Need to re-evaluate data *and* data_end and do new bounds checking
-	 * after adjusting head
-	 */
-	eth = (void *)(long)ctx->data;
-	data_end = (void *)(long)ctx->data_end;
-	if (eth + 1 > data_end)
-		return -1;
-
-	/* Copy back the old Ethernet header and update the proto type */
-	__builtin_memcpy(eth, &eth_cpy, sizeof(*eth));
-	eth->h_proto = h_proto;
-
-	return vlid;
-}
-
-/* Pushes a new VLAN tag after the Ethernet header. Returns 0 on success,
- * -1 on failure.
- */
-static __always_inline int vlan_tag_push(struct xdp_md *ctx,
-                                         struct ethhdr *eth, int vlid)
-{
-	void *data_end = (void *)(long)ctx->data_end;
-	struct ethhdr eth_cpy;
-	struct vlan_hdr *vlh;
-
-	/* First copy the original Ethernet header */
-	__builtin_memcpy(&eth_cpy, eth, sizeof(eth_cpy));
-
-	/* Then add space in front of the packet */
-	if (bpf_xdp_adjust_head(ctx, 0 - (int)sizeof(*vlh)))
-		return -1;
-
-	/* Need to re-evaluate data_end and data after head adjustment, and
-	 * bounds check, even though we know there is enough space (as we
-	 * increased it).
-	 */
-	data_end = (void *)(long)ctx->data_end;
-	eth = (void *)(long)ctx->data;
-
-	if (eth + 1 > data_end)
-		return -1;
-
-	/* Copy back Ethernet header in the right place, populate VLAN tag with
-	 * ID and proto, and set outer Ethernet header to VLAN type.
-	 */
-	__builtin_memcpy(eth, &eth_cpy, sizeof(*eth));
-
-	vlh = (void *)(eth + 1);
-
-	if (vlh + 1 > data_end)
-		return -1;
-
-	vlh->h_vlan_TCI = bpf_htons(vlid);
-	vlh->h_vlan_encapsulated_proto = eth->h_proto;
-
-	eth->h_proto = bpf_htons(ETH_P_8021Q);
-	return 0;
-}
-
-/*
- * Swaps destination and source MAC addresses inside an Ethernet header
- */
-static __always_inline void swap_src_dst_mac(struct ethhdr *eth)
-{
-	__u8 h_tmp[ETH_ALEN];
-
-	__builtin_memcpy(h_tmp, eth->h_source, ETH_ALEN);
-	__builtin_memcpy(eth->h_source, eth->h_dest, ETH_ALEN);
-	__builtin_memcpy(eth->h_dest, h_tmp, ETH_ALEN);
-}
-
-/*
- * Swaps destination and source IPv6 addresses inside an IPv6 header
- */
-static __always_inline void swap_src_dst_ipv6(struct ipv6hdr *ipv6)
-{
-	struct in6_addr tmp = ipv6->saddr;
-
-	ipv6->saddr = ipv6->daddr;
-	ipv6->daddr = tmp;
-}
-
-/*
- * Swaps destination and source IPv4 addresses inside an IPv4 header
- */
-static __always_inline void swap_src_dst_ipv4(struct iphdr *iphdr)
-{
-	__be32 tmp = iphdr->saddr;
-
-	iphdr->saddr = iphdr->daddr;
-	iphdr->daddr = tmp;
-}
-
-#endif /* __REWRITE_HELPERS_H */

---
base-commit: c5d5786d3dc938af0b279c542d1e43bce381b4b9
change-id: 20251003-fix-forwarder-build-2889ea9aec91

-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

Re: [PATCH v3] tools/xdp-forwarder: Do not include libc headers in eBPF programs

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 4633 bytes --]

Demi Marie Obenour <demiobenour@gmail.com> writes:

> The build happened to work on arm64 because the glibc arm64 headers
> don't support multilib.  On x86_64, glibc headers assume that BPF is a
> 32-bit platform (because __x86_64__ isn't defined) and fail to find the
> 32-bit headers.  This is not a glibc bug.  Rather, BPF programs should
> not be including glibc headers.
>
> Most Linux headers are not trivial to include in BPF programs.  The
> version of the headers meant for userspace use do include glibc headers,
> and that isn't supported in BPF.  The version meant for building
> kernel modules does not, but using it requires much more complicated
> build system.
>
> Solve this problem by only including headers intended for use in BPF
> programs.  These headers include declarations explicitly intended for
> use in BPF programs, so if they do pull in libc headers that is a bug.
> This also allows using an unwrapped clang.  Nix's wrapped clang is not
> suitable when a target is explicitly specified, so this is another a
> bug fix.
>
> This prevents vendoring the example headers, so replace them with a
> header that only includes common functionality common to both eBPF
> programs.  A single struct including both Ethernet II and 802.1Q headers
> is used, massively simplifying the code.

Could we put that cleanup into a separate patch, and keep this patch as
the smallest change possible necessary to fix the build?

> diff --git a/tools/default.nix b/tools/default.nix
> index 2c6846c80073e7b64fb7a19488103f6cf97a4420..8fd3491010303f981dac80cd6581614d594f993c 100644
> --- a/tools/default.nix
> +++ b/tools/default.nix
> @@ -6,7 +6,7 @@ import ../lib/call-package.nix (
>  { src, lib, stdenv, fetchCrate, fetchurl, runCommand, buildPackages
>  , meson, ninja, pkg-config, rustc
>  , clang-tools, clippy, jq
> -, dbus
> +, dbus, linuxHeaders
>  # clang 19 (current nixpkgs default) is too old to support -fwrapv-pointer
>  , clang_21, libbpf
>  , buildSupport ? false
> @@ -88,7 +88,7 @@ stdenv.mkDerivation (finalAttrs: {
>      ++ lib.optionals (appSupport || driverSupport) [ pkg-config ]
>      ++ lib.optionals hostSupport [ rustc ]
>      ++ lib.optionals driverSupport [ clang_21 ];
> -  buildInputs = lib.optionals appSupport [ dbus ] ++ lib.optionals driverSupport [ libbpf ];
> +  buildInputs = lib.optionals appSupport [ dbus ] ++ lib.optionals driverSupport [ libbpf linuxHeaders ];
>  
>    postPatch = lib.optionals hostSupport (lib.concatMapStringsSep "\n" (crate: ''
>      mkdir -p subprojects/packagecache
> @@ -104,11 +104,11 @@ stdenv.mkDerivation (finalAttrs: {
>      "-Dtests=false"
>      "-Dunwind=false"
>      "-Dwerror=true"
> +  ] ++ lib.optionals driverSupport [
> +    "-Dclang=${clang_21.cc}/bin/clang"

llvmPackages_21.clang-unwrapped would prevent having to go through the
unwrapped compiler.

> +    "-Dlinux-headers=${linuxHeaders}/include"
>    ];
>  
> -  # Not supported for target bpf
> -  hardeningDisable = lib.optionals driverSupport [ "zerocallusedregs" ];
> -
>    passthru.tests = {
>      clang-tidy = finalAttrs.finalPackage.overrideAttrs (
>        { name, src, nativeBuildInputs ? [], ... }:
> diff --git a/tools/meson.options b/tools/meson.options
> index 301efb9f677fdec57c8491fd6a6868f2d35cb076..2aa82a3188f2912487d075c8f320a63c14878f91 100644
> --- a/tools/meson.options
> +++ b/tools/meson.options
> @@ -13,6 +13,13 @@ option('driver', type : 'boolean', value : false,
>  option('hostfsrootdir', type : 'string', value : '/run/host',
>    description : 'Path where the virtio-fs provided by the host will be mounted')
>  
> +option('clang', type : 'string',
> +  description : 'Path to clang')
> +
> +option('linux-headers',
> +  type : 'string',
> +  description : 'Path to Linux kernel headers')

Let's make this the package prefix, rather than the include directory.

> +
>  option('tests',
>    type : 'boolean',
>    description : 'Build tests')
> diff --git a/tools/xdp-forwarder/meson.build b/tools/xdp-forwarder/meson.build
> index b73130eb27b8000a102b0a8847ecb06b93a955d2..dec75c55884c4317af747b9a12e8f791d09d0a19 100644
> --- a/tools/xdp-forwarder/meson.build
> +++ b/tools/xdp-forwarder/meson.build
> @@ -9,10 +9,18 @@ executable('set-router-iface', 'set_router_iface.c',
>    dependencies : libbpf,
>    install : true)
>  
> -clang = find_program('clang', native : true)
> +clang_path = get_option('clang')
> +if clang_path == ''
> +  error('clang must be provided to build XDP forwarder')
> +endif

What was wrong with find_program()?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]