Add linux-headers-bbrv3 meta so DKMS builds and ZFS-root boots

The kernel shipped image-only: install.sh installed just linux-image-bbrv3,
which pulled no headers, so DKMS (zfs-dkms) had no build tree and never built
zfs.ko for the new kernel. On a ZFS-on-root box that left zfs out of the
initramfs -> root could not be mounted -> the machine failed to boot
(hit live on baguette; maple/wunder are ext4-root so they booted but ran
docker on an unimported, empty rpool).

Permanent fix:
- build-meta-package.sh now also builds a linux-headers-bbrv3 meta (amd64) and
  makes linux-image-bbrv3 Depend on it, so headers always install in the same
  apt transaction as the kernel and DKMS rebuilds on every upgrade (mirrors
  Debian's linux-image-amd64 / linux-headers-amd64 pair). arm64 ships no headers
  (nokernelheaders cross build), so the dep and meta are amd64-only.
- build.yml derives the headers package name and passes it to the meta builder.
- registry-has-all.sh counts the headers meta as part of a complete publish.
- install.sh installs linux-headers-bbrv3 alongside the image (latest path) and
  the matching versioned headers in the specific-version path.

Also bundles pre-existing WIP: docs/README notes on the two-stage (Cloudflare
edge + origin nginx) upload body-limit, and setup_apt_source key-download
HTTP-status error handling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.github/workflows/build.yml

@@ -18,13 +18,16 @@ env:
   # https://git.chilldove.com/api/packages/icePigeon/debian
   PKG_REGISTRY: ${{ github.server_url }}/api/packages/${{ github.repository_owner }}/debian
   PKG_USER: ${{ github.repository_owner }}
-  # Optional origin pin for uploads. The kernel image .deb is >100 MB, which trips
-  # the request-body cap of a CDN/proxy fronting the registry (Cloudflare free/pro
-  # caps at 100 MB and returns 413 at the edge, before the body reaches Gitea — the
-  # origin itself accepts it). Set the Gitea repo/org variable PKG_ORIGIN_IP to the
-  # registry origin's address (the host behind the CDN) and the publish step pins the
-  # upload straight there via curl --resolve, bypassing the CDN. vars (unlike secrets)
-  # ARE available to workflow-level env. Leave empty if no CDN fronts uploads.
+  # Optional origin pin for uploads. The kernel image .deb is >128 MB and a 413 can
+  # come from TWO request-body caps in series: (1) a CDN fronting the registry
+  # (Cloudflare free/pro = 100 MB, rejected at the edge before the body reaches the
+  # origin) and (2) the origin reverse proxy's own client_max_body_size (e.g. nginx).
+  # Set the Gitea repo/org variable PKG_ORIGIN_IP to the registry origin's address
+  # (the host behind the CDN) and the publish step pins the upload straight there via
+  # curl --resolve, bypassing cap (1). Cap (2) is server-side: the origin proxy must
+  # allow client_max_body_size >= the .deb size, or it still 413s after the bypass.
+  # vars (unlike secrets) ARE available to workflow-level env. Leave empty if no CDN
+  # fronts uploads.
   PKG_ORIGIN_IP: ${{ vars.PKG_ORIGIN_IP }}
   # NOTE: PKG_TOKEN (the write:package secret) is declared at JOB level below. The
   # secrets context is not available to workflow-level env, so a value set here
@@ -294,7 +297,20 @@ jobs:
           img_pkg=$(dpkg-deb -f "$img_deb" Package)
           echo "Kernel image package: $img_pkg"
 
-          bash scripts/build-meta-package.sh "$KERNEL_VERSION" "$DEB_ARCH" "$img_pkg" ./out
+          # Headers package: present on amd64, absent on the arm64 (nokernelheaders)
+          # cross build. When present, build-meta-package.sh emits a linux-headers-bbrv3
+          # meta and makes linux-image-bbrv3 depend on it, so DKMS (e.g. ZFS) always has
+          # a build tree and the box can't end up image-only / unbootable.
+          hdr_deb=$(find ./kernel -maxdepth 1 -name 'linux-headers-*.deb' ! -name '*-dbg*' | sort | head -n1)
+          hdr_pkg=""
+          if [ -n "$hdr_deb" ]; then
+            hdr_pkg=$(dpkg-deb -f "$hdr_deb" Package)
+            echo "Kernel headers package: $hdr_pkg"
+          else
+            echo "No linux-headers .deb (expected on arm64); headers meta skipped."
+          fi
+
+          bash scripts/build-meta-package.sh "$KERNEL_VERSION" "$DEB_ARCH" "$img_pkg" ./out "$hdr_pkg"
 
           # bbrv3-config is Architecture:all; build it once (on the primary arch).
           if [ "$IS_PRIMARY" = "true" ]; then
@@ -322,12 +338,13 @@ jobs:
             exit 1
           fi
 
-          # The kernel image .deb is >100 MB, which trips the request-body cap of a
-          # CDN/proxy in front of the registry (Cloudflare free/pro = 100 MB) and 413s
-          # at the edge before the body reaches Gitea — the origin itself accepts it.
-          # If PKG_ORIGIN_IP is set, pin the upload straight to that origin so the
-          # oversized PUT never touches the CDN. SNI/Host stay $reg_host, so TLS and
-          # package routing are unchanged; only the TCP connect target moves.
+          # The kernel image .deb is >128 MB and can 413 on either of two body caps:
+          # a CDN edge (Cloudflare free/pro = 100 MB) and the origin proxy's own
+          # client_max_body_size. If PKG_ORIGIN_IP is set, pin the upload straight to
+          # that origin so the oversized PUT never touches the CDN (clears the edge
+          # cap). SNI/Host stay $reg_host, so TLS and package routing are unchanged;
+          # only the TCP connect target moves. The origin proxy must still be
+          # configured to accept the .deb size, or it 413s even after the bypass.
           reg_host=${PKG_REGISTRY#*://}; reg_host=${reg_host%%/*}
           resolve=()
           if [ -n "${PKG_ORIGIN_IP:-}" ]; then

README.md

@@ -170,7 +170,7 @@ modinfo tcp_bbr 2>/dev/null | grep '^version:'
 
 构建不会自动更新 BBR patch 本身。
 
-发布所需：CI 配置一个有 `write:package` 权限的 `PKG_TOKEN` secret；Gitea 实例需启用 Packages → Debian registry。若 registry 前面有 CDN/反代对上传请求体设限（如 Cloudflare 免费/Pro 版 100 MB 上限），内核镜像 `.deb`（>100 MB）会在边缘被 413 拒绝——把 Gitea 源站地址（CDN 背后主机的 IP）配成 Gitea 仓库变量 `PKG_ORIGIN_IP`，发布步骤会用 `curl --resolve` 把上传直连源站、绕过 CDN（SNI/Host 不变，证书与包路由不受影响）。源站本身不受此限。
+发布所需：CI 配置一个有 `write:package` 权限的 `PKG_TOKEN` secret；Gitea 实例需启用 Packages → Debian registry。内核镜像 `.deb`（>128 MB）的上传可能撞上**两道串联的请求体上限**，任一道都会 413：（1）registry 前面的 CDN/反代边缘限制（如 Cloudflare 免费/Pro 版 100 MB），在 body 到达源站前就拒掉；（2）源站反代自身的 `client_max_body_size`（如 nginx）。绕过第（1）道：把 Gitea 源站地址（CDN 背后主机的 IP）配成 Gitea 仓库变量 `PKG_ORIGIN_IP`，发布步骤会用 `curl --resolve` 把上传直连源站、绕过 CDN（SNI/Host 不变，证书与包路由不受影响）。第（2）道是服务端配置：源站反代的 `client_max_body_size` 必须 ≥ `.deb` 体积（实践中曾遇到源站 nginx 默认 128 MB 卡住，需调高到 1G 或 0），否则绕过 CDN 后仍会 413。
 
 ## 免责声明
 

docs/superpowers/specs/2026-06-17-apt-repo-migration-design.md

@@ -58,10 +58,13 @@
 - CI 需 `PKG_TOKEN` secret（有 `write:package` scope）；自动注入的 `GITHUB_TOKEN` 不保证有
   包写权限。
 - registry 保留历史版本以支撑「装指定版本」；将来再议 retention。
-- **上传路径不能经过有 body 上限的 CDN**（落地后实测补记）：`git.chilldove.com` 对境外
-  runner 走 Cloudflare（orange-cloud），CF 免费/Pro 版 100 MB 请求体上限会把 >100 MB 的内核
-  镜像 `.deb` 在边缘 413（源站 Baguette 实测可收 ≥110 MB）。CI 须配 Gitea 仓库变量
-  `PKG_ORIGIN_IP`，发布步骤用 `curl --resolve` 直连源站绕过 CF。详见
+- **上传路径有两道串联的 body 上限，缺一不可**（落地后实测补记）：内核镜像 `.deb` >128 MB，
+  两道任一都会 413。第①道 CDN 边缘：`git.chilldove.com` 对境外 runner 走 Cloudflare
+  （orange-cloud），CF 免费/Pro 版 100 MB 请求体上限在边缘 413。配 Gitea 仓库变量
+  `PKG_ORIGIN_IP`，发布步骤用 `curl --resolve` 直连源站 `51.77.132.151` 绕过 CF。但绕过 CF 后
+  又撞第②道源站反代：Baguette 源站 nginx `client_max_body_size` 实测仅 **128m**（128 MiB 收、
+  129 MiB 413），>128 MB 的 deb 仍 413。修法是把源站 vhost 调高（已改 `client_max_body_size
+  1G`，实测 1100 MB 才 413）。两道都过才发得上。详见
   [[工程实践/案例/2026-06-17 Gitea Debian registry 上传被 Cloudflare 100MB body 上限 413]]。
 
 ## 不做（YAGNI）

install.sh

@@ -59,6 +59,11 @@ KEYRING="/etc/apt/keyrings/gitea-$GITEA_OWNER.asc"
 SOURCE_LIST="/etc/apt/sources.list.d/bbrv3.list"
 # 跟随最新 stable 的 meta-package；sysctl 配置由 bbrv3-config 包负责
 META_PKG="linux-image-bbrv3"
+# 头文件 meta-package：跟随最新 stable 的 linux-headers-<版本>-bbrv3。装上它，
+# DKMS 模块（如 ZFS）才能在每次内核安装/升级时为新内核编译出 .ko。否则内核只装
+# 了 image、没有 build tree，ZFS-on-root 机器的 initramfs 里没有 zfs.ko，根文件
+# 系统挂不上、直接无法开机。仅 amd64 提供（arm64 交叉构建不带头文件）。
+HEADERS_META="linux-headers-bbrv3"
 SYSCTL_DROPIN="/etc/sysctl.d/99-bbrv3.conf"
 # 可选：访问私有仓库时给拉取签名公钥用（支持 GITEA_TOKEN / GITHUB_TOKEN / GH_TOKEN）
 GITEA_API_TOKEN="${GITEA_TOKEN:-${GITHUB_TOKEN:-${GH_TOKEN:-}}}"
@@ -142,18 +147,31 @@ setup_apt_source() {
     echo -e "\033[36m正在配置 BBRv3 apt 源...\033[0m"
     sudo install -d -m 0755 /etc/apt/keyrings
 
-    # 拉取 Gitea registry 的签名公钥（armored，可直接用作 signed-by）
-    local key_args=()
+    # 拉取 Gitea registry 的签名公钥（armored，可直接用作 signed-by）。
+    # 先下载到临时文件再落盘：旧写法 `curl | sudo tee` 的管道退出码取自 tee
+    # （写空文件也成功），会掩盖 curl 的失败，把「HTTP 401」误报成「公钥为空」。
+    local key_args=() http_code tmp_key
     [[ -n "$GITEA_API_TOKEN" ]] && key_args=(-H "Authorization: token $GITEA_API_TOKEN")
-    if ! curl -fsSL "${key_args[@]}" "$REG_BASE/repository.key" | sudo tee "$KEYRING" > /dev/null; then
-        echo -e "\033[31m下载仓库签名公钥失败：$REG_BASE/repository.key\033[0m"
-        echo -e "\033[33m提示：私有仓库可先 export GITEA_TOKEN=你的令牌 再重试。\033[0m"
+    tmp_key="$(mktemp)"
+    http_code=$(curl -sS -L -o "$tmp_key" -w '%{http_code}' "${key_args[@]}" "$REG_BASE/repository.key" || echo 000)
+
+    if [[ "$http_code" == "401" || "$http_code" == "403" ]]; then
+        rm -f "$tmp_key"
+        echo -e "\033[31m无法读取签名公钥：registry 需要鉴权（HTTP $http_code）。\033[0m"
+        echo -e "\033[31m地址：$REG_BASE/repository.key\033[0m"
+        echo -e "\033[33mBBRv3 apt 源必须可匿名读取。请在 Gitea（$GITEA_HOST）把套件 owner『$GITEA_OWNER』的可见性设为『公开 / Public』，\033[0m"
+        echo -e "\033[33m让匿名用户能下载公钥、apt 索引与 .deb；否则每台机器都得写凭据，违背 curl|bash 公共源的设计。\033[0m"
+        echo -e "\033[33m（若确需保持私有：export GITEA_TOKEN=令牌 可拉到公钥，但 apt 拉取仍需把凭据写入 /etc/apt/auth.conf.d/，当前脚本未实现。）\033[0m"
         return 1
     fi
-    if [[ ! -s "$KEYRING" ]]; then
-        echo -e "\033[31m签名公钥为空，无法继续。\033[0m"
+    if [[ "$http_code" != "200" ]] || [[ ! -s "$tmp_key" ]]; then
+        rm -f "$tmp_key"
+        echo -e "\033[31m下载仓库签名公钥失败（HTTP $http_code）：$REG_BASE/repository.key\033[0m"
+        echo -e "\033[33m请检查网络、DNS 或 registry 是否可访问。\033[0m"
         return 1
     fi
+    sudo install -m 0644 "$tmp_key" "$KEYRING"
+    rm -f "$tmp_key"
 
     echo "deb [signed-by=$KEYRING] $REG_BASE $REG_DIST $REG_COMP" | sudo tee "$SOURCE_LIST" > /dev/null
     echo -e "\033[1;32m✔ 已写入 apt 源：$SOURCE_LIST\033[0m"
@@ -181,12 +199,19 @@ install_latest_version() {
     assert_supported_kernel_install_system || return 1
     setup_apt_source || return 1
 
-    echo -e "\033[36m正在安装/升级 $META_PKG（自动跟随最新 stable，旧内核会保留以便回退）...\033[0m"
+    # amd64 一并装 headers meta（DKMS 的 build tree）。image meta 在 amd64 上也
+    # Depends 它，这里显式列出更直观，并且即便 image 的依赖被改成 Recommends 也不漏装。
+    # arm64 无 headers 包，只装 image meta。
+    local install_pkgs=("$META_PKG")
+    [[ "$(get_deb_arch)" == "amd64" ]] && install_pkgs+=("$HEADERS_META")
+
+    echo -e "\033[36m正在安装/升级 ${install_pkgs[*]}（自动跟随最新 stable，旧内核会保留以便回退）...\033[0m"
     # 内核包的 postinst 会自动更新引导（update-grub / kernel-install），无需手动处理；
-    # bbrv3-config 的 postinst 会写入并应用 BBR + CAKE + ECN 的 sysctl 配置。
-    if sudo apt-get install -y "$META_PKG"; then
+    # headers 与 image 同一个事务装入，DKMS（如 zfs-dkms）会在内核 postinst 里为新内核
+    # 编译模块；bbrv3-config 的 postinst 会写入并应用 BBR + CAKE + ECN 的 sysctl 配置。
+    if sudo apt-get install -y "${install_pkgs[@]}"; then
         echo -e "\033[1;32m内核安装并配置完成！默认网络栈：BBR + CAKE + ECN。\033[0m"
-        echo -e "\033[33m如不希望以后 apt upgrade 自动更换内核，可执行：sudo apt-mark hold $META_PKG\033[0m"
+        echo -e "\033[33m如不希望以后 apt upgrade 自动更换内核，可执行：sudo apt-mark hold ${install_pkgs[*]}\033[0m"
         prompt_reboot
     else
         echo -e "\033[1;31m内核安装失败！系统未变更内核，可重试或检查上面的 apt 报错。\033[0m"
@@ -224,15 +249,24 @@ install_specific_version() {
     local pkg="${PKGS[$((CHOICE-1))]}"
     echo -e "\033[36m已选择：\033[0m\033[1;32m$pkg\033[0m"
 
-    if sudo apt-get install -y "$pkg"; then
-        echo -e "\033[1;32m已安装 $pkg。\033[0m"
+    # 同时安装对应版本的 headers（DKMS build tree），否则 zfs 等模块无法为该内核编译，
+    # ZFS-on-root 机器会无法开机。headers 仅 amd64 提供，仓库里有才装。
+    local hdr_pkg="${pkg/linux-image-/linux-headers-}"
+    local specific_pkgs=("$pkg")
+    if apt-cache show "$hdr_pkg" &> /dev/null; then
+        specific_pkgs+=("$hdr_pkg")
+        echo -e "\033[36m将同时安装匹配的头文件：\033[0m\033[1;32m$hdr_pkg\033[0m"
+    fi
+
+    if sudo apt-get install -y "${specific_pkgs[@]}"; then
+        echo -e "\033[1;32m已安装 ${specific_pkgs[*]}。\033[0m"
         # 装指定版本通常是想钉死，提示如何阻止 apt 自动更换内核
         if dpkg -s "$META_PKG" &> /dev/null; then
             echo -n -e "\033[33m检测到已安装 $META_PKG，是否钉死当前内核、阻止 apt upgrade 自动更换？ (y/N): \033[0m"
             read -r HOLD_NOW
             if [[ "$HOLD_NOW" == "y" || "$HOLD_NOW" == "Y" ]]; then
-                sudo apt-mark hold "$META_PKG" "$pkg"
-                echo -e "\033[1;32m已钉死：$META_PKG $pkg（解钉：sudo apt-mark unhold ...）\033[0m"
+                sudo apt-mark hold "$META_PKG" "${specific_pkgs[@]}"
+                echo -e "\033[1;32m已钉死：$META_PKG ${specific_pkgs[*]}（解钉：sudo apt-mark unhold ...）\033[0m"
             fi
         fi
         prompt_reboot

scripts/build-meta-package.sh

@@ -1,38 +1,76 @@
 #!/usr/bin/env bash
-# Build the linux-image-bbrv3 meta-package. It depends on a specific built kernel
-# package plus bbrv3-config, so `apt upgrade` tracks the newest BBRv3 kernel while
-# previously installed kernels stay around for fallback.
+# Build the BBRv3 meta-packages that track the newest stable build:
 #
-# The dependency is the *actual* produced package name (passed in), not a guessed
-# one, so any localversion/suffix drift from the kernel build can't break it.
+#   linux-image-bbrv3    Depends on the freshly built kernel image + bbrv3-config,
+#                        and (amd64) on linux-headers-bbrv3 so a kernel build tree
+#                        is ALWAYS installed alongside the image. Without the headers
+#                        dependency the kernel installs image-only and DKMS modules
+#                        (e.g. ZFS) silently can't build for the new kernel -> on a
+#                        ZFS-on-root box the initramfs has no zfs.ko, root won't mount
+#                        and the machine fails to boot. The dependency makes apt pull
+#                        headers in the same transaction so DKMS rebuilds on every
+#                        kernel upgrade, exactly like Debian's stock linux-image-amd64
+#                        / linux-headers-amd64 pair.
+#
+#   linux-headers-bbrv3  Depends on the matching linux-headers-<version>-bbrv3.
+#                        amd64 only: the arm64 cross build ships no headers
+#                        (DEB_BUILD_PROFILES=pkg.linux-upstream.nokernelheaders), so
+#                        there is nothing to track there and the meta is not built.
+#
+# Depending on the *actual* produced package names (passed in) keeps this immune to
+# localversion/suffix drift from the kernel build.
 set -euo pipefail
 
-meta_version=${1:?usage: build-meta-package.sh <meta_version> <deb_arch> <depends_pkg> <out_dir>}
+meta_version=${1:?usage: build-meta-package.sh <meta_version> <deb_arch> <image_pkg> <out_dir> [headers_pkg]}
 arch=${2:?missing deb arch (amd64|arm64)}
-depends_pkg=${3:?missing kernel image package name to depend on}
+image_pkg=${3:?missing kernel image package name to depend on}
 out_dir=${4:?missing output dir}
+headers_pkg=${5:-}   # empty on arm64 (no headers package is built there)
 
 work=$(mktemp -d)
 trap 'rm -rf "$work"' EXIT
+mkdir -p "$out_dir"
 
-pkgdir="$work/meta"
-mkdir -p "$pkgdir/DEBIAN"
-
-cat > "$pkgdir/DEBIAN/control" <<EOF
-Package: linux-image-bbrv3
+# build_meta <package> <Depends:> <Description first line> <extended description body>
+build_meta() {
+  local pkg=$1 depends=$2 short=$3 body=$4
+  local pkgdir="$work/$pkg"
+  mkdir -p "$pkgdir/DEBIAN"
+  cat > "$pkgdir/DEBIAN/control" <<EOF
+Package: $pkg
 Version: $meta_version
 Architecture: $arch
 Maintainer: linux-kernel-bbrv3
-Depends: $depends_pkg, bbrv3-config
+Depends: $depends
 Section: kernel
 Priority: optional
-Description: BBRv3 kernel (tracks the latest stable build)
- Meta-package pulling in the newest BBRv3 kernel built by this project
- ($depends_pkg) plus the bbrv3-config network defaults. Run
- 'apt-mark hold linux-image-bbrv3' to pin your kernel and stop auto-tracking.
+Description: $short
+$body
 EOF
+  dpkg-deb --build --root-owner-group "$pkgdir" \
+    "$out_dir/${pkg}_${meta_version}_${arch}.deb"
+  echo "Built $out_dir/${pkg}_${meta_version}_${arch}.deb"
+}
 
-mkdir -p "$out_dir"
-dpkg-deb --build --root-owner-group "$pkgdir" \
-  "$out_dir/linux-image-bbrv3_${meta_version}_${arch}.deb"
-echo "Built $out_dir/linux-image-bbrv3_${meta_version}_${arch}.deb"
+# linux-image-bbrv3 — on amd64 it also pulls linux-headers-bbrv3 (pinned to the same
+# version) so DKMS has a build tree at install time and on every upgrade.
+image_depends="$image_pkg, bbrv3-config"
+if [ -n "$headers_pkg" ]; then
+  image_depends="$image_depends, linux-headers-bbrv3 (= $meta_version)"
+fi
+build_meta "linux-image-bbrv3" "$image_depends" \
+  "BBRv3 kernel (tracks the latest stable build)" \
+  " Meta-package pulling in the newest BBRv3 kernel built by this project
+ ($image_pkg) plus the bbrv3-config network defaults. On amd64 it also pulls
+ linux-headers-bbrv3 so DKMS modules such as ZFS build for the kernel and the
+ machine stays bootable. Run 'apt-mark hold linux-image-bbrv3' to pin your
+ kernel and stop auto-tracking."
+
+# linux-headers-bbrv3 — only where a headers package exists (amd64).
+if [ -n "$headers_pkg" ]; then
+  build_meta "linux-headers-bbrv3" "$headers_pkg" \
+    "BBRv3 kernel headers (tracks the latest stable build)" \
+    " Meta-package pulling in the headers ($headers_pkg) for the newest BBRv3
+ kernel built by this project, so DKMS modules (e.g. ZFS) can build against it.
+ Installed automatically as a dependency of linux-image-bbrv3 on amd64."
+fi

scripts/registry-has-all.sh

@@ -9,6 +9,7 @@
 #   - linux-image-bbrv3  == <version>   meta-package tracking THIS version
 #   - bbrv3-config                      Architecture: all sysctl defaults
 #   - linux-headers-<version>-bbrv3     amd64 only (arm64 cross build omits headers)
+#   - linux-headers-bbrv3 == <version>  amd64 only: headers meta tracking THIS version
 #
 # Architecture: all packages (bbrv3-config) appear in every binary-<arch> index,
 # so reading the single binary-<arch>/Packages file covers the whole set.
@@ -42,6 +43,7 @@ has_name_ver "linux-image-bbrv3" "$version"  || missing+=("linux-image-bbrv3=$ve
 has_name     "bbrv3-config"                  || missing+=("bbrv3-config")
 if [ "$arch" = "amd64" ]; then
   has_name     "linux-headers-$version-bbrv3"  || missing+=("linux-headers-$version-bbrv3")
+  has_name_ver "linux-headers-bbrv3" "$version" || missing+=("linux-headers-bbrv3=$version")
 fi
 
 if [ ${#missing[@]} -eq 0 ]; then