# Containerised build at a canonical path `make container-build` compiles capOS inside a container with the worktree bind-mounted at `/build`. The host build path is unchanged and remains the default. ## Why a container sccache hashes the compiler process's current working directory unconditionally for every Rust compilation, so two worktrees at different host paths never share a first-party cache entry. This is upstream-tracked and unfixed: [mozilla/sccache#2652](https://github.com/mozilla/sccache/issues/2652) (the Rust hash key ignores `SCCACHE_BASEDIRS`) and [#2595](https://github.com/mozilla/sccache/issues/2595) (this exact worktree case). `SCCACHE_BASEDIRS`, `--remap-path-prefix` and `-Zremap-cwd-prefix` were each measured at zero additional hits; the first does not apply to Rust at all despite its README, and the latter two rewrite the artifact rather than the key. Making the directory genuinely identical is the only mechanism that works, and it is the only one that stays honest: the artifacts really do carry `/build`, rather than carrying whichever worktree happened to compile them first. ## What must match for a cache hit | Must be identical | Need not be identical | |---|---| | The container mount path (`/build`) | The host worktree path | | `CARGO_HOME` (`/cargo`) | The container uid | | The toolchain mount path (`/rust`) | `HOME` | | Every other `CARGO_*` variable, name and value | Any other variable no compilation reads | | Any other variable a compilation reads | The base image | | The rustc binary's *content* | | The toolchain mount path belongs in the left column for the same reason the worktree path does: cargo runs each unit's rustc with the working directory set to that package's own root, and a `-Zbuild-std=core,alloc` unit's package root is under `/lib/rustlib/src/rust/library/`. Every userspace build alias in `.cargo/config.toml` uses `-Zbuild-std`, so parametrizing the toolchain mount would silently lose every `core` and `alloc` hit. `capos-container-build.sh` hardcodes `/rust`, which is why this is a latent trap rather than a live defect, and the `ubuntu:24.04` versus `python:3.12-slim` spike is not counter-evidence: it held the mount at `/rust` in both arms. The environment rule has two halves, and only the first is unconditional: - **Every `CARGO_*` variable enters the key regardless of whether anything reads it**, because `env!()` could. An unrelated `CARGO_UNRELATED=1` is enough to zero the hit rate. `capos-container-build.sh` therefore starts the container from an empty environment and sets the `CARGO_*` set explicitly, so no host variable can leak in. - **Any other variable enters the key when the compilation actually reads it.** rustc records such a read as an `env-dep` in dep-info, and sccache tracks it. Measured with a build script emitting `cargo:rustc-env` and a library doing `env!()`: changing a non-`CARGO_` variable missed, changing it back hit, and each rlib carried the value it was built with. Reading the first half alone invites the conclusion that a non-`CARGO_` variable is invisible to the key. It is not, and the difference matters here: `kernel/build.rs` emits `CAPOS_BUILD_COMMIT` / `CAPOS_BUILD_TIMESTAMP` / `CAPOS_BUILD_VERSION` as `cargo:rustc-env`, and `kernel/src/cap/system_info.rs` reads them with `env!()`. Treating them as invisible would be a wrong-build-identity bug: a kernel serving another commit's identity. The observed behaviour is the safe one -- see the reuse limits below. ## Measured reuse Two worktrees at the same commit, at different host paths, building `make kernel` (init plus kernel, so both the `x86_64-unknown-capos` userspace target and the `x86_64-unknown-none` kernel target, including the kernel link). Each arm starts from an empty cache and clean target directories, builds the first worktree to populate, then builds the second. | Arm | Populate | Second worktree | Cross-worktree hits | |---|---|---|---| | Host | 0 hits / 91 misses, 87 s | 79 hits / 12 misses, 64 s | 86.8% | | Container at `/build` | 0 hits / 91 misses, 91 s | 91 hits / 0 misses, 41 s | 100% | This table is **not reproduced**: it was taken once per cell, on a host running concurrent builds, and the run recorded neither the cache-isolation procedure nor the stats capture. Treat the hit and miss counts as indicative and the wall-clock figures as unreliable; the reuse *ratio* is the durable part, and the artifact identity below is separately reproducible. Re-measuring needs two worktrees at one commit, a dedicated `SCCACHE_DIR` per arm so the live host cache is neither read nor disturbed, drained target directories, and `CAPOS_CONTAINER_SCCACHE_STATS=1` for the container arm (the server dies with the container, so its counters can only be read from inside). The headline host rate is high because this goal is dominated by registry dependencies, and those already share: cargo runs each unit's rustc with the working directory set to that package's own root, so a crate unpacked under `CARGO_HOME` -- or a `-Zbuild-std` unit compiled from the rustup `rust-src` component -- is worktree-independent already. The 12 host misses are precisely the worktree-local first-party crates: `capos-abi`, `capos-lib`, `capos-config`, `capos-tls`, `capos-rt`, `capos-init`, `capos-kernel`, the vendored trees, and the build scripts. Those are the expensive units, which is why closing a 12-unit gap moved wall clock at all rather than trivially -- though see the load caveat on the table above before quoting a figure. So the rate itself is workload-dependent -- a goal that builds `demos/` or `shell/` has a far higher first-party fraction and a correspondingly lower host baseline. The invariant underneath it is that first-party cross-worktree reuse goes from 0% to 100% over the cacheable set, at equal commits and at differing commits alike -- see the different-commit measurement below, which also corrects what `capos-kernel`'s exclusion actually is. ### The different-commit case, measured -- and `capos-kernel` is not a miss at all The equal-commit table above is the best case. The autopilot case is two worktrees on different branch tips, and it was left unmeasured. Measured 2026-07-26, and the result corrects the explanation this document previously gave. Procedure, against the live shared container cache (so the numbers are the ones a real worker sees, not an isolated ideal): Create two detached worktrees at the same mainline commit. In the second, commit one added line in a kernel source file, so its tip, its sources and its `CAPOS_BUILD_COMMIT` all differ from the first's. Then, in each worktree in turn: ```sh CAPOS_CONTAINER_SCCACHE_STATS=1 make container-build CONTAINER_GOALS=kernel ``` The stats variable is required: the container gets its own network namespace, so its sccache server dies with the container and its counters can only be read from inside. | Worktree | Requests | Executed | Hits | Misses | Non-cacheable | |---|---|---|---|---|---| | First worktree | 123 | 91 | 91 | 0 | 29 | | Second worktree, +1 kernel commit | 123 | 91 | 91 | 0 | 29 | **First-party cross-worktree reuse at a different commit is 100% of the cacheable set: 91 of 91, zero misses.** Both worktrees are at different host paths, and the second differs from the first in a kernel source file, its commit and its build identity. The zero-miss column is the part that needed explaining, because a changed kernel source cannot possibly hit. It does not: **`capos-kernel` is never offered to the cache.** Isolated directly -- touch one kernel source in the second worktree and rebuild, so the kernel is the only unit that recompiles: ``` Compile requests 3 Compile requests executed 0 Cache hits 0 Cache misses 0 Non-cacheable calls 1 reason: crate-type Non-compilation calls 2 ``` So the earlier claim here -- that `CAPOS_BUILD_COMMIT` in the unit's key makes `capos-kernel` *miss* across branch tips -- described the wrong mechanism. sccache declines the kernel's `rustc` invocation outright on `crate-type`, in every build, at every commit, in both host and container mode. The environment rule is real (see above) but has no bearing on the kernel, because the kernel's compile never reaches key computation. The consequence is a **ceiling, not a regression**: the container path cannot accelerate the kernel compile-and-link, and neither can the host path, so nothing is lost by switching. Of the 123 requests in this goal, 29 are non-cacheable (`crate-type` 15, `missing input` 9, stdin 5) and 3 are non-compilation probes; the reuse claim covers the other 91. Also directly observed: two container builds of the same tree at different commits produce a byte-identical `capos-init` and a *different* kernel. With the mechanism above, that difference is simply the kernel being compiled fresh both times, not a cache decision. ## The kernel linker script is not an obstacle Worth recording, because it is easy to assume otherwise. The kernel linker script is passed as a **relative** path from `.cargo/config.toml` (`-C link-arg=-Tkernel/linker-x86_64.ld`), resolved against the compile working directory, which is the workspace root. It does not travel through `CARGO_ENCODED_RUSTFLAGS` -- the Makefile only ever *clears* that variable, for the standalone wasm payload builds. So the kernel-target cache key carries no absolute path from the link step, and the kernel ELF links inside the container unchanged. ## Layout - `tools/container-build/Dockerfile` -- distribution packages only: `gcc` plus glibc headers, `make`, and `git`. No Rust toolchain, no capOS tool cache. Pinned by base-image digest. - `tools/container-build/capos-container-build.sh` -- resolves the host paths and runs `make ` inside the container. Its pre-flight refusals all come before any provisioning work: an unsupported goal, a missing container runtime, a missing `rustc`, a missing `sccache`, a missing toolchain, an image that is not present locally, a bind-mount path containing `,` or `=` (the `--mount` option string has no escaping for either), a container build of the same worktree already running, an active host build reservation, and a target directory in the other build mode. Ordering matters because the capnp build stamp depends on the `Makefile`, so any `Makefile` edit can otherwise make a from-source capnp build the price of being told `sccache` is missing. The image check matters because `docker run` treats a missing local image as a registry reference and reports a Docker Hub pull failure instead of "run `make container-image` first". The already-running check matters because `--rm` only removes a container after it exits, so a `SIGKILL` to the CLI leaves one running; the container is named from the worktree so concurrent builds of *different* worktrees stay allowed, which is the normal parallel case. - `tools/container-build/capos-build-mode.sh` -- the build-mode guard and its self-test. The container is not a sandbox and `make container-build` is not a hardening measure. `$HOME/.cargo` is bind-mounted read-write of necessity, since cargo unpacks registry sources into it; that directory holds `credentials.toml` and a `bin/` that is on the host `PATH`, so a build script running in the container can read a crates.io token or overwrite a host binary. This is not a regression -- a host build already runs the same build scripts with the same user's rights -- and nothing in this path claims isolation. It is recorded so the container is not later read as a trust boundary. `--network none` narrows what a build can reach, but it is there to make an unexpected fetch fail loudly, not to contain a hostile build script. Everything version-pinned stays on the host and is bind-mounted: - **Rust toolchain** -- `rust-toolchain.toml` is already the pin, and the measured cache key depends on the compiler's content rather than its path. Baking a toolchain into the image would add a second pin that can drift from the first, and would make image reproducibility a precondition for cache-key reproducibility. Mounted read-only at `/rust`. - **capnp** -- `capos-config`'s build script executes it, so it must be reachable inside the container, and `tools/capnp-build` asserts its exact path under `CAPOS_TOOLS_ROOT`. The Makefile's existing SHA-256-pinned provisioning is unchanged; the tool cache is mounted read-only at `/capos-tools` and `capnp-ensure` inside the container only re-verifies it. - **cue, limine** -- reachable inside the container through the same read-only `CAPOS_TOOLS_ROOT` mount, still SHA-256-pinned by the Makefile. Their *provisioning* stays on the host: the extraction recipes write into that mount, which fails inside the container as a read-only-filesystem error from within a `flock`, and pinned provisioning should not silently happen in a container anyway. So the host pre-flight runs `cue-ensure` and `limine-ensure` alongside `capnp-ensure`, and the container only consumes them. - **xorriso** -- installed in the image, because unlike `cue` and `limine` it has no host provisioning to mount. It is the one image package that masters shipped artifact bytes, so it is recorded in `docs/trusted-build-inputs.md` rather than treated like `gcc` (whose output never reaches the shipped ELFs). - **mdbook, typst** -- docs are not compilation. They stay on the host, outside this harness. The git common directory is mounted at its own host path, read-only, because a worktree's `.git` file names that absolute path and `kernel/build.rs` derives the build identity (`CAPOS_BUILD_COMMIT`, `CAPOS_BUILD_TIMESTAMP`) from git. Without it the kernel would stamp itself `unknown`. It is not a `CARGO_*` variable and is not a compile working directory, so it cannot perturb the key. ## Cache directory Container and host builds compute different keys for the same crate, so they share nothing. Pointing both at one directory would only have them compete for the same `SCCACHE_CACHE_SIZE` bound, so the container defaults to a separate `$HOME/.cache/capos-container-sccache`. Override with `CAPOS_CONTAINER_SCCACHE_DIR`, and collapse the two once the containerised path is the only one in use. The consequence is that the host's cache bound and the container's are independent, so the total budget is their sum; both are stated explicitly rather than left to sccache's default. Stating the container's bound explicitly is not optional tidiness. `HOME` inside the container is a `tmpfs`, so no host `~/.config/sccache/config` is readable and an operator's configured size (or any other option set there) does not apply. Without an explicit `SCCACHE_CACHE_SIZE` the container silently falls back to the built-in default, and this paragraph would be reasoning about a bound the container never reads. ## Not covered - **Running the QEMU gates inside the container.** `CONTAINER_GOALS` admits compilation and packaging, but not the gates themselves: they would need `qemu` in the image plus `--device /dev/kvm` and a supplementary `kvm` group, because `QEMU_KVM_FLAGS` falls back to TCG when `/dev/kvm` is unavailable -- correct, but far too slow for the smoke timeouts. Networking needs nothing extra: the harnesses use QEMU `user` networking, not a TAP device. The C and wasm payload goals are also out, because their recipes drive a C compiler and a second target triple that are not verified here. - **Booting a container-built ISO from a host QEMU target -- available, but only for `run-smoke` so far.** `$(ISO)` depends on the compile targets, so a plain host QEMU target over a container-built tree is the mixed-mode hazard above and the guard refuses it. `CAPOS_PREBUILT_ISO=1` is the way through: it drops the ISO from the target's prerequisites and takes the UART map from the ISO's sidecar instead of recomputing it with a host `mkmanifest`, so the whole flagged path compiles nothing. Only `run-smoke` carries the flag today. Every other QEMU target still reaches the ISO rule and is still refused in a container-mode worktree -- including `run-default-boot`, which CLAUDE.md names as *the* gate for the default `system.cue` boot and which the container can build (`all` produces `capos-default.iso`) but not yet boot. - **Verification parity between the two packaging modes.** `ovmf-verify` resolves `OVMF_CODE` by `wildcard` over host firmware paths, none of which exist in the image, so it degrades to its NOTICE and skips the pin check for every container-built ISO. The ISO does not embed OVMF and the firmware only matters to `run-uefi`, so the impact is small, but nothing records which checks the container mode skipped. - **Build-provenance identity across modes.** `tools/build-provenance.sh` runs on the host: it reads the mode from `.capos-build-mode` and would therefore report `Build mode: container` beside the *host's* `xorriso` version, describing a tool that did not master the artifact. The ISO's sidecar records the mode that built it, which is the seam to wire in. ## One build mode per worktree, enforced This is a correctness rule, not a performance note, and it is enforced rather than documented. Cargo's freshness check for a workspace member does **not** notice the switch: it tracks source files by a path relative to the package, and the mount makes those relative paths and their mtimes identical. So a build in one mode on top of the other mode's `target/` can mark a unit fresh and **relink the rlib the other mode produced**, which carries that mode's build root in its DWARF. Both directions reproduce, and the host-over-container direction is the worse one: the host path is the default that `make run`, the QEMU smokes and `make build-provenance` all consume. Observed there: a plain `make kernel` over a clean container-built target directory exited 0 with no warning and produced a kernel in which one compilation unit (`capos-abi`) had `DW_AT_comp_dir = /build`, a directory that does not exist on the host. Nothing in the build output hinted at it. The artifact looked correct. The extent is narrower than the mechanism suggests, which matters for triaging a real occurrence: in that reproduction 92 of 93 units recompiled and exactly one crate, `capos-abi`, survived stale. The reason is not simply that `CARGO_HOME` differs between the modes. That explains why registry units rebuild, but not why a first-party unit resolving no registry path rebuilds -- `capos-lib` recompiled too. The operative rule is cargo's dependency-output comparison: once a dependency's rlib is rewritten, every transitive dependent is dirty regardless of its own dep-info. That predicts the survivor set is exactly the **dependency-free first-party leaves**, which is a property of the workspace graph rather than a constant. `capos-abi` is currently the only workspace member with no dependencies and no build-dependencies, which is why it was the only survivor. If it ever gains a dependency, nothing survives a mode switch and the hazard disappears by accident; if a second dependency-free crate is added, two survive. The mechanism is inferred from cargo's fingerprint model rather than measured -- what was measured is the outcome, 92 of 93 -- but the graph premise is checkable against the workspace manifests, and the practical consequence stands either way: one mixed unit is enough to make the artifact wrong, and a mostly-reused build is not what an occurrence looks like. `tools/container-build/capos-build-mode.sh` closes this. It records the mode that last wrote build output in `.capos-build-mode` (gitignored, repository root), reserves the worktree through `.capos-build-mode-reservation`, and refuses a build in the other mode while either that mode has an active reservation or any target directory still holds output. `capos-container-build.sh` acquires its explicit reservation before provisioning or `docker run`, reuses it in both make invocations by passing the wrapper-issued reservation ID as a non-exported make command-line variable, and releases it from its exit trap. `make container-build-mode-test` is the self-test. Ordinary host reservations are bound to the outermost `make` process by PID, process start time, and boot identity. A later invocation reclaims one only after that exact process identity is gone, avoiding PID-reuse guesses. The same owner-generation rule applies to an explicitly named host-mode reservation; only container-mode reservations remain unreclaimable from host process state because their container may still be writing after the wrapper exits. A replacement reservation records the validated new owner generation rather than reusing fields read from the stale reservation. Reservation inspection, stale-owner reclamation, replacement, and owned release are serialized by the host's `flock`, so a contender cannot replace the directory between another guard's identity check and rename. A persistent rename failure refuses immediately instead of retrying while holding the lock. The in-container re-entry must present the wrapper-issued ID, validates it against the active reservation, and does not need `flock` inside the image. The wrapper arms `EXIT`, `INT`, and `TERM` cleanup before attempting acquisition, so an ordinary Ctrl-C or termination releases a reservation even during host capnp provisioning or the remaining Docker preflight. Its explicit reservation is released only by its matching identity. If the wrapper receives `SIGKILL`, or the host loses power before its trap runs, the guard fails closed rather than assuming the named container stopped writing. Two independent make invocations in one worktree are refused even when both request the same mode: host builds have distinct process-generation identities, and container-mode re-entry without the wrapper-issued reservation ID fails. The build-mode mix would be absent, but concurrent Cargo writes to one target tree are already outside this harness's contract and can race independently of provenance. Parallel builds in different worktrees use different reservation and lock files and never serialize. After verifying that no container for the worktree remains, an operator can release such an orphan with the ID stored in `.capos-build-mode-reservation/id`: ```sh tools/container-build/capos-build-mode.sh --release-reservation \ "$(sed -n 1p .capos-build-mode-reservation/id)" ``` Its reach into the Makefile is the assumption everything else here rests on, so it is asserted rather than described. The guard is a prerequisite of `capnp-ensure`, which covers every target that compiles capOS code for a capOS target — but that is not the same as every cargo invocation, and the difference was larger than a reading of the Makefile suggested. Writing the exception list by hand produced six targets. Computing the prerequisite closure from make's own rule database found **sixteen** cargo-invoking rules that did not reach the guard. Three families mattered: - `model-scheduler-nohz-loom`, a host-triple `capos-config` Loom test, and `wasi-env-negative-check`, a host-triple `mkmanifest` test, both write the **root `target/`** — the directory a container build fills. An unguarded write there can rewrite `target/.rustc_info.json` with the host sysroot inside a directory holding container output, which is the evidence the unmarked-worktree inference below depends on. - Eleven `demos/wasi-*/target/wasm32-wasip1/release/*.wasm` artifact rules, reachable through their `wasi-*-build` parents. These compile `wasm32-wasip1`, which disproves the reasoning offered for the hand list — that every bypass was host-triple and therefore harmless. Their directories are already in the guard's discovery set, so the guard could refuse a build over output nothing had declared. - `capos-tls-test` (deliberately capnp-free, since it builds only `-p capos-tls`), `webui-login-peer-logic-test`, `userspace-broker-policy-test`, `task-coordinator-logic-test`, `fuzz-build` and `fuzz-smoke`. The first two are `make check` members, so without the prerequisite `make check` in a container-mode worktree compiled into `demos/target/` before failing at `capos-rt-check`. All of them now reach the guard; the eleven artifact rules take it as an order-only prerequisite so a `.PHONY` dependency does not make every `.wasm` perpetually out of date. Only three cargo-invoking rules are deliberately unguarded, and `tools/container-build/check-cargo-guard-coverage.py` asserts *set equality* against exactly that allowlist: `fmt` and `fmt-check` compile nothing, and `clean` is the remedy for a refusal, so guarding it would strand a worktree that could then neither build in the recorded mode nor drain out of it. A newly unguarded rule fails the gate, and so does an allowlisted rule that becomes guarded. Widening the allowlist is a reviewed diff to that file. Two checks run, deliberately: the closure computation above is complete but parses `make -p` output, so it also asserts a floor on the number of cargo rules it found — a degraded parse would otherwise report everything as guarded. Beside it, `capos-build-mode.sh --check-guarded-targets` asks `make --dry-run` what a sample of targets would actually run. One is complete, the other is authoritative. Three properties are worth stating because they are what make it fail closed: - **Target directories are discovered, not listed.** Any directory named `target` at any depth that holds build output counts. The walk prunes `.git` and each target directory after finding it, so its cost does not scale with the contents of build output. This is the property that makes the guard cover the whole goal set rather than the default goal: nine standalone crate manifests build through `.cargo/config.toml` aliases -- `init/`, `demos/`, `shell/`, `capos-rt/`, `capos-service/`, `capos/`, `libcapos/`, `libcapos-posix/`, `capos-wasm/` -- each with its own `/target`, and the `demos/wasi-*` payloads are separate workspaces one level deeper. `CONTAINER_GOALS` is overridable, so `shell`, `demos`, `capos-rt-check` and friends are ordinary usage, and by the reuse argument above they are the goals with the *highest* first-party fraction -- the ones this path pays best on. A `target` plus `init/target` list would leave every one of them exposed. `make container-build-mode-test` asserts the whole set and the existing depth-four `tools/remote-session-client/src-tauri/target`: restricting discovery to the former depth bound or to the first two directories fails the gate. - **An unmarked worktree is classified from evidence.** Cargo records the rustc sysroot in `/.rustc_info.json`, and the container's sysroot is the fixed mount path `/rust`, so an unmarked directory still declares which mode filled it. Without this, a worktree holding container output produced before the guard existed would be misread as host and permitted -- exactly the mix the guard is for. Every occupied directory is classified rather than the first one that answers, so a tree that is *already* mixed is reported as such and refuses both modes instead of being adopted as whichever mode was found first. - **It refuses rather than wiping.** Discarding build output is the operator's call; the refusal names the directories and prints the command. The reservation lock is acquired before the mode marker is checked or written. This closes both empty-tree startup races: a host build cannot enter while the container wrapper is starting Docker, and a container wrapper cannot enter after a host guard passes but before that host build writes its first target artifact. The self-test exercises both directions without pre-populating a target directory, refuses a second independent same-mode owner, verifies that the OS lock remains held while stale-owner reclaim is paused, and races opposite-mode contenders over a stale host reservation to assert that exactly one proceeds. It also proves that a persistent reclaim error returns after one attempt, a stale explicit host reservation is reclaimed by owner generation, the replacement retains the new owner's generation, an orphaned explicit container reservation is never reclaimed, and an invalid owner reports its actual preflight failure without waiting for a partial claim. The cost is that the modes are exclusive for the *whole* worktree, not per goal. After a full host ISO build, switching to container builds means discarding every populated target directory, and vice versa. That is the same "pick one mode per worktree" rule as before, now with a build-time signal instead of a convention. Note `target/.gitkeep` is tracked, so `rm -rf target` needs it restored afterwards. Consequently a container-mode worktree cannot run the host gates (`make check` and everything under it reaches `capnp-ensure`), and a host-mode worktree cannot run `make container-build` without draining first. Drain a worktree -- empty target directories and no marker -- to leave it free for either mode. A completed host make may leave its process-bound reservation record behind; the next guard reclaims it after verifying that owner generation is no longer live. ## Artifacts Two worktrees at different host paths, each built clean in a container, produce a **byte-identical** kernel ELF (same SHA-256) and a byte-identical `capos-init` ELF. The kernel contains no host path at all: every first-party compilation unit's `DW_AT_comp_dir` is `/build`, and so are the embedded source-path strings. A host build of the same tree carries the worktree path in both. Measured on the `232e1791` container-built kernel: ```sh K=target/x86_64-unknown-none/debug/capos-kernel sha256sum "$K" init/target/x86_64-unknown-capos/release/capos-init readelf --debug-dump=info "$K" | grep -c 'DW_AT_comp_dir.*: /build$' # 49 readelf --debug-dump=info "$K" | grep DW_AT_comp_dir | sed 's/.*: //' \ | sort -u | wc -l # 22 strings -a "$K" | grep -oE '^/build/[A-Za-z0-9._/-]+' | sort -u # 5 paths strings -a "$K" | grep -c '/home/' # 0 ``` The 49 are the first-party units; the 22 distinct `comp_dir` values are those plus the `/cargo/registry/...` and `/rustc/` roots, which are worktree-independent in either mode. The five embedded source-path strings are `capos-rt/src/{panic,lib,entry}.rs`, `capos-config/src/validation.rs`, and the generated `capos_capnp.rs` under the build-script `OUT_DIR`. All five counts are commit-specific -- the `OUT_DIR` one embeds a build hash -- so re-measure rather than quoting them. That is what makes serving one worktree's cached object to another sound rather than a fudge: the compiler really did run in `/build` both times. Key normalisation alone would leave the artifact carrying whichever build root compiled it first, which is unacceptable for a kernel that prints paths. ## Remaining caveats - **`DW_AT_comp_dir` becomes `/build`.** Host-side debuggers resolving source from container-built artifacts need a path mapping (`set substitute-path /build ` in gdb). - **Panic locations change for the standalone crates.** Workspace members are unaffected: cargo passes relative source paths for them, so their `file!()` strings are identical either way. `init/` and its outside-the-package path dependencies are not, because cargo passes those as absolute paths. The container-built kernel, which embeds `capos-init` via `include_bytes!`, carries `/build/capos-rt/src/panic.rs`, `/build/capos-rt/src/lib.rs`, `/build/capos-rt/src/entry.rs` and `/build/capos-config/src/validation.rs` where a host build of the same tree carries the worktree path. So init-side panic output names a directory that does not exist on the host. - **The total sccache budget doubles.** The container defaults to its own cache directory (see above), and neither cache is a subset of the other, so the bound is the host's plus the container's -- currently 10G each. Both are set explicitly (`SCCACHE_CACHE_SIZE`, and `CAPOS_CONTAINER_SCCACHE_CACHE_SIZE` for the container) rather than inherited from sccache's default. A server that is already running keeps the size it started with. - **Build mode is recorded in provenance, and the two are not comparable.** `make build-provenance` reports `Build mode` and `Build container image` from the build-mode marker, because every other identity field in that record describes the host while the artifact bytes may not. A host artifact and a container artifact differ by construction, so a comparison across modes is not a reproducibility result; see the retention and comparison policy in `docs/trusted-build-inputs.md`. - **The image is a build input.** It is pinned by digest and inventoried in `docs/trusted-build-inputs.md`, which also carries the refresh procedure.