feat(port): CARGO_CRATES drift check vs Cargo.lock + CI gate #111

Merged
clawdie merged 1 commit from feat/port-cargo-crates-drift-check into main 2026-06-20 17:53:18 +02:00
4 changed files with 90 additions and 16 deletions

View file

@ -39,3 +39,11 @@ jobs:
- uses: actions/checkout@v4
- name: Markdown format gate
run: ./scripts/check-format.sh
port:
runs-on: ubuntu-latest
container: python:3.12
steps:
- uses: actions/checkout@v4
- name: FreeBSD port CARGO_CRATES in sync with Cargo.lock
run: ./packaging/freebsd/port/check-cargo-crates.sh

View file

@ -13,17 +13,21 @@ sysutils/colibri/
└── pkg-plist installed file list
```
## What's generated on the build host (NOT committed)
## Generated content
Two files are derived and must be generated in the ports tree before building —
they are not hand-edited and not stored here:
- **`CARGO_CRATES`** (committed, in the `Makefile`) — the crates.io dependency
closure from `Cargo.lock` (#109). Regenerate with `make cargo-crates` after any
dependency change. **`check-cargo-crates.sh`** verifies it stays in sync with
`Cargo.lock` (no network, any host) and runs in CI, so drift fails the build:
- **`distinfo`** — checksums of the source tarball + every crate distfile.
Generate with `make makesum`.
- **`CARGO_CRATES`** — the full crate list from `Cargo.lock` (hundreds of lines).
Generate with `make cargo-crates` and paste the block into the `Makefile`
(replacing the empty placeholder). Without it, the clean-jail build cannot
fetch crates offline.
```sh
./packaging/freebsd/port/check-cargo-crates.sh
```
- **`distinfo`** (NOT committed) — checksums of the source tarball + every crate
distfile. Generated on the build host with `make makesum`; it is intentionally
not hand-authored (the Forgejo source tarball hash must come from the host that
fetches it).
## Build it
@ -38,9 +42,7 @@ they are not hand-edited and not stored here:
cp -R sysutils/colibri \
/usr/local/poudriere/ports/clawdie/sysutils/colibri
cd /usr/local/poudriere/ports/clawdie/sysutils/colibri
make makesum # -> distinfo
make cargo-crates # -> paste the CARGO_CRATES block into the Makefile
make makesum # re-run now that crates are listed
make makesum # -> distinfo (CARGO_CRATES is already in the Makefile)
```
3. **Build + sign** via the wrapper:

View file

@ -0,0 +1,62 @@
#!/bin/sh
# Verify the port Makefile's CARGO_CRATES matches the crates.io dependency
# closure in Cargo.lock. Run after any dependency change (and in CI): if cargo
# adds/removes/bumps a registry dep, CARGO_CRATES must be regenerated with
# `make cargo-crates` or the offline poudriere build fetches the wrong set.
#
# Exit: 0 = in sync, 1 = drift (prints the delta), 2 = usage / IO error.
# Requires python3 >= 3.11 (tomllib). No network, runs on any host.
set -u
_here=$(cd "$(dirname "$0")" && pwd)
LOCK="${1:-${_here}/../../../Cargo.lock}"
MK="${2:-${_here}/sysutils/colibri/Makefile}"
[ -f "$LOCK" ] || { echo "ERROR: Cargo.lock not found: $LOCK" >&2; exit 2; }
[ -f "$MK" ] || { echo "ERROR: Makefile not found: $MK" >&2; exit 2; }
command -v python3 >/dev/null 2>&1 || { echo "ERROR: python3 required" >&2; exit 2; }
python3 - "$LOCK" "$MK" <<'PY'
import sys, tomllib
lock_path, mk_path = sys.argv[1], sys.argv[2]
# Expected: every package sourced from the crates.io registry. Workspace-local
# crates (no source) and git deps (handled by CARGO_GIT_*) are not CARGO_CRATES.
with open(lock_path, "rb") as fh:
want = {
f"{p['name']}-{p['version']}"
for p in tomllib.load(fh)["package"]
if str(p.get("source", "")).startswith("registry+")
}
# Parse the CARGO_CRATES= assignment block (line-continued with trailing '\').
have, in_block = set(), False
for line in open(mk_path, encoding="utf-8"):
if line.startswith("CARGO_CRATES="):
in_block, line = True, line[len("CARGO_CRATES="):]
elif not in_block:
continue
cont = line.rstrip("\n").endswith("\\")
have.update(line.replace("\\", "").split())
if not cont:
break
missing = sorted(want - have) # in Cargo.lock, absent from Makefile
extra = sorted(have - want) # in Makefile, absent from Cargo.lock
if not missing and not extra:
print(f"OK: CARGO_CRATES in sync with Cargo.lock ({len(want)} crates).")
sys.exit(0)
if missing:
print(f"MISSING from Makefile ({len(missing)}): regenerate with `make cargo-crates`")
for c in missing:
print(f" + {c}")
if extra:
print(f"STALE in Makefile ({len(extra)}): no longer in Cargo.lock")
for c in extra:
print(f" - {c}")
sys.exit(1)
PY

View file

@ -17,10 +17,12 @@ MASTER_SITES= https://code.smilepowered.org/clawdie/colibri/archive/
DISTFILES= ${DISTVERSIONFULL}${EXTRACT_SUFX}
WRKSRC= ${WRKDIR}/colibri
# CARGO_CRATES is generated from Cargo.lock. Regenerate on the build host before
# the first real build (the workspace pulls in hundreds of crates):
# make cargo-crates >> Makefile # then move the generated block here
# Empty in this draft, so `make makesum` only sums the main tarball.
# CARGO_CRATES — the crates.io dependency closure, generated from Cargo.lock.
# Regenerate after any dependency change and verify it stays in sync:
# make cargo-crates # on a FreeBSD build host
# packaging/freebsd/port/check-cargo-crates.sh # drift check (any host, CI)
# distinfo (source tarball + crate checksums) is produced separately by
# `make makesum` on the build host.
CARGO_CRATES= ahash-0.8.12 \
aho-corasick-1.1.4 \
allocator-api2-0.2.21 \