--- name: freebsd-os-upgrade description: Minor (same-major) FreeBSD upgrade runbook for hive nodes — pkgbase or freebsd-update, reboot-needed detection, Bastille thin/thick jail upgrade, pre/post verification, and the clawdie-iso FREEBSD_VERSION bump. --- # FreeBSD OS Upgrade (minor / point release) How we move a hive node across a FreeBSD point release within the same major (e.g. `15.0-RELEASE` → `15.1-RELEASE`). Same-major upgrades are low-risk: the package ABI is unchanged, so no package rebuild and no PostgreSQL dump/restore are required. The detailed reboot rules and verification live in [`references/freebsd-update-reboot.md`](references/freebsd-update-reboot.md); this is the procedure that wraps them. A host manages its base system one of two **mutually exclusive** ways — detect which before upgrading: - **pkgbase** — base installed via `pkg` (you'll see `FreeBSD-*` packages like `FreeBSD-kernel-generic`). Upgrade with `pkg`. This is OSA's method. - **freebsd-update** — binary base updates via `freebsd-update(8)`. Detect: `pkg info -e FreeBSD-runtime && echo pkgbase || echo freebsd-update`. Reboot detection, verification, and the clawdie-iso side are identical for both; only the "fetch + install the new base" step differs. ## Quick reference Run the privileged steps as root, or via the host's escalation — `mdo` on the operator image, `sudo`/`doas` elsewhere. ```sh # 0. Which base-management method? (mutually exclusive) pkg info -e FreeBSD-runtime && echo "pkgbase" || echo "freebsd-update" # 1. Detect installed vs running kernel (both methods) freebsd-version -k # installed kernel freebsd-version -u # installed userland uname -r # running kernel # 2a. pkgbase (base via pkg, e.g. FreeBSD-kernel-generic): # INSPECT the existing base repo first — a pkgbase host already has one: pkg -vv | grep -A6 -i 'FreeBSD-base' grep -rn 'base_release\|base_latest\|FreeBSD-base' /etc/pkg /usr/local/etc/pkg/repos/ # EDIT that existing entry in place (do NOT append a second FreeBSD-base # block — duplicate repo names give undefined, last-wins behavior). A pinned # base_release_0 only delivers 15.0 patch levels; change it to base_release_1 # (or base_latest) to cross to 15.1. If it's already base_latest, skip. pkg update pkg upgrade -n # DRY RUN first — preview the 15.1 base move, applies nothing pkg upgrade # apply once the plan looks right (base + ports together) # 2b. freebsd-update (binary base updates): freebsd-update -r 15.1-RELEASE upgrade freebsd-update install # stages new kernel; run again after reboot pkg update -f && pkg upgrade # ports packages (separate from base here) # 3. Reboot ONLY on operator go-ahead — a new kernel is staged until reboot. # Same major: ABI FreeBSD:15:amd64 unchanged, no rebuild / no PG dump-restore. ``` ## When to use - A new FreeBSD point release in the current major is available and a node should track it (OSA / mother, build host, deployed hosts). - Before building a clawdie-iso image for the new point release (build the image on a host at the same series). ## Runbook 1. **Capture pre-status** for after-the-fact comparison — see *Pre-reboot status capture* in the reference (hostname, `freebsd-version -k` / `-u`, `uname -r`, services, `jls`, `pfctl -s info`). Record permission-limited checks as such, not as "down". 2. **Upgrade base** (by the method from step 0): - **pkgbase**: a pkgbase host already has a `FreeBSD-base` repo — **inspect it** (`pkg -vv | grep -A6 -i FreeBSD-base`) and **edit that existing entry in place**. A pinned `base_release_0` only delivers 15.0 patch levels; point it at `base_release_` / `base_latest` to cross to the new release. Do **not** append a second `FreeBSD-base` block — duplicate repo names give undefined, last-wins behavior. Then `pkg update`, **dry-run** with `pkg upgrade -n` to confirm 15.1 base packages are actually offered, then `pkg upgrade` (base + ports together). - **freebsd-update**: `freebsd-update -r upgrade` then `freebsd-update install`. Either way the new kernel is staged; the system runs the old one until reboot. 3. **Confirm a reboot is needed**: `freebsd-version -k` newer than `uname -r` means staged-not-active. State that plainly and **reboot only on explicit operator go-ahead** — never reboot the always-on board host autonomously. 4. **After reboot**: on freebsd-update hosts, run `freebsd-update install` again to finish userland. Then the *Post-reboot verification* block — `-k`/`-u`/ `uname -r` must all match, and the app-readiness checks (Clawdie control plane, Forgejo, jails, PF, Tailscale) must pass. 5. **Packages**: same-major ABI (`FreeBSD:15:amd64`) is unchanged, so this is a freshness refresh, not a rebuild — pkgbase already covered it in step 2; freebsd-update hosts do `pkg update -f && pkg upgrade`. A same-major PostgreSQL bump needs no dump/restore (restart/reboot to load new binaries). 6. **Upgrade the jails** — the host upgrade does NOT touch them. Do this after the host is on the new kernel. See *Jails* below. 7. **Vulnerability audit**: if `pkg audit` still flags packages (host or jails), do not imply the upgrade failed — the upgrade completed; unrelated packages remain vulnerable until fixed versions land. (Wording in the reference.) ## Jails Jails carry their **own userland** — a host base upgrade leaves them on the old release. Upgrade them as part of the same process, **after** the host is on the new kernel (jails run on the host kernel; a same-major userland mismatch is tolerated, but move them up for consistency + security). OSA uses **Bastille** (`/usr/local/bastille/jails/`). - **Thick jail** — a full, independent base copy. Upgrade each on its own. - **Thin jail** — a clone/overlay of a bootstrapped release template. Bootstrap the new release once, then bring each thin jail up off it. Each jail's base is managed pkgbase or freebsd-update. For a **thick** jail (independent base) detect it directly: `bastille cmd pkg info -e FreeBSD-runtime` (present = pkgbase). On a **thin** jail this may be empty or error — a thin jail has no independent pkg-managed base; its method follows the release **template** it was bootstrapped from, and you upgrade it at the template level (re-bootstrap / re-clone or `bastille upgrade`), not per-jail. Bastille flow (confirm against the installed Bastille version + bootstrap method): ```sh bastille list # jails, thin/thick # freebsd-update-managed jails: bastille bootstrap 15.1-RELEASE # new release template (for thin) bastille upgrade 15.1-RELEASE # pkgbase-managed jail: repoint its base repo (edit-existing, not append), # then bastille pkg upgrade bastille cmd freebsd-version # verify each jail moved to 15.1 ``` Same-major ABI (`FreeBSD:15:amd64`) is unchanged, so packages inside jails need no rebuild — `bastille pkg upgrade` is a freshness refresh. Restart each jail (or its services) so new binaries load, then re-check `jls` and per-jail service health from the reference's *Post-reboot verification*. ## clawdie-iso image side The operator image tracks the series through a single variable. To build for the new point release: ```sh # build.cfg derives the memstick URL, checksum URL, cache path, and # build-manifest from FREEBSD_VERSION: FREEBSD_VERSION="${FREEBSD_VERSION:-15.1-RELEASE}" # or override per-build without editing git: FREEBSD_VERSION=15.1-RELEASE ./build.sh ``` Docs are kept version-agnostic (`FreeBSD 15.x`) so they don't drift on point bumps. `build-vps.sh` (mfsbsd) and `scripts/poudriere/poudriere-setup.sh` carry their own version knobs — bump those separately if that path is in use. ## Sequence for a release Upgrade the build host (OSA) first → refresh its package cache → then build the image for the new point release, so the image is assembled on a host at the same series. ## Validation evidence - **OSA** uses **pkgbase** (`FreeBSD-kernel-generic 15.0p10`). Pre-status clean: `freebsd-version -k`, `-u`, and `uname -r` all matched — no pending reboot. _Pending: `15.0 → 15.1` post-upgrade capture () — confirm the base repo targets 15.1; fold in host pre/post `freebsd-version -k`/`-u` + `uname -r`, services, PF, and **per-jail** `bastille cmd freebsd-version` after each jail is upgraded (record thin/thick + bootstrap method per jail)._