layered-soul/skills/zfs-snapshot-audit/SKILL.md

---
name: zfs-snapshot-audit
description: Audit ZFS snapshots and sanoid config — find orphaned/leaked snapshots, verify dataset coverage, safely destroy dead weight.
category: freebsd
---

# ZFS Snapshot Audit & Sanoid Coverage

Use this when disk space is tight and you suspect ZFS snapshots are holding dead weight, or when you need to verify sanoid coverage across datasets.

## 1. Quick check — pool and dataset overview

```bash
zpool list zroot
zfs get -H -o property,value used,refer,usedbysnapshots,available zroot/home/clawdie
df -h /
```

If `usedbysnapshots` is high (>1G), investigate.

## 2. List all snapshots with size

```bash
zfs list -t snapshot -o name,used,creation -S creation -r zroot 2>/dev/null
```

Look for snapshots from dates that exceed the sanoid retention policy.

## 3. Check sanoid coverage

Config lives at `/usr/local/etc/sanoid/sanoid.conf`. Every dataset that has
`autosnap` snapshots should have a matching `[dataset]` entry with a template.
Without one, `sanoid --prune-snapshots` won't touch them — they accumulate
forever.

```bash
# List all datasets with snapshots
zfs list -t snapshot -r zroot 2>/dev/null | awk -F@ '{print $1}' | sort -u

# Cross-reference against sanoid config
grep '^\[' /usr/local/etc/sanoid/sanoid.conf
```

Any dataset with snapshots but no sanoid entry = **orphaned**.

## 4. Available templates (sanoid.conf)

| Template | hourly | daily | monthly | Use case |
|----------|--------|-------|---------|----------|
| `operator_home_minimal` | 6 | 3 | 0 | Operator home dir |
| `operator_home_full` | 24 | 14 | 3 | Full home retention |
| `persistent_service` | 12 | 7 | 2 | Jails (cms, git, etc.) |
| `critical_data` | 24 | 14 | 3 | Databases (pgdata, pgwal) |

## 5. Destroy orphaned snapshots

**Safe:** destroying ZFS snapshots does NOT touch the live filesystem. Only the
unique blocks held by that point-in-time copy are freed. Verify first:

```bash
# Confirm: usedbysnapshots = dead weight, referenced = live data
zfs get -H -o property,value used,refer,usedbysnapshots zroot/home/clawdie
```

Destroy individual snapshots:

```bash
zfs destroy zroot/home/clawdie@autosnap_2026-04-20_00:15:00_daily
```

Or batch by pattern (FreeBSD 15 ZFS supports `%` glob):

```bash
zfs destroy zroot/home/clawdie@autosnap_%
```

## 8. Disk-pressure after large deletions (snapshot vacuum)

When you delete a large directory (e.g. `cargo clean` freeing 5.5G of
`target/`), the space is NOT freed if sanoid-managed snapshots captured the
files. df shows no change. The deleted blocks are locked in the snapshot chain
until every snapshot that captured them is rotated out.

**Symptom:** `df` unchanged after `cargo clean` or `rm -rf large-dir`.

**Check:**

```bash
zfs get -H -o value usedbysnapshots zroot/home/clawdie
# If high (>5G) and you just did a large deletion → vacuum needed
zfs list -t snapshot -o name,used,creation -r zroot/home/clawdie
# Find the snapshots taken during/after the files were created
```

**Fix — reclaim space immediately:**

```bash
# Destroy ALL snapshots on the dataset (aggressive, zero history retained):
sudo zfs destroy zroot/home/clawdie@autosnap_%

# Or: destroy specific snapshots that captured pre-deletion state:
sudo zfs destroy zroot/home/clawdie@autosnap_2026-06-24_14:00:00_hourly
# Repeat for each snapshot, then verify:
zfs get -H -o value usedbysnapshots zroot/home/clawdie
# Should approach 0B
```

**After reclaim:**

```bash
df -h /home/clawdie         # free space should jump
zpool list zroot            # pool capacity drops
```

Sanoid will begin taking fresh snapshots on its next cron tick.

**Prevent next time:** after any large deletion, run `sudo sanoid --prune-snapshots`
to immediately rotate out hourlies that captured the deleted data, without
losing the daily safety net.

## 9. Add missing dataset to sanoid config

Append to `/usr/local/etc/sanoid/sanoid.conf` (root-owned, use `sudo tee -a`):

```
[zroot/home/clawdie]
        use_template = operator_home_minimal
```

Verify it takes effect (next `sanoid --prune-snapshots` cron run, or manually):

```bash
sudo sanoid --prune-snapshots --verbose 2>&1 | grep home/clawdie
```

## 10. Verify after cleanup

```bash
zfs get -H -o value usedbysnapshots zroot/home/clawdie
# Should be 0B (or low if new autosnaps have been taken)
zpool list zroot
df -h /
```

## Pitfalls

- **Prefer sanoid prune over manual destroy.** If disk pressure is not urgent,
  run `sudo sanoid --prune-snapshots` and let retention policy rotate out old
  snapshots. This preserves the daily safety net.
- **Exception: snapshot vacuum after large deletions.** When you need the space
  NOW (e.g. `cargo clean` freed 5.5G but df shows no change), manual destroy
  of sanoid-managed snapshots is warranted. See §8 above. Destroy all snapshots
  on the target dataset, then let sanoid rebuild them.
- **Don't destroy snapshots on datasets with `autoprune=yes`** during normal
  operations — you'll fight the cron job. This is only for the vacuum case.
- The config header says "do not edit by hand" but the dataset list is the
  operator's domain — adding a dataset is safe.

## Discovery log

2026-06-24: Hit "snapshot vacuum" — `cargo clean` freed 5.5G but df showed 16G
unchanged. `usedbysnapshots` = 26.6G. Sanoid's hourly snapshots (14:00-20:00)
captured the target/ directory before deletion. Destroyed a May boot environment
(3.5G), old checkpoints (14M), pre-reinstall snaps (3.2M), then all 9 sanoid
hourly+daily snaps on home/clawdie. Final: 16G → 29G free, pool 80% → 72%.
Lesson: after any large deletion, either `sanoid --prune-snapshots` to rotate
immediately, or manual destroy if desperate. Added §8 to this skill.

2026-06-22: `zroot/home/clawdie` was missing from sanoid config. 10 orphaned
snapshots from April 20-22 held 23.6G of dead weight (`usedbysnapshots`).
Added `operator_home_minimal` template and destroyed all 10. Freed 23.5G.