Acceptance record: Ansible slice 1 (preflight, base_packages, maintenance_jobs)
Current status: this page preserves the scope and evidence of slice 1. All later roles were subsequently delivered; see
acceptance-slice2.mdandacceptance-slice3.md.homepage_configwas later retired when Grafana replaced Homepage.
This records the proofs for the first vertical slice of the Ansible
host-convergence re-platform (the behavior contract). It implements three of
the twelve roles from the behavior contract:
preflight (PRE-1..5), base_packages (PKG-1..4), and maintenance_jobs
(JOB-1..6). The other nine roles remain deliverable-3 stubs that fail
"not implemented".
Per the contract's proof strategy, every behavior carries two independent proofs. This file keeps them strictly separated:
- VM-simulation proofs: automatable on the disposable Debian 13 VM; these are green now (evidence below).
- PROD-validation proofs: provable only against the real host at the single scheduled cutover (real mergerfs/SnapRAID pool, real parity disk, real containers). These are deferred and listed so the cutover has a checklist.
The VM cannot reproduce the real pool, so the mount preflight is proven with
present-vs-absent / wrong-FSTYPE stand-ins (loop + bind + a real mergerfs
union), never the real disks. install.sh stayed frozen as the rollback and was
not edited (retired 2026-07-14 once every proof closed).
Rehearsal harness
ansible/tests/vm/ is an Ansible-native harness with no Molecule or Galaxy
dependency:
fixtures.sh(runs on the VM as root): seeds the stand-ins.mkfs/mergerfslive only here, never under theansible/role tree, so the destructive-storage guard stays green./mnt/user: a realfuse.mergerfsunion (good) or a bind mount (wrong FSTYPE) or unmounted (absent)./mnt/cache: a btrfs loop (good) or an ext4 loop (wrong FSTYPE) or unmounted (absent).- three stub
.envfiles (present/absent),0600owned byhomelab_user. rehearse.sh(runs on the workstation): rsyncs the worktree toVM:/opt/homelab,apt installsansible-core(the documented bootstrap prereq) + fixture deps, then drives each scenario exactly the way cutover runs: on the box,sudo ansible-playbook -c local playbooks/host.yml --tags .... Its private-site safety rail refuses the production host.
Reproduce: cd ansible/tests/vm && VM=<operator>@<vm-address> ./rehearse.sh all.
VM-simulation proofs - PASSING
Rehearsal run on a fresh Debian 13 (trixie) VM, systemd 257, ansible-core 2.19.4 (apt), local connection.
Negative tests (fail-closed contract) - all abort as required
| Contract | Precondition seeded | Result |
|---|---|---|
| PRE-1 | /mnt/user bind mount (wrong FSTYPE) |
converge aborts ✓ |
| PRE-1 | /mnt/user unmounted (absent) |
converge aborts ✓ |
| PRE-2 | /mnt/cache ext4 loop (wrong FSTYPE) |
converge aborts ✓ |
| PRE-2 | /mnt/cache unmounted (absent) |
converge aborts ✓ |
| PRE-3 | one .env removed |
converge aborts ✓ |
| PRE-4 | repo copied to /tmp/hbp (not /opt/homelab) |
converge aborts ✓ |
| PRE-5 | required group overridden to a non-existent name | converge aborts ✓ |
Positive convergence + idempotence
| Proof | Evidence |
|---|---|
| First converge (good stand-ins) succeeds | play returns rc 0 |
| Idempotence: second converge | ok=18 changed=0 failed=0 |
Check-mode gate - --check on a healthy host |
play returns rc 0 (preflight read-only probes carry check_mode: false so the asserts have data under --check) |
--tags maintenance_jobs selects only that role |
only JOB-* tasks run; no PRE-* |
PRE / PKG / JOB state after converge
| Contract | Evidence on VM |
|---|---|
| PKG-1..3 | hd-idle 1.21, jq, restic, fail2ban, smartmontools 7.4, btrfs-progs 6.14, curl all ii (installed); binaries resolve (/usr/sbin/hd-idle, /usr/sbin/smartctl) |
| PKG-4 | ansible-core 2.19.4-0+deb13u1 present from apt; role asserts, never installs |
| JOB-1 | 6 service + 6 timer units at /etc/systemd/system, mode 644 |
| JOB-2 | all six timers is-enabled == enabled |
| JOB-3 | scripts/*.sh mode 755 |
| JOB-4 | /etc/default/hd-idle content matches repo; hd-idle.service ActiveState=active, is-enabled == enabled |
| JOB-5 | smartmontools.service is-enabled == masked |
| JOB-6 | homelab-watchdog.timer carries OnCalendar=*:0/5 |
Reboot persistence (no re-converge)
After systemctl reboot (verified by a changed /proc/sys/kernel/random/boot_id
so a rejected reboot cannot yield a false pass), re-checked enablement only:
- all six maintenance timers still
enabled✓ smartmontools.servicestillmasked✓
Full rehearse.sh all: 28 checks passed, 0 failed.
PROD-validation proofs - 2026-07-13 production cutover and 2026-07-14 next-day checks
The mutating converge ran once on the production host (changed=4, failed=0, all
four changes matching the dry-run classification), followed by an idempotence
re-run (changed=0).
| Contract | PROD proof | Result (2026-07-13 cutover, 2026-07-14 next-day) |
|---|---|---|
| PRE-1 | findmnt /mnt/user == fuse.mergerfs on the real pool |
✅ fuse.mergerfs |
| PRE-2 | /mnt/cache real btrfs present |
✅ btrfs |
| PRE-3 | the three real escrowed .env files present |
✅ all three present, 0600 homelab_user:homelab_user |
| PRE-5 | real homelab_user in real docker group |
✅ in docker group; docker ps works without sudo |
| PKG-1..3 | package set already present (converge a no-op) | ✅ no package tasks changed |
| JOB-4 | configured parity disk actually parks - the VM has no such disk, so only config content is VM-proven | ✅ found broken, fixed, and an unattended park was observed. The root cause was a bare by-id name passed to hd-idle -a, which matches nothing; the corrected full by-id path parks parity after the configured idle timeout, and the collector reports standby without waking it |
| JOB-5 | masked smartmontools keeps the parked parity asleep (no SMART-poll wake) | ✅ smartmontools.service masked; with parity manually parked on 2026-07-14 nothing woke it (hdparm -C stayed standby across the hourly collector run) |
| JOB-6 | watchdog fires every 5 min against the real mergerfs/SAB/qbit checks | ✅ 6 runs in the 30 min after the converge |
Out of scope for this slice
The later roles were delivered in slices 2 and 3. The backup timer issue below was resolved in slice 2 and remains here as historical evidence.
Known cross-role follow-up (backup timer ordering)
JOB-2 enables --now all six timers, including appdata-backup.timer and
restic-deep-check.timer, which are Persistent=true. On a converge where the
scheduled OnCalendar moment has already passed, enable --now triggers an
immediate catch-up run of the backup service. Because maintenance_jobs runs
before backup in host.yml, the backup can fire before the backup role
initialises RESTIC_CACHE_DIR and the restic repos (BAK-1/BAK-2) - the reverse
of install.sh, which inits restic before enabling the timers. The backup
slice must resolve this ordering (init restic before the backup timers' first
activation, e.g. via role ordering or by having backup own their activation).
Observed on the VM: the catch-up run happened and exited 0 only because the stub
.env carries no restic creds; on a real fresh host it would fail or race.
Resolved in slice 2:
host.ymlnow ordersbackupbeforemaintenance_jobs, so the restic cache dir and repos are initialised (or the init knowingly skipped without creds) before JOB-2'senable --nowcan catch-up-fire the backup timers - matchinginstall.sh's original ordering. Seeacceptance-slice2.md.