Acceptance record: Ansible slice 3 (docker_host, hardening, ssh_hardening)

This records the proofs for the final host-convergence slice of the Ansible re-platform (the behavior contract). It implements the last three roles from the behavior contract: docker_host (DOCKER-1..6), hardening (HARD-1..3), and ssh_hardening (SSH-1..2). All twelve host.yml roles were implemented and VM-rehearsed in this run.

Status update (2026-07-13): the previously rehearsed homepage_config role was later retired when Grafana replaced Homepage as the service launcher. The current play has eleven roles; the 12-role counts below remain the historical evidence for this rehearsal.

Per the contract's proof strategy, the evidence remains strictly separated:

• VM-simulation proofs - automatable on the disposable Debian 13 VM; all green below.
• PROD-validation proofs - require the real mounts, containers, HBA, and a fresh external SSH login; deferred to the single scheduled cutover.

Nothing in this slice ran on production. install.sh stayed frozen and unchanged as the rollback until the cutover was accepted (retired 2026-07-14).

Safety design

Docker restart transaction

The role validates the repository daemon.json, semantically compares its JSON directives, and normalizes the effective systemd drop-in directives before any mutation. It then queries Docker and its running containers. A real effective change on an active daemon with containers aborts before any role mutation unless allow_docker_restart=true was explicitly passed. Equivalent JSON ordering or whitespace does not schedule a restart.

After the gate passes, handlers reload systemd before restarting Docker. A fresh daemon with no containers needs no flag. The shared proxy network is managed through the Docker CLI, not community.docker.

SSH candidate transaction

ssh_hardening remains last in host.yml. It validates the current combined configuration before treating it as last-known-good, stages the candidate outside sshd_config.d, runs sshd -t against the candidate, installs it, then validates the full combined configuration. Only that successful path can reload ssh.service.

The .lkg copy is outside Debian's *.conf include glob. Any candidate or combined-config failure restores the prior drop-in, validates the restoration, fails the role, and leaves SSH unreloaded.

Rehearsal harness extensions

ansible/tests/vm/ remains Ansible-native, with no Galaxy collection or Molecule dependency:

• fixtures.sh now seeds Docker fresh/equivalent/drift states, a pinned BusyBox container, a deliberately invalid sshd candidate, and real systemd .mount units for /mnt/user/data and /mnt/cache/appdata.
• The negative mount unit uses an existing but invalid ext4 loop source, so it fails immediately and deterministically. The positive units use real bind mounts. docker.socket is stopped while the dependency fixture changes so socket activation cannot race the proof.
• rehearse.sh installs the VM-only Docker/OpenSSH prerequisites, detects SSH on port 22 or the configured hardened port, refuses the production host, and recovers correctly after a prior rehearsal reboot removes the disposable loop mounts.
• Mount setup now has postconditions and avoids remounting an already-correct stand-in. This prevents asynchronous loop teardown from creating false preflight failures.

Reproduce from ansible/tests/vm:

./rehearse.sh all

The default target is homelab_user@<production-address>; override it with VM= and SSH_PORT=. the configured SSH port must be reachable after the SSH role converges and after the reboot proof.

VM-simulation proofs - ALL PASSING

Full rehearsal on Debian 13 (trixie), systemd 257, ansible-core 2.19.4, and Docker 26.1.5: 73 checks passed, 0 failed.

Negative and disruptive-path proofs

Contract	Precondition	Result
DOCKER-2	No daemon.json/drop-in and no containers	fresh converge succeeds without the restart flag
DOCKER-1/2	Reordered/reformatted but equivalent daemon.json; container running	changed=0; dockerd PID unchanged
DOCKER-2	Real valid directive drift; container running; no flag	aborts before mutation; daemon.json hash and dockerd PID unchanged
DOCKER-2	Same real drift with allow_docker_restart=true	restart succeeds; dockerd PID changes; BusyBox container remains running through live-restore
DOCKER-3	Real required .mount unit backed by an invalid ext4 loop source	mount unit fails and docker.service refuses to start
DOCKER-3	Both required .mount units backed by valid bind sources	both mount and Docker starts
SSH-2	Syntactically invalid staged candidate	sshd -t rejects it; role aborts; active drop-in hash is unchanged; active and .lkg copies match; SSH stays valid

The ten PRE/STO/NET negative tests from slices 1 and 2 also remain green in the same run.

Positive convergence and universal gates

Proof	Evidence
First full converge	succeeds with all twelve roles
Idempotence	ok=72 changed=0 failed=0 skipped=16
Check mode	full tagged ansible-playbook --check succeeds on the healthy host
Concern tagging	--tags maintenance_jobs selects only that role
Destructive-storage guard	clean across all playbooks and role task files

Per-role state

Contract	Evidence on VM
DOCKER-1	effective JSON equals the repository candidate; file 0644 root:root; Docker reports live-restore enabled
DOCKER-3	drop-in 0644 root:root; systemctl cat docker includes both required paths
DOCKER-4	wireguard loaded; /etc/modules-load.d/wireguard.conf matches the repository
DOCKER-5	docker network inspect proxy succeeds
DOCKER-6	apt drop-in 0644 root:root; apt-config dump parses the Docker CE origin
HARD-1	udev rule matches; every exposed policy is max_performance; the VM's empty SATA-policy set no-ops cleanly
HARD-2	fail2ban jail matches; service enabled and active
HARD-3	homelab_user is in docker; second converge is unchanged
SSH-1	drop-in matches all six directives; sshd -T reflects the configured SSH port and key-only/no-root state
SSH-2	active drop-in equals its persistent .lkg; sshd -t succeeds; a fresh key-based connection on the configured port succeeds

Reboot persistence

The harness verifies a changed boot ID before accepting any post-reboot proof. Without another converge:

• all six maintenance and five monitoring timers remain enabled;
• smartmontools.service remains masked;
• WireGuard is loaded through modules-load.d;
• fail2ban remains enabled and active; and
• sshd remains valid and reachable on the configured SSH port.

PROD-validation proofs - CONFIRMED at the 2026-07-13 production cutover

Contract	Production proof	Cutover result (2026-07-13)
DOCKER-1	docker info reports live-restore enabled and effective json-file rotation matches 50m / 3	✅ Live Restore Enabled: true, json-file driver, daemon.json carries 50m/3
DOCKER-2	Run the one cutover with -e allow_docker_restart=true; Docker restarts exactly once and the real container fleet recovers cleanly	✅ flag passed; the effective daemon.json already matched, so zero restarts fired and the fleet (40 containers) was untouched - the no-op branch is the correct behaviour when there is no config diff
DOCKER-3	systemctl cat docker shows the dependency; a genuinely unavailable data/cache mount keeps Docker down by design	✅ RequiresMountsFor=/mnt/user/data /mnt/cache/appdata present (fail-closed path not live-fired)
DOCKER-4	WireGuard is loaded and qBittorrent VPN starts unprivileged	✅ wg0 has a private tunnel address, WebUI answering 200
DOCKER-5	proxy exists and all three Compose projects can attach	✅ proxy network present; all three projects converged against it
DOCKER-6	Docker CE packages remain eligible for unattended upgrades	✅ 51unattended-upgrades-docker origin Docker,label=Docker CE in place
HARD-1	all eight real SATA policies are max_performance; hard-reset log count stays flat	✅ 8× max_performance; 3 hard resetting link lines this boot, all from 2026-07-09 pre-fix, none since
HARD-2	fail2ban is active and observes SSH on the configured SSH port	✅ sshd jail enabled with the configured port, responding to fail2ban-client
HARD-3	a fresh homelab_user login has non-root Docker access	✅ fresh session ran docker ps without sudo
SSH-1	existing cutover session survives; effective config is key-only, no-root, the configured SSH port	✅ session survived; all six hardening directives asserted effective by the role
SSH-2	before closing the old session, a fresh key-based login on the configured port succeeds	✅ fresh a fresh SSH command login + sudo -n true verified before proceeding; .lkg copy recorded (one of the four expected changes)

The deferred proofs from acceptance-slice1.md and acceptance-slice2.md remain part of the same cutover checklist. The separate Compose wrapper in playbooks/deploy.yml was out of scope for this slice (delivered later; see acceptance-deploy.md); Compose remains authoritative for workload lifecycle.