1. docker events was leaking zombies.
docker_events_loop spawned the child, read its stdout to EOF and returned without
calling wait(). Dropping a Child does not reap it, so every reconnect - which happens
whenever Docker restarts or the stream ends - left a defunct process behind for the
lifetime of the daemon. The reader loop moved into its own function so the wait happens
on every exit path, and the exit status is logged rather than discarded.
2. sysctl status still decided ownership with contains().
The destructive paths were fixed earlier, but inspect_sysctl_tuning kept
first.contains(MANAGED_MARKER) in the disabled branch, so a file whose first line merely
embedded the marker was reported as managed. Diagnostic-only, but it contradicted the
rule the rest of the module follows. Now an exact match on the trimmed first line.
3. Swap was reconciled at the traffic interval.
ensure_swap ran every reconcile_interval_secs, default 15, and each run spawns
swapon --show, sysctl reads, the persistence check and the final verify. A swapfile, its
fstab entry and vm.swappiness change when an operator changes them and at no other time,
so this was process spawning for state that does not drift. Swap now runs on its own
300-second schedule; traffic and firewall keep the configured interval. The startup
reconcile is unchanged, so a fresh daemon still converges immediately.
4. The event log recorded routine success.
Every successful cycle appended a line, and appending means reading the whole file,
rewriting it and truncating. At 15 seconds that is roughly 17k entries a day, and the
entries worth reading were buried among them. Only transitions are written now: a new
failure, a failure whose message changed, and the recovery after one. set_ok and set_err
report whether the state actually changed, which is what makes that possible. journald
still has the full per-cycle picture through the tracing output.
5. Swapfile creation is one command.
mkswap --file <path> --size <N>M creates and formats the file, replacing the
fallocate → dd fallback → chmod → mkswap chain: four processes, three failure branches,
and a window where the file existed but was not yet formatted. util-linux on Debian 13
also handles filesystem-specific requirements - Btrfs in particular - that raw fallocate
does not. fallocate, dd and chmod drop out of the required host tools when swap is
enabled.
cargo test --workspace: 183 + 4 passed. clippy -D warnings clean.
1. The state directory was created too late and never verified.
ensure_state_dir only ran when the ownership record was written, well after
assert_swap_path_is_safe had already required every parent of swap.path to exist. On a
clean host that meant a perfectly legal /var/lib/awg2-harness/swapfile was rejected
because the directory this package is supposed to create did not exist yet.
Worse, when the directory did exist the check only confirmed it was a directory. A
pre-existing /var/lib/awg2-harness that was group- or world-writable defeated the whole
point of restricting swap.path to root-owned roots: another user could still swap the
swapfile out from under a root-owned fallocate or mkswap.
ensure_state_dir now runs first in ensure_swap and asserts what the restriction actually
depends on: the parent is a real directory and not a symlink, the state directory itself
is not a symlink, it is owned by uid 0, and it is not writable by group or others. A
mode that is merely narrower than 0700 is tightened rather than rejected; anything
group- or world-writable is a hard error, because the safety argument for the whole
allow-list rests on it.
The units also declare StateDirectory=awg2-harness with mode 0700 and
RuntimeDirectory=awg2-harness, so in the common case systemd creates and owns both and
the binary only has to verify. The runtime checks stay regardless: the CLI subcommands
run outside the service lifecycle.
2. The fstab copies could be more permissive than the file they mirror.
The backup and pristine copies were written at a hardcoded 0644 with a comment claiming
they must not be wider than the original. On a host where /etc/fstab is 0600 they were,
and the rollback then handed that wider mode to the live file too. FstabTransaction now
captures the live mode when it opens and uses it for the backup, the pristine copy, the
candidate and the rollback.
One test added, unix-only since it asserts a mode: a 0600 fstab produces 0600 copies and
stays 0600 after the commit.
cargo test --workspace: 180 + 4 passed. clippy -D warnings clean. Unit files not verified
here - no systemd on this host; tools/build.sh runs systemd-analyze verify on the build
machine.
The previous commit moved the slow part of a resize out of the swapless window but left
the window itself unrecoverable. Both remaining transitions could take a host from a
working state to no swap at all on a single failure.
1. Live resize destroyed the old file before the new one was active.
The sequence ended: swapoff old, remove old, swapoff staging, rename staging into
place, and only later swapon. A failure at the rename, at the ownership write, or at
the activation left the old file already deleted and the staging file inactive - no
swap, and nothing on disk to go back to. On a host given swap because memory is tight,
this is the worst possible outcome of a routine size change.
The old file is now moved aside as <path>.awg2-harness.previous instead of deleted, and
is only removed once the replacement is confirmed present in the active swap list.
Every step past that point rolls back: the staging file is discarded, the previous file
is renamed back, ownership is rewritten and the original priority reactivated. The
returned error carries both the original cause and the outcome of the rollback, so the
operator can see whether the host still has swap.
A related failure mode is closed with it: rename succeeding but the ownership write
failing left a file with a new inode and a stale record, which the next run correctly
classified as StaleOwnership and refused to touch. Ownership is written inside the
rolled-back section, so that state cannot persist.
2. A priority change could leave the host swapless.
swapoff followed by swapon is two steps and the second can fail. Changing one integer
in the config had no recovery path. change_swap_priority now reactivates at the previous
priority if the new one is rejected, and says explicitly in the error whether the
restore worked.
3. A partially created swapfile wedged the module.
In the create-from-nothing branch, a failure after the file existed but before ownership
was recorded left an unowned regular file at the managed path. The next run classified
it as Foreign - correctly - and refused to proceed, so a transient mkswap failure
required manual intervention. The file is created by this run inside an allowed
root-owned location, so it is now removed on failure, leaving the host as it was found.
Activation moved out of ensure_swap into the reconciliation branches. That is what makes
the above possible: each branch owns both the old and the new file for the whole of its
transition, so it can undo it. A transition also refuses to report success while the
expected swap is not in the active list.
Also in this commit, from the same audit: the legacy ownership sidecar was removed on the
basis of being a regular file with the right name, without checking its contents, so a
file an operator happened to create at that path would be deleted. It now goes through the
same marker check as every other managed file - foreign files are left alone with a
warning.
Two tests added: the suffixed path helpers, and that the sidecar is removed only when it
carries the old marker.
cargo test --workspace: 180 + 4 passed. clippy -D warnings clean.
Four defects from the follow-up audit.
1. Legacy migration swallowed every error.
drop_legacy_managed_file was `let _ = file.remove(); Ok(())`, which tolerated not only
the expected "that path is someone else's now" but also PermissionDenied, read errors
and failed unlinks. If /etc/sysctl.d/99-awg2-harness.conf survived, it would keep
winning over the new 90- file - it sorts later - while the reconcile reported success,
so vm.swappiness would silently stay at the old value. The three cases are now
distinguished: Absent is fine, Foreign warns and continues, Managed removes and any
failure propagates.
2. The disable branches ran outside the lock.
Both ensure_swap and ensure_sysctl_tuning checked `enabled` before acquiring their
lock, so the cleanup path - which deletes managed files - was unsynchronised. A manual
CLI invocation and the daemon's periodic reconcile could write and delete the same file
concurrently. The lock is now taken first in both.
3. A failed teardown left foreign swaps half-disabled.
disable_foreign_swaps aborted the loop on the first failure, so a host with two foreign
swaps could end with the first one off and the second still up - a state nobody asked
for and the error message did not describe. It now tracks what it disabled and
re-enables it, in reverse order and with the original priority, before returning the
error. The restore is best-effort and warns per entry rather than replacing the reported
failure with a rollback failure.
4. Resizing dropped swap for the whole allocation.
A live swapfile cannot be resized in place, and the previous code did swapoff, remove,
fallocate/dd, mkswap, swapon. On a host with no other swap that is a swapless window
spanning the slowest and most failure-prone part of the operation - on a machine that
was given swap precisely because memory is tight, with no way back if any step failed.
The replacement is now built and activated at a staging sibling while the old swap is
still serving, and only then is the old file released. The remaining gap is
swapoff/rename/swapon, which allocates nothing. A failure while staging removes the
staging file and leaves the original swap untouched; a failure at the final rename says
explicitly that the file is in place and can be activated with swapon.
Related, same function: changing only swap.priority used to fall through to the full
recreate path, destroying and rebuilding a correct multi-gigabyte file to change one
integer. Size and priority are now separate decisions - a priority-only change is a
reactivation.
The reconciliation is now decided by an explicit match over (size correct, currently
active) rather than a single `geometry_ok` flag, which is what hid the priority case.
Three tests added around the new decision surface: the priority-only case is not a resize,
the staging path is a sibling of the swapfile, and the managed swap is never part of the
foreign teardown set.
cargo test --workspace: 179 + 4 passed. clippy -D warnings clean.
Two defects that together made the previous ownership model unsafe, and reverse an earlier
deviation that the follow-up audit correctly rejected.
1. The sidecar marker identified a path, not a file.
Ownership was `<swap.path>.awg2-harness-owned` sitting next to the swapfile, and the
check was whether that sidecar existed and started with our marker. Nothing tied it to
the file it was supposed to describe. The sequence that loses data: our swapfile is
removed by hand, the sidecar survives, an operator puts an unrelated file at the same
path - and the next reconcile calls it managed and, if the size differs, deletes it.
The claim in the previous commit that a wrong path "can now only fail, never destroy"
was wrong.
Ownership now lives in /var/lib/awg2-harness/swap-ownership.conf, a root-only directory
created 0700, and records version, path, device and inode. A path is not an identity: a
file can be deleted and another created in its place, but (device, inode) cannot be
reused while our file exists. Classification gained StaleOwnership for "right path,
different file", which refuses exactly like Foreign but says which situation the
operator is actually in. Only ManagedValid and ManagedWrongSize ever authorise a
delete, and that is asserted directly.
The record is written after the file exists, from its real metadata, so until it lands
the swapfile is not considered managed. The old sidecar is now only ever removed, never
read.
classify_ownership is a pure function taking the recorded state and the observed
identity, so the cases that matter - stale record, record for another path, no record -
are unit-testable rather than requiring a Linux filesystem.
2. An arbitrary swap.path was still exploitable.
The previous commit kept arbitrary absolute paths and argued the ownership marker was
sufficient protection. It is not, and the audit is right to reject it. Between
assert_swap_path_is_safe and fallocate/dd/mkswap running, a user who can write the
parent directory replaces the final component with a symlink and redirects a root-owned
operation. That is a TOCTOU; stat-ing before use cannot close it, because the check and
the use are separate syscalls. Paths like /tmp/awg.swap or /home/user/awg.swap made this
reachable by any local user.
swap.path is now restricted to /swapfile or a file directly under
/var/lib/awg2-harness/. Both are root-owned, so the window does not exist rather than
being detected. /swapfile is kept so existing deployments need no config change. The
deny-list it replaces only ruled out the paths someone had thought of; what actually
matters is who can write the parent.
Eleven tests: the five classification states, a stale record, a record for another path,
the invariant that no refusal state is managed, record round-trip, malformed and
unsupported-version records, the newly allowed and newly rejected path shapes.
cargo test --workspace: 176 + 4 passed. clippy -D warnings clean.
Two modules wrote into /etc/sysctl.d and did it differently. sysctl_tuning refused to
touch a file whose first line lacked its marker and cleaned up on disable. swap wrote
/etc/sysctl.d/99-awg2-harness.conf with a different marker, no ownership check at all -
an administrator's file at that path was silently clobbered - and never removed it when
the module was turned off, so vm.swappiness stayed pinned forever.
New module managed_file.rs holds the rules once:
- a read that fails is never an absent file;
- ownership is Absent / Managed / Foreign, decided by the marker on the first line;
- write refuses Foreign, skips when the content already matches, and is otherwise atomic:
temp sibling created with create_new, sync_all, mode 0644, rename, directory flush;
- remove refuses Foreign and treats Absent as success.
Both modules now go through it. sysctl_tuning loses its private write_atomic,
ensure_managed_or_absent and remove_managed_sysctl_file_at.
The swappiness file moves to /etc/sysctl.d/90-awg2-harness-swap.conf with the marker
"# Managed by awg2-harness swap", matching the 90- prefix every other managed file uses.
The old path is removed on first run of the new version, but only when it still carries
the previous "# Managed by awg2-harness" marker; a foreign file there is left alone and
does not fail the migration, since the current file lives elsewhere.
Disable semantics are now explicit: swap.enabled = false removes the managed file and
does nothing else. Runtime vm.swappiness is deliberately not reset - picking a value
would be inventing a policy - and active swap is not deactivated, because merely turning
a feature off should not trigger a destructive action. Previously ensure_swap returned
early and left the persisted file behind.
Also replaced a vacuous test: swappiness_conf_render_is_stable asserted that a string
literal equals a format! of the same literal, touching no production code at all. It now
exercises render_swappiness_conf, and a companion test proves the rendered file is
recognised as managed by the same marker check that guards overwriting it - otherwise
every reconcile would refuse to touch its own output.
Twelve tests: the three ownership states, refusal to overwrite and refusal to remove a
foreign file with content asserted intact, idempotent write, stale temp handling, an
unreadable path reported as an error, plus legacy migration in all three cases - ours,
foreign, absent.
cargo test --workspace: 148 + 4 passed. clippy -D warnings clean.
Two more release blockers in the swap module.
1. Foreign files could be destroyed.
swap.path was validated against a blacklist of exact strings. /etc/../etc/fstab is not
string-equal to /etc/fstab but resolves to it, and nothing normalized the path or
checked the parent chain for symlinks. ensure_desired_swapfile then removed any
existing regular file whose size did not match the configured one, so a mistyped or
hostile path meant deleting /etc/passwd, an application config, or any other regular
file. disable_foreign_swaps additionally deleted foreign swapfiles outright.
Ownership is now explicit. A sidecar marker <swap.path>.awg2-harness-owned sits next to
the managed swapfile, and inspect_swapfile_ownership classifies the path as Missing,
ManagedValid, ManagedWrongSize or Foreign. Foreign is always a hard error - the file is
never resized, rewritten or removed. Foreign active swaps are now only swapped off;
deleting an administrator's swapfile is not this package's call, and swapoff plus the
fstab rewrite already produce the exclusive runtime state.
Path validation is split in two. The lexical half rejects relative paths, '.' and '..'
components, whitespace, control characters and backslashes, the system path list, and
anything under /proc, /sys or /dev; it touches no filesystem, so validate-config stays
side-effect free on a host where the file does not exist yet. It runs after '..' is
rejected, so the string is already normalized and the blacklist cannot be bypassed. The
filesystem half runs just before use and refuses a symlink anywhere in the parent chain,
not only at the final component.
One deliberate deviation from the audit: arbitrary absolute paths stay allowed rather
than being restricted to a fixed managed location. Pinning the path would break
deployments that already configured their own, and the ownership marker is the guard
that actually prevents data loss - a wrong path can now only fail, never destroy.
2. The migration order left the host swapless on failure.
ensure_swap ran disable_foreign_swaps first, then created the swapfile, then activated
it. Any failure in fallocate, dd, mkswap or swapon after that first step left the host
with no swap at all - on a box provisioned with swap precisely because it is short on
memory.
The sequence is now a transaction: preflight, prepare the managed swapfile, swapon,
verify it actually appears in the active list, and only then swapoff the others,
rewrite fstab, apply swappiness and verify. Nothing destructive happens until the
replacement is confirmed live. The one unavoidable gap is deactivating our own swap to
resize it, which cannot be done in place; foreign swap stays up throughout.
chmod 0600 now also happens before mkswap, so the file is never briefly world-readable
while it already holds swap structure.
config.rs no longer carries its own copy of the path blacklist and calls the shared
validator.
Twelve tests: the four ownership states, refusal to recreate a foreign file with its
content asserted intact afterwards, '..' traversal, dangerous and virtual paths,
whitespace and control characters, relative and directory paths, a missing parent
directory, an accepted chain, and a symlinked parent (unix only).
Docs updated: IMPLEMENTATION_SPEC section 17 gained the ownership table, the ordered
transaction and the path rules; CONFIGURATION and the acceptance checklist gained the
operator-facing constraints and nine checks.
cargo test --workspace: 136 + 4 passed. clippy -D warnings clean.
Two release blockers, both in the most destructive operation the package performs.
1. An unreadable /etc/fstab was treated as an empty one.
let content = fs::read_to_string(fstab_path).unwrap_or_default();
PermissionDenied, an I/O error, a damaged filesystem and "file absent" all collapsed
into an empty string. The rewrite then emitted a fstab containing only the managed swap
line and installed it atomically and successfully, dropping the root and boot mounts.
Consequence: a host that does not boot, guarded by one unwrap_or_default. The same
masking was present in read_fstab_swap_entries, so an unreadable fstab was reported by
status as "no swap entries", which reads as healthy drift-free state.
2. The backup was best-effort.
let _ = fs::write(bak_path, &content);
If it failed, the rewrite proceeded anyway. IMPLEMENTATION_SPEC and the acceptance
checklist both asserted the backup exists. The scenario where the backup fails is
exactly the scenario where it is needed.
New module fstab.rs owns the operation:
- read_required distinguishes NotFound from every other error and fails on both. A
missing /etc/fstab on Debian is an anomaly, not a blank slate, so it is never created;
- a blank candidate is refused outright;
- the backup is mandatory and flushed before the temp file is created, so there is no
window where the target is about to be replaced and no recovery copy exists;
- two copies are kept: .awg2-harness.bak is the state before the current rewrite,
.awg2-harness.original is written once with create_new and holds the host as it was
before this package first touched it;
- the temp file is a sibling of the target, created with create_new, sync_all'd, and
inherits the original permissions; a stale temp from a killed run is removed explicitly
rather than reused;
- the candidate is validated with findmnt --verify --tab-file before the swap. The step is
skipped when findmnt cannot be spawned; a rejection is fatal;
- commit is an atomic rename followed by a directory fsync;
- after the rename the target is re-read and compared to what was written. A mismatch
restores the backup and fails.
Also fixed: render_fstab_line interpolated the path raw. fstab fields are
whitespace-separated, so a space or tab in swap.path would shift every following column.
Paths now go through octal escaping (\040, \011, \012, \134).
Ten tests, all portable - the "unreadable but present" case is exercised by pointing the
reader at a directory, which fails as a non-NotFound error on every platform. Covered:
unreadable target is an error rather than an empty file, missing target is an error,
backup failure leaves the target byte-identical, blank candidate refused, unchanged
candidate is a no-op that writes no backup, the pristine copy survives repeated rewrites,
a stale temp does not block commit, and field escaping.
Docs updated: IMPLEMENTATION_SPEC section 17 gained the full rewrite contract, the
acceptance checklist gained seven checks including the two failure modes above.
cargo test --workspace: 124 + 4 passed, ten new. clippy -D warnings clean.
- Swap: допуск небольшой дельты usable size при verify/compare
- Firewall: поддержка Debian 13 dynamic set/add @set вместо только literal meter
- WG: приоритет docker exec для peer/runtime discovery с безопасным fallback
- Полностью переписан swap-модуль в режиме exclusive ownership\n- Добавлены verify/checks, статус фактического swap-state и drift\n- Интегрирован ensure-swap в daemon и systemd ExecStartPre\n- Усилен lock API (таймауты) и применение в traffic/firewall/swap\n- Усилен parser firewall (JSON-first + устойчивый text fallback)\n- Обновлены config validation, runtime tools и default priority\n- Синхронизированы docs и build/package hygiene\n- Удалены member Cargo.lock в пользу корневого lockfile
founder