Fleet changelogs · dev.ecs0.net
rdmsm4x-changelog-20260908-1453-jdmbair13m5-disk-full-not-tyrell

jdmbair13m5 is 100% full — and Tyrell is not why. It was also never unreachable.

Read-only disk measurement across all six fleet hosts, from Rich's "jdmbair13m5 is low on disk — I need to free local disk", relayed by the Tyrell cloud oversight thread.

The answer

jdmbair13m5 has 105 MiB free of 926 GiB — 100% full. Confirmed.

Tyrell is not the cause. Its entire footprint on that host is ~1.99 GB, 0.22% of the 901 GiB in use, and ~/TyrellStore — the unbounded CAS that ISSUES.md #57 is about — is 0 B there.

What Size Share of used
~/Pictures 331 GB 37%
~/Documents 269 GB 30%
~/Library 50 GB 5.5%
all Tyrell paths combined 1.99 GB 0.22%

Deleting every byte Tyrell owns on that host returns 1.99 GB against a 105 MiB deficit. Freeing meaningful space means Pictures and Documents — 600 GB between them, and Rich's call alone.

⚠ The trap that matters before anyone cleans that host

tyrelld on jdmbair13m5 runs from ~/dev/apps/Tyrell/.build/release/tyrelld (pid 5006, verified over the LAN). .build/ is 941 MB and is the obvious first delete on a Swift project. Deleting it deletes the running daemon's binary. Five of six hosts share this layout; only rdmpw3275m uses a versioned release directory.

It was never unreachable — correcting my own earlier finding

Two prior checkpoints recorded jdmbair13m5 as unreachable with an ssh accept-then-reset, attributed to a launchd inetd cap. That was wrong about the cause.

~/.ssh/config pins jdmbair13m5 to HostName 100.86.185.90 (tailnet). Tailscale is stopped on that host — tailscaled has 0 processes — and that IP does not answer ping. The host is healthy on the LAN at 192.168.1.225 with port 22 open; a full shell was obtained there immediately.

ping jdmbair13m5 resolved the LAN address (2/2, 3.1 ms) while ssh used the tailnet one. The two commands were never testing the same route, and the passing one was read as proof of life.

Does a full disk explain it? Partly, and not as proposed. A full disk did not stop sshd — it is serving right now with 105 MiB free. What stopped is Tailscale, and a daemon maintaining a local state DB is a plausible casualty of a disk with no room, which would make the full disk a root cause one step removed. Not established: no ENOSPC entries in the last 3 h of its log, though log storage is itself constrained on a full disk. No credential was touched at any point.

ISSUES.md #57 measured — 98.95 GiB unreferenced

Hub only; the CAS is hub-only (four of five spokes hold 0 blobs).

Not a reclaim target. "Unreferenced" is exactly the state a blob enters when its source file is deleted and pruneVanished drops the index row — so this is most likely where the only copy of ~99 GiB of deleted content now lives. It argues for tyrell cas gc --dry-run and a Rich-set retention policy, not for rm. The hub has 3.0 TiB free of 7.3 TiB; no emergency.

New issue #65 — a second unbounded store

~/.tyrell/bundles/ holds 81.8 GiB on the hub across 1,171 files, oldest 2026-08-24, newest today, nothing pruned. 37.9 GiB of that is twelve byte-identical copies of one 3.2 GB file (cmp confirms identical). rdmpw3275m carries three of the same and 11 GB total. Producer not traced — that is a source read, not another probe. Nothing deleted.

Fleet headroom

Host Data vol Used Free Cap TyrellStore .tyrell .build daemon
jdmbair13m5 926 GiB 901 GiB 105 MiB 100% 0 B 838 MB 941 MB .build ⚠
rdmsm4x 7.3 TiB 4.2 TiB 3.0 TiB 58% 370 GB 82 GB — .build ⚠
rdmbair15m5 3.6 TiB 2.5 TiB 1.1 TiB 69% 0 B 2.6 GB 1.3 GB .build ⚠
rdmpw3265m 7.3 TiB 2.5 TiB 4.8 TiB 34% 0 B 913 MB 258 MB .build ⚠
rdmbair13m5 3.6 TiB 1.0 TiB 2.6 TiB 28% 8 KB 1.0 GB 1.8 GB .build ⚠
rdmpw3275m 7.3 TiB 1.3 TiB 5.9 TiB 19% 0 B 11 GB 3.1 GB versioned ✓

jdmbair13m5 is the only host under pressure, and the only one whose pressure has nothing to do with Tyrell.

Files touched

File Change
~/dev/apps/Tyrell/SESSION-STATE.md new top checkpoint (+142 lines)
~/dev/apps/Tyrell/ISSUES.md #57 updated with the measurement; new #65

No source, script, config or data file was modified anywhere. Nothing was deleted on any host.

Git

Ref Before After
local / backup/main / fleet/main 8d7cbab 7d71bd0 (all three)

Commit via TYRELL_CANONICAL_INTEGRATION=1 (documentation only). Invariants verified: SESSION-STATE 2575 → 2717 lines, headings 87 → 88, zero lost; ISSUES 1099 → 1164 lines, distinct numbers 49 → 50, zero lost, exactly #65 added.

Outstanding owner actions

No secrets, credentials, tokens or signed URLs appear in this record.