Frank
327e78b950
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for payloads that no longer run. A throughput figure for a runtime nobody operates is not a record: AI moves faster than the vault, so the number is wrong before anyone reads it, and a wrong number in a runbook gets believed. Per the new purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed. Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest): - the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM, the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two "Running Qwen3.6 …" pages; - the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag) and the multi-device/x4-pair/hybrid/offload experiment pages; - every vLLM page — no vLLM endpoint exists on this fleet; - `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff, self-hosted app census, the TTS survey) that were obsolete by construction; - `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the page that owns the thing it governs, never to a diary. Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras tables keep the window, payload, runtime, reason and rollback pointer; the rate columns are blanked. The live shapes stay verbatim — those numbers are what the deployer and `models.yml` consume.
157 lines
7.8 KiB
Markdown
157 lines
7.8 KiB
Markdown
---
|
|
type: reference
|
|
title: Dankclaw Swap Configuration
|
|
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
|
|
tags:
|
|
- dankclaw
|
|
- host
|
|
- memory
|
|
- swap
|
|
timestamp: 2026-09-12
|
|
---
|
|
|
|
# Dankclaw Swap Configuration
|
|
|
|
**Host:** Dankclaw (192.168.0.143) — **Filesystem:** ext4 on `/dev/nvme0n1p2`
|
|
|
|
| Item | Value |
|
|
| --- | --- |
|
|
| Swapfile | `/swap.img` (68,719,476,736 B = 64 GiB, owner root 0600, mtime 2026-09-12 21:56 — `ls -la /swap.img`) |
|
|
| Size | **64 GiB** (raised from 8 GiB on 2026-09-12); `/proc/swaps` reports 67,108,860 KiB |
|
|
| Type / priority | `file`, priority **-1** — the only swap device on the host (`swapon --noheadings` 2026-10-18: `/swap.img file 64G 187.6M -1`) |
|
|
| Filesystem | ext4 on `/dev/nvme0n1p2` (`df -T /` 2026-10-18) — swapfiles fully supported, no btrfs `nocow` handling needed |
|
|
| Allocation | `fallocate` (preallocated, hole-free; **530 extents** — `sudo filefrag -v /swap.img` → "530 extents found", 2026-10-18) |
|
|
| Persistence | `/etc/fstab` line 12: `/swap.img none swap sw 0 0` (`grep -n -i swap /etc/fstab` 2026-10-18) |
|
|
| systemd unit | `swap.img.swap` — `LoadState=loaded`, `FragmentPath=/run/systemd/generator/swap.img.swap`, `UnitFileState=generated`, `is-active` → `active` (2026-10-18) |
|
|
| `vm.swappiness` | `60` (distro default, unchanged — `sysctl vm.swappiness` 2026-10-18) |
|
|
| zswap | disabled (`/sys/module/zswap/parameters/enabled` = `N`, 2026-10-18) |
|
|
| Swap in use | 187 MiB of 63 GiB (`free -h` 2026-10-18) |
|
|
| Free on `/` | **424 GiB of 1.9 TiB, 77 % used** (`df -T /` 2026-10-18). The 748 GiB that stood right after the 2026-09-12 resize is gone — ordinary fleet writes since then |
|
|
|
|
The unit is **generated from fstab**, not a hand-written unit file. fstab is the
|
|
only control point; `systemctl daemon-reload` re-reads it. Nothing in this
|
|
repository references the swapfile — it is host-level state, documented here so
|
|
it is not invisible.
|
|
|
|
## Why 64 GiB
|
|
|
|
Dankclaw is shared. The 2026-09-12 decision measured `frank-dragoon` alone holding
|
|
**103.5 GiB of the 121.2 GiB** host, leaving roughly **7.1 GiB** for the then-28
|
|
co-tenant containers whose working set was ~4.3 GiB. How that total is split
|
|
between weights, KV pool and working memory is the engine's own ledger — it lives
|
|
on [[frank-dragoon]]. That is a decision-time reading, not today's: on
|
|
2026-10-18 the host reports **91 GiB used / 29 GiB available of 121 GiB**
|
|
(`free -h`), 30 containers are up (`podman ps -q | wc -l`), and the
|
|
`frank-dragoon-halogen` container accounts for **70.33 GiB**
|
|
(`podman stats --no-stream`, same day).
|
|
The engine itself warned at the time that when the headroom runs out,
|
|
frank-dragoon and its neighbours "can stall for minutes at a time at full CPU with
|
|
no disk activity and no output". Swap is the valve that turns an OOM kill into slow
|
|
paging.
|
|
|
|
Swap does **not** relieve the pinned-weights problem — see the caveat below.
|
|
It protects the *co-tenants* and buys headroom for burst allocation.
|
|
|
|
## The safe resize ordering
|
|
|
|
You cannot grow an active swapfile; it must be swapped off first. On a host
|
|
like this, **never `swapoff` before the replacement is online.** The old 8 GiB
|
|
held 4 GiB of live pages, and `MemAvailable` read 73 GiB while only ~5 GiB was
|
|
genuinely free — the pinned model weights are counted as reclaimable page
|
|
cache, so the headline number is a lie by roughly the size of the model
|
|
(see [[frank-dragoon]]). Swapping off into that apparent space, with
|
|
nothing else to receive the pages, is how you invite the OOM killer.
|
|
|
|
The ordering that avoids it — create and enable the new swap **first**, so the
|
|
pages have somewhere to go:
|
|
|
|
```bash
|
|
# 1. Preallocate + format + enable the replacement, old swap still live
|
|
sudo fallocate -l 64G /swap2.img
|
|
sudo chmod 600 /swap2.img
|
|
sudo mkswap /swap2.img # also rejects files with holes
|
|
sudo swapon /swap2.img # total swap now 72 GiB
|
|
|
|
# 2. Only now drain the old one — pages migrate into /swap2.img, not into RAM
|
|
sudo swapoff /swap.img # took 65 s for 4 GiB
|
|
|
|
# 3. Consolidate onto the conventional path so fstab needs no edit
|
|
sudo swapoff /swap2.img # 725 MiB left, trivial
|
|
sudo mkdir -p ~/.AGENTTRASH/20260912-swapfile-8g
|
|
sudo mv /swap.img ~/.AGENTTRASH/20260912-swapfile-8g/swap.img
|
|
sudo mv /swap2.img /swap.img
|
|
sudo swapon /swap.img
|
|
```
|
|
|
|
Step 3 is chained tightly because it leaves the host with **zero swap** for a
|
|
few seconds. Do not leave that window open.
|
|
|
|
`mkswap` succeeding is itself a check: it refuses swapfiles containing holes,
|
|
so a `fallocate` that produced a sparse file fails here rather than at 3 a.m.
|
|
under memory pressure.
|
|
|
|
## Verification
|
|
|
|
```bash
|
|
swapon --show=NAME,TYPE,SIZE,USED,PRIO # /swap.img file 64G 187.6M -1 (2026-10-18)
|
|
systemctl is-active swap.img.swap # active; FragmentPath=/run/systemd/generator/swap.img.swap
|
|
grep -n swap /etc/fstab # line 12 intact -> boot persistence
|
|
free -h | tail -2 # Swap: 63Gi total, 187Mi used (2026-10-18)
|
|
```
|
|
|
|
Then confirm the inference service is unaffected — a completion, not just
|
|
`/health`:
|
|
|
|
```bash
|
|
curl -s http://127.0.0.1:7770/v1/chat/completions \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"model":"frank-dragoon","messages":[{"role":"user","content":"Reply with exactly: SWAP64-OK"}],"max_tokens":48,"reasoning_effort":"low"}'
|
|
```
|
|
|
|
As run 2026-09-12: `finish_reason: stop`, `SWAP64-OK`, 28 containers up,
|
|
frank-dragoon `health=200` throughout.
|
|
|
|
## Caveats
|
|
|
|
- **Swap cannot help the pinned weights.** frank-dragoon's weights are pinned
|
|
deliberately; paging them is a stall, not a relief. More swap does not make a
|
|
larger KV pool affordable — see the reserve/pool ceiling in
|
|
[[frank-dragoon]].
|
|
- **`vm.swappiness=60` is untested against this workload.** Under real
|
|
pressure the kernel will happily page anonymous memory, and frank-dragoon's KV
|
|
pool and working set *are* anonymous. Swapping the KV pool would be far worse
|
|
than the OOM kill it prevents. If the box ever starts thrashing, lowering
|
|
swappiness (or capping the engine's cgroup) is the knob, not more swap.
|
|
- **`free` still lies after the change.** On 2026-09-12 `MemAvailable` read ~70 GiB
|
|
while the model was loaded; on 2026-10-18 it reads **29 GiB** with 90 GiB sitting in
|
|
`buff/cache` (`free -h`). The headline number moves with the payload; trust the
|
|
engine's `host memory left for everything else` line, not `free`.
|
|
|
|
- **Filesystem prerequisite (other hosts).** ext4 and xfs take swapfiles
|
|
directly. btrfs needs `chattr +C` (NOCOW), no compression, and no
|
|
snapshots in the swapfile path — check `df -T` before planning, or
|
|
`mkswap` will reject the file.
|
|
- **Renaming an *active* swapfile works (inode-based) but leaves a stale
|
|
path in `/proc/swaps`** — swap it off first, then rename, then `swapon`
|
|
(which is exactly what the step-3 consolidation does). Migration is
|
|
observable while it runs: in the 2026-09-12 run `swapoff` of 4 GiB took
|
|
65 s and used-RAM rose only ~3 GiB (pages touched during migration), the
|
|
rest relocating on disk. If used-RAM climbs toward the whole swapped
|
|
amount, let it finish rather than aborting.
|
|
|
|
## Trash
|
|
|
|
The previous 8 GiB `/swap.img` (dated 2026-07-21) was staged, not deleted, at
|
|
`dankclaw:~/.AGENTTRASH/20260912-swapfile-8g/swap.img` (8.1 GiB). **That staging
|
|
directory no longer exists** — `ls /home/dank/.AGENTTRASH` on 2026-10-18 lists only
|
|
`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`
|
|
`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`
|
|
`20261017-prism-build`, `20261017-v2abl-trial`, and there is no `/root/.AGENTTRASH`.
|
|
The old swapfile has therefore been purged; nothing on the host still holds it.
|
|
|
|
## See Also
|
|
|
|
- [[frank-dragoon]] — the memory model that motivated this
|
|
- [[Dankclaw System Specs]] — the Strix Halo box
|
|
- [[Dankclaw Improvements]] — other host-level changes
|