Frank
a5fe045b58
Structural repair: - 13 pages were the same document twice (each exactly 2x its HEAD size), a second frontmatter block whose closing fence had been glued onto a body line by a frontmatter rewrite that re-read its own output. Un-doubled by cutting at the second frontmatter and keeping the copy whose body is a superset; the orphan census reports 0 lost tokens against HEAD. - log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first — 66 dated headings merged into 34, titles demoted to bold leads, every content line verified present. The file had been descending at the head and ascending at the tail. Ownership moves: - New `LLM/frank-medium.md` is the locator for the retired Dankserver endpoint: era windows quoted from each era page's own banner, no numbers restated, and the two unresolved date conflicts recorded instead of silently resolved. `frank-archon.md` links to it instead of restating the pre-rename window. - The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone; `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor verdicts and cross-split spec decode, and the four links that pointed at the stubs were repointed at it or at the drafter page. - Authentik's three stub pages fold into `Authentik.md`, with the process detail and the outpost reference split out so the parent stays under the length hint. Verified corrections (host census + live probe, 2026-10-19): - The MXFP4 checkpoint two pages called the "on-disk rollback" is gone: /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on the era page that owns that payload; the swap page now links to it. - frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2 arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run census is kept once (155 requests) instead of twice (~120 and 155). - Fleet shapes re-confirmed against the live endpoints before touching any page: 3x200000 fp8 KV, 2x200704, 2x262144. Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
157 lines
7.8 KiB
Markdown
157 lines
7.8 KiB
Markdown
---
|
|
type: reference
|
|
title: Dankclaw Swap Configuration
|
|
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
|
|
tags:
|
|
- dankclaw
|
|
- host
|
|
- memory
|
|
- swap
|
|
timestamp: 2026-09-12
|
|
---
|
|
|
|
# Dankclaw Swap Configuration
|
|
|
|
**Host:** Dankclaw (192.168.0.143) — **Filesystem:** ext4 on `/dev/nvme0n1p2`
|
|
|
|
| Item | Value |
|
|
| --- | --- |
|
|
| Swapfile | `/swap.img` (68,719,476,736 B = 64 GiB, owner root 0600, mtime 2026-09-12 21:56 — `ls -la /swap.img`) |
|
|
| Size | **64 GiB** (raised from 8 GiB on 2026-09-12); `/proc/swaps` reports 67,108,860 KiB |
|
|
| Type / priority | `file`, priority **-1** — the only swap device on the host (`swapon --noheadings` 2026-10-18: `/swap.img file 64G 187.6M -1`) |
|
|
| Filesystem | ext4 on `/dev/nvme0n1p2` (`df -T /` 2026-10-18) — swapfiles fully supported, no btrfs `nocow` handling needed |
|
|
| Allocation | `fallocate` (preallocated, hole-free; **530 extents** — `sudo filefrag -v /swap.img` → "530 extents found", 2026-10-18) |
|
|
| Persistence | `/etc/fstab` line 12: `/swap.img none swap sw 0 0` (`grep -n -i swap /etc/fstab` 2026-10-18) |
|
|
| systemd unit | `swap.img.swap` — `LoadState=loaded`, `FragmentPath=/run/systemd/generator/swap.img.swap`, `UnitFileState=generated`, `is-active` → `active` (2026-10-18) |
|
|
| `vm.swappiness` | `60` (distro default, unchanged — `sysctl vm.swappiness` 2026-10-18) |
|
|
| zswap | disabled (`/sys/module/zswap/parameters/enabled` = `N`, 2026-10-18) |
|
|
| Swap in use | 187 MiB of 63 GiB (`free -h` 2026-10-18) |
|
|
| Free on `/` | **424 GiB of 1.9 TiB, 77 % used** (`df -T /` 2026-10-18). The 748 GiB that stood right after the 2026-09-12 resize is gone — ordinary fleet writes since then |
|
|
|
|
The unit is **generated from fstab**, not a hand-written unit file. fstab is the
|
|
only control point; `systemctl daemon-reload` re-reads it. Nothing in this
|
|
repository references the swapfile — it is host-level state, documented here so
|
|
it is not invisible.
|
|
|
|
## Why 64 GiB
|
|
|
|
Dankclaw is shared. The 2026-09-12 decision measured `frank-dragoon` alone holding
|
|
**103.5 GiB of the 121.2 GiB** host, leaving roughly **7.1 GiB** for the then-28
|
|
co-tenant containers whose working set was ~4.3 GiB. How that total is split
|
|
between weights, KV pool and working memory is the engine's own ledger — it lives
|
|
on [[frank-dragoon]]. That is a decision-time reading, not today's: on
|
|
2026-10-18 the host reports **91 GiB used / 29 GiB available of 121 GiB**
|
|
(`free -h`), 30 containers are up (`podman ps -q | wc -l`), and the
|
|
`frank-dragoon-halogen` container accounts for **70.33 GiB**
|
|
(`podman stats --no-stream`, same day).
|
|
The engine itself warned at the time that when the headroom runs out,
|
|
frank-dragoon and its neighbours "can stall for minutes at a time at full CPU with
|
|
no disk activity and no output". Swap is the valve that turns an OOM kill into slow
|
|
paging.
|
|
|
|
Swap does **not** relieve the pinned-weights problem — see the caveat below.
|
|
It protects the *co-tenants* and buys headroom for burst allocation.
|
|
|
|
## The safe resize ordering
|
|
|
|
You cannot grow an active swapfile; it must be swapped off first. On a host
|
|
like this, **never `swapoff` before the replacement is online.** The old 8 GiB
|
|
held 4 GiB of live pages, and `MemAvailable` read 73 GiB while only ~5 GiB was
|
|
genuinely free — the pinned model weights are counted as reclaimable page
|
|
cache, so the headline number is a lie by roughly the size of the model
|
|
(see [[frank-dragoon]]). Swapping off into that apparent space, with
|
|
nothing else to receive the pages, is how you invite the OOM killer.
|
|
|
|
The ordering that avoids it — create and enable the new swap **first**, so the
|
|
pages have somewhere to go:
|
|
|
|
```bash
|
|
# 1. Preallocate + format + enable the replacement, old swap still live
|
|
sudo fallocate -l 64G /swap2.img
|
|
sudo chmod 600 /swap2.img
|
|
sudo mkswap /swap2.img # also rejects files with holes
|
|
sudo swapon /swap2.img # total swap now 72 GiB
|
|
|
|
# 2. Only now drain the old one — pages migrate into /swap2.img, not into RAM
|
|
sudo swapoff /swap.img # took 65 s for 4 GiB
|
|
|
|
# 3. Consolidate onto the conventional path so fstab needs no edit
|
|
sudo swapoff /swap2.img # 725 MiB left, trivial
|
|
sudo mkdir -p ~/.AGENTTRASH/20260912-swapfile-8g
|
|
sudo mv /swap.img ~/.AGENTTRASH/20260912-swapfile-8g/swap.img
|
|
sudo mv /swap2.img /swap.img
|
|
sudo swapon /swap.img
|
|
```
|
|
|
|
Step 3 is chained tightly because it leaves the host with **zero swap** for a
|
|
few seconds. Do not leave that window open.
|
|
|
|
`mkswap` succeeding is itself a check: it refuses swapfiles containing holes,
|
|
so a `fallocate` that produced a sparse file fails here rather than at 3 a.m.
|
|
under memory pressure.
|
|
|
|
## Verification
|
|
|
|
```bash
|
|
swapon --show=NAME,TYPE,SIZE,USED,PRIO # /swap.img file 64G 187.6M -1 (2026-10-18)
|
|
systemctl is-active swap.img.swap # active; FragmentPath=/run/systemd/generator/swap.img.swap
|
|
grep -n swap /etc/fstab # line 12 intact -> boot persistence
|
|
free -h | tail -2 # Swap: 63Gi total, 187Mi used (2026-10-18)
|
|
```
|
|
|
|
Then confirm the inference service is unaffected — a completion, not just
|
|
`/health`:
|
|
|
|
```bash
|
|
curl -s http://127.0.0.1:7770/v1/chat/completions \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"model":"frank-dragoon","messages":[{"role":"user","content":"Reply with exactly: SWAP64-OK"}],"max_tokens":48,"reasoning_effort":"low"}'
|
|
```
|
|
|
|
As run 2026-09-12: `finish_reason: stop`, `SWAP64-OK`, 28 containers up,
|
|
frank-dragoon `health=200` throughout.
|
|
|
|
## Caveats
|
|
|
|
- **Swap cannot help the pinned weights.** frank-dragoon's weights are pinned
|
|
deliberately; paging them is a stall, not a relief. More swap does not make a
|
|
larger KV pool affordable — see the reserve/pool ceiling in
|
|
[[frank-dragoon]].
|
|
- **`vm.swappiness=60` is untested against this workload.** Under real
|
|
pressure the kernel will happily page anonymous memory, and frank-dragoon's KV
|
|
pool and working set *are* anonymous. Swapping the KV pool would be far worse
|
|
than the OOM kill it prevents. If the box ever starts thrashing, lowering
|
|
swappiness (or capping the engine's cgroup) is the knob, not more swap.
|
|
- **`free` still lies after the change.** On 2026-09-12 `MemAvailable` read ~70 GiB
|
|
while the model was loaded; on 2026-10-18 it reads **29 GiB** with 90 GiB sitting in
|
|
`buff/cache` (`free -h`). The headline number moves with the payload; trust the
|
|
engine's `host memory left for everything else` line, not `free`.
|
|
|
|
- **Filesystem prerequisite (other hosts).** ext4 and xfs take swapfiles
|
|
directly. btrfs needs `chattr +C` (NOCOW), no compression, and no
|
|
snapshots in the swapfile path — check `df -T` before planning, or
|
|
`mkswap` will reject the file.
|
|
- **Renaming an *active* swapfile works (inode-based) but leaves a stale
|
|
path in `/proc/swaps`** — swap it off first, then rename, then `swapon`
|
|
(which is exactly what the step-3 consolidation does). Migration is
|
|
observable while it runs: in the 2026-09-12 run `swapoff` of 4 GiB took
|
|
65 s and used-RAM rose only ~3 GiB (pages touched during migration), the
|
|
rest relocating on disk. If used-RAM climbs toward the whole swapped
|
|
amount, let it finish rather than aborting.
|
|
|
|
## Trash
|
|
|
|
The previous 8 GiB `/swap.img` (dated 2026-07-21) was staged, not deleted, at
|
|
`dankclaw:~/.AGENTTRASH/20260912-swapfile-8g/swap.img` (8.1 GiB). **That staging
|
|
directory no longer exists** — `ls /home/dank/.AGENTTRASH` on 2026-10-18 lists only
|
|
`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`,
|
|
`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`,
|
|
`20261017-prism-build`, `20261017-v2abl-trial`, and there is no `/root/.AGENTTRASH`.
|
|
The old swapfile has therefore been purged; nothing on the host still holds it.
|
|
|
|
## See Also
|
|
|
|
- [[frank-dragoon]] — the memory model that motivated this
|
|
- [[Dankclaw System Specs]] — the Strix Halo box
|
|
- [[Dankclaw Improvements]] — other host-level changes
|