agent: repoint roles to frank-medium/dankclaw-3090; sync niri tweaks, memories, managed skills

- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh
- models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes
- niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0
- memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills
- managed-skills: 40+ new fleet runbook skills
- agent/skills: import built-in skill set
This commit is contained in:
Frank committed 2026-09-17 12:27:08 -07:00
1 parent 83c9c92b3f
commit 135bb2194d
103 files changed
+5333 -170

No files matched your search

@@ -0,0 +1,95 @@
---
name: frank-medium-batch-split-retune
description: "Retune frank-medium (Dankserver, llama.cpp layer split across 2x R9700) batch/ubatch geometry and the tensor-split balance: sweep with scripts/bench-batch-ubatch.sh, re-measure the split at the deployed shape behind a VRAM settle gate, re-fit the seam, deploy, verify."
---
# frank-medium batch geometry + split rebalance
Use when changing `--batch-size`, `--ubatch-size`, `--ctx-size`, `--parallel` or the
model on frank-medium, or when someone reports that prefill "does not respond" to
those knobs. The batch and the tensor split are **coupled**: the compute buffers are
not symmetric across the two cards, so any batch change moves the balance point.
## Order of work (do not skip the second half)
1. **Sweep the batch geometry with llama-bench**, which is the only tool that can hold
a shape constant across dozens of configs:
`scripts/bench-batch-ubatch.sh`, defaults `-p 65536 -n 128 -d 65536`, one invocation
per cell, `-r 3 --no-warmup`.
2. **Re-measure the split at the DEPLOYED shape** (262144, two slots, drafter loaded) by
loading the real service with a per-config env override and reading `rocm-smi`. A
llama-bench number cannot decide the split: it has no `--parallel`, sizes its own
context as `n_prompt + n_gen + n_depth`, and cannot load the drafter.
3. **Re-fit the balance seam** in `tests/frank_medium_unit.bats` on the new deployed
point, then **deploy** and verify.
## 1. The sweep
```bash
# from a transient unit, never a shell: the deploy's LimitMEMLOCK=infinity is
# load-bearing for ROCm staging buffers and an SSH shell gets 8 MiB here
sudo systemd-run --unit=fm-bench --collect --pipe --property=Type=oneshot \
--property=LimitMEMLOCK=infinity --property=User=dank \
--setenv=OUT=/home/dank/models/logs/bench-bub \
--setenv=BATCHES=6144 --setenv=UBATCHES="256 384 512 768 1024 1536 2048" \
/home/dank/serverAppConfigs/scripts/bench-batch-ubatch.sh
```
The script stops `frank-medium`, runs one `llama-bench` per cell into `cells.tsv`
(appended as it goes, so progress is readable), samples VRAM per cell, and restarts
the service from an EXIT trap. A re-run skips cells that already have a `bench.json`.
Read `n_ubatch` out of each `bench.json` to prove the knob was applied — if it does not
track the request, nothing else in the table means anything. The VRAM columns scaling
monotonically with ubatch is the independent confirmation.
Expected shape (2026-09-16, 2x R9700, UD-Q8_K_XL + DFlash2, 64k prefill at 64k depth):
an optimum at **ubatch 512** (~1193 tok/s) with losses on both sides (256 -> 1046,
384 -> 1068, 2048 -> 1077). The logical batch is throughput- and VRAM-neutral
(2048..8192 within 2 %), and decode never moves (15.88 tok/s at depth 65536 on every
cell). Test **both** ends of the range: sampling only upward makes a shallow basin look
like a plateau.
Sanity rule: the same ubatch measured under different logical batches should agree to
under 1 tok/s. If it does not, the table is noise and the first suspect is a stray
GPU user, not the knob.
## 2. The split, at the deployed shape
`/tmp/measure-ub512b.sh` (pattern; rewrite per run) copies `scripts/frank-medium.env`
to a temp file with `sed` overrides, launches `scripts/serve-frank-medium.sh` with
`FRANK_MEDIUM_ENV_FILE=<temp>`, polls `/health` for `"status":"ok"`, reads
`rocm-smi --showmeminfo vram --csv`, kills, and repeats per split. Run it under
`systemd-run --pipe --property=LimitMEMLOCK=infinity --property=User=dank`.
**The settle gate is mandatory.** Between configs, wait until `rocm-smi` shows BOTH
cards under ~1 GiB used before the next load. amdgpu does not return ~26 GiB when the
process dies — a 180 s sleep fails, and even 600 s has failed while the process is still
alive inside the unit. Skipping the gate produces
`error loading model: unable to allocate ROCm0 buffer`, which is indistinguishable from
"this split does not fit" and will send you chasing a phantom OOM.
Free bytes to expect, and the arithmetic behind them:
`card = weights_share + KV_share + constant + ubatch_term`, where the ubatch term costs
the drafter-carrying card roughly **1.85x** what it costs the other (measured 2.41 GiB
against 1.30 GiB going 1536 -> 512). Balance by measurement only: one `-ts` step is one
layer, worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on an attention one, so the
reachable minimum gap is about one DeltaNet layer, never zero.
2026-09-16 reference: 36,29 @1536 -> 4.71 / 4.91; 36,29 @512 -> 6.01 / 7.32;
**35,30 @512 -> 6.44 / 6.90 (deployed)**.
## 3. Seam, deploy, verify
- Re-fit both ubatch coefficients in `tests/frank_medium_unit.bats` from the measured
1536 -> 512 deltas at the deployed shape, and re-anchor the per-card constants so the
model reproduces the new point. Check that neighbouring splits still FAIL the balance
gate, or the seam has stopped discriminating. Keep the floor at 1.5 GiB/card and the
slack at one DeltaNet layer.
- Do not pin a literal split elsewhere: the unit test asserts the *shape* (two shares
summing to the model's 65 blocks, ROCm1 the smaller).
- `./tests/run.sh`, commit, `./scripts/deploy.sh dankserver --service frank-medium`.
- Verify live: the flags in `/proc/<pid>/cmdline`, `/slots` = 2 x 131072, VRAM matching
the pre-deploy measurement, a tool call, two concurrent requests, `draft acceptance`
lines on BOTH slots, and 0 ROCm errors in the journal. The VRAM figure sits ~0.2 GiB
lower once the service has served traffic than straight after load.