Compare commits

...
Author SHA1 Message Date
Frank b651d61056 docs(niri): portable desk runs landscape, HDMI left of the laptop
record the normalized layout that landed on main as 9b88208 and the law
that surprised it: positions are absolute, so rotating a panel widens
its logical box and every x to the right of it moves — recompute the
row, never just the rotated block.
2026-10-30 15:56:39 -07:00
Frank e197c80dcb Merge remote-tracking branch 'origin/main' into docs/niri-portable-desk 2026-10-30 15:56:39 -07:00
Frank 9b8820827c niri: desk-2 panels landscape, HDMI to the left of the laptop
Frank asked for normal rotation on both PM1601 panels and the HDMI-A-1
panel to the left of eDP-1. rotation drops 90 -> normal, so each panel
goes 1200x1920 -> 1920x1200 and the absolute x of everything to its
right moves with it: HDMI-A-1 now x=0, eDP-1 x=1920 (y=60, centred
against the taller panels), DP-2 x=3840. modes unchanged.
2026-10-30 15:56:35 -07:00
Frank b8584cd6a6 docs(niri): own the portable desk and the no-match defect
displays.md still described desk-2-portable as an EDID-pending skeleton
and left the PM1601 identity unnamed. record the second desk's rig,
its position math, and the law that a rig with no eligible profile runs
the previous desk's values rather than "unchanged" — the failure that
left the portable panels on 3440x1440@180 fallbacks for weeks.
2026-10-30 15:54:43 -07:00
Frank 5bfdc98e61 niri: make desk-2-portable a real profile for the second desk
The portable rig (laptop + two 16" PM1601 panels in portrait) had no
profile of its own, so display.kdl stayed on desk-1-home: the Sceptre
identity block matched nothing, its connector-keyed fallbacks carried
3440x1440@180.000 that no panel here reports, and niri silently drove
every panel at its preferred mode with positions written for the home
desk. key the two panels by captured EDID identity, pin the modes they
actually report, and centre the laptop against the 1920-tall panel band.

desk.sh match now selects this profile, so blankpad-desk-autoswitch no
longer dies at its initial match with "no desk profile matches".
2026-10-30 15:54:34 -07:00
Frank 9f142b7eaf live: heartbeat sync 2026-10-30T15:20:04-07:00 (Refs #11) 2026-10-30 15:20:04 -07:00
Frank ad5903c294 live: heartbeat sync 2026-10-30T14:17:40-07:00 (Refs #11) 2026-10-30 14:17:40 -07:00
Frank ef7478f9c7 live: heartbeat sync 2026-10-30T13:16:49-07:00 (Refs #11) 2026-10-30 13:16:49 -07:00
Frank 05f4fa412e live: heartbeat sync 2026-10-30T12:13:49-07:00 (Refs #11) 2026-10-30 12:13:49 -07:00
Frank 7508541310 live: heartbeat sync 2026-10-30T11:11:49-07:00 (Refs #11) 2026-10-30 11:11:49 -07:00
Frank d5b6b0e930 live: heartbeat sync 2026-10-30T10:10:49-07:00 (Refs #11) 2026-10-30 10:10:49 -07:00
Frank 28fd76fa21 live: heartbeat sync 2026-10-30T09:07:40-07:00 (Refs #11) 2026-10-30 09:07:40 -07:00
Frank b6e3bf2769 live: heartbeat sync 2026-10-30T08:05:49-07:00 (Refs #11) 2026-10-30 08:05:49 -07:00
Frank 05f818ac03 live: heartbeat sync 2026-10-30T07:04:49-07:00 (Refs #11) 2026-10-30 07:04:49 -07:00
Frank 223768cc91 live: heartbeat sync 2026-10-30T06:02:49-07:00 (Refs #11) 2026-10-30 06:02:49 -07:00
Frank 57fc14057e live: heartbeat sync 2026-10-30T04:59:49-07:00 (Refs #11) 2026-10-30 04:59:49 -07:00
Frank b9dcb88f9b live: heartbeat sync 2026-10-30T03:56:49-07:00 (Refs #11) 2026-10-30 03:56:49 -07:00
Frank fb9b8b7d22 live: heartbeat sync 2026-10-30T02:53:49-07:00 (Refs #11) 2026-10-30 02:53:49 -07:00
Frank 53dd12e73d live: heartbeat sync 2026-10-30T01:50:49-07:00 (Refs #11) 2026-10-30 01:50:49 -07:00
Frank 52e8cca8f3 live: heartbeat sync 2026-10-30T00:48:49-07:00 (Refs #11) 2026-10-30 00:48:49 -07:00
Frank 7b7566928f live: heartbeat sync 2026-10-29T23:45:49-07:00 (Refs #11) 2026-10-29 23:45:49 -07:00
Frank ee7489b546 live: heartbeat sync 2026-10-29T22:44:49-07:00 (Refs #11) 2026-10-29 22:44:49 -07:00
Frank 4b032e011b live: heartbeat sync 2026-10-29T21:41:49-07:00 (Refs #11) 2026-10-29 21:41:49 -07:00
Frank dc56eb5b02 live: heartbeat sync 2026-10-29T20:38:49-07:00 (Refs #11) 2026-10-29 20:38:49 -07:00
Frank a5e00a09eb live: heartbeat sync 2026-10-29T19:37:40-07:00 (Refs #11) 2026-10-29 19:37:40 -07:00
Frank 04ada2c913 live: heartbeat sync 2026-10-29T18:36:49-07:00 (Refs #11) 2026-10-29 18:36:49 -07:00
Frank 4df5a28ba6 live: heartbeat sync 2026-10-29T17:35:49-07:00 (Refs #11) 2026-10-29 17:35:49 -07:00
Frank 03bf48956f live: heartbeat sync 2026-10-29T16:32:39-07:00 (Refs #11) 2026-10-29 16:32:39 -07:00
Frank 4d76c6038e live: heartbeat sync 2026-10-29T15:29:37-07:00 (Refs #11) 2026-10-29 15:29:37 -07:00
dank 3eb1ada55b Merge pull request 'feat: add the DeepSeek platform provider and deepseek-flash' (#84) from feat/83-deepseek-provider into main 2026-10-29 14:52:48 -07:00
37 changed files with 936 additions and 49 deletions

No files matched your search

@@ -1,9 +1,9 @@
---
name: agenttrash-purger-invocation
description: "Invoke the .AGENTTRASH purger (scripts/purge-agenttrash.sh) to clear staged deletions, and diagnose its guaranteed exit 1 on hosts whose find is uutils (Rust) rather than GNU."
description: "Invoke the .AGENTTRASH purger (scripts/purge-agenttrash.sh) — plus its two refusal traps: ENOSPC masquerading as a stale lock, and an unreadable container-storage directory poisoning the whole root scan."
---
# Purging .AGENTTRASH — invocation and the uutils-find trap
# Purging .AGENTTRASH — invocation, and the two traps that make it refuse
`scripts/purge-agenttrash.sh` (serverAppConfigs, merged in PR #79 `fad87aa`) is the tool form of the fleet's deletion policy: files land in `.AGENTTRASH`, and an approved purge runs this instead of being improvised per session.
@@ -15,14 +15,28 @@ sh scripts/purge-agenttrash.sh --execute # perform the plan
sh scripts/purge-agenttrash.sh --execute /some/root # scope to one root (default: $HOME)
```
- `--quiet` silences per-line output; errors and the summary still print. `--log PATH` overrides the default `~/.AGENTTRASH-cleanup.log` (one dated line per run — written even for a dry run, so pass a scratch `--log` when you only want to look).
- `--quiet` silences per-line output; errors and the summary still print. `--log PATH` overrides the default `~/.AGENTTRASH-cleanup.log` (one dated line per run — written even for a dry run).
- Every printable line is fixed at plan time in `$HOME/.agenttrash-purge.lock/plan`, so dry-run and `--execute` output cannot drift. Exit `0` ok · `1` any refusal or failure · `2` usage.
- Rule: a root's own `.AGENTTRASH` is *emptied* (the directory survives); deeper `.AGENTTRASH` trees are *removed*; symlinked or foreign-owned candidates are refused, never followed; `/` is refused outright.
- Deleting a big staged item by hand (e.g. `find <dir> -depth -delete`) is the fallback **only** when the tool cannot run at all — see trap 1 — and the operator's purge approval covers exactly that.
## The trap: uutils find makes it exit 1 on the dev box
## Trap 1: a full filesystem makes it lie about a lock (measured 2026-10-30)
On the workstation, `sh scripts/purge-agenttrash.sh` reports `refused /home/blank (find failed)` and exits 1 **even though the plan is correct** — and an `--execute` there acts on the recorded plan while still reporting failure.
With **0 bytes free**, three invocations refused with
- Cause: `find --version` reports `find 0.8.0` (Rust **uutils**, errors as `Error: <path>: Permission denied (os error 13)`), not GNU findutils. It exits 1 whenever any directory under the root is unreadable — and `~/.local/share/containers/storage/**` (podman overlay layers, mode 0700 dirs) is unreadable. `collect_root` turns any nonzero find status into a root refusal.
- Proved by: `find /home/blank -xdev -name .AGENTTRASH >/dev/null 2>&1; echo $?` → `1`, while `find /home/blank/.omp -xdev ...` → `0`.
- Fix, if it ever comes up (don't invent it): treat unreadable-directory errors as warnings, or accept find's partial output and reserve refusal for a hard find failure. Note `bash -n` is not the gate — shellcheck / `sh -n` is.
```
purge-agenttrash: refused /home/<user>: exists and is not ours to reclaim
```
and no lock directory existed anywhere. The script writes its lock *and* its plan file before scanning; both hit `ENOSPC`, and the failure surfaced as the lock-reclaim message. There is nothing to reclaim — **free space first** (delete the staged item, move it to another volume, `journalctl --vacuum-size=`), then run plan + `--execute`. Never hunt a phantom lock because the message says so; check `df` first.
## Trap 2: an unreadable directory spoils the whole root
`collect_root` turns any nonzero `find` status into a root refusal, so the plan is built from a partial scan and the root reports `refused=1` even though `find` printed every `.AGENTTRASH` it found.
- On the dev box the cause is uutils find (`find --version` → `find 0.8.0`), which exits 1 on `~/.local/share/containers/storage/**`. Proof: `find /home/blank -xdev -name .AGENTTRASH >/dev/null 2>&1; echo $?` → `1`, while a narrower root → `0`.
- On **dankserver** GNU find does the same, because rootless podman storage holds foreign-UID directories (`overlay/*/diff/var/lib/mysql`, `valkey`, `calibre/plugins`, `apt/.../partial`): `find /home/dank -xdev -name .AGENTTRASH 2>&1 >/dev/null | head` shows the denials.
- Fix to land in the script (not to improvise around): prune `~/.local/share/containers` from the scan — a `.AGENTTRASH` never lives inside container storage — or accept partial output and reserve refusal for a hard find failure. `sh -n` / shellcheck is the gate; `bash -n` is not.
## Verified good path (dankserver, 2026-10-30)
After freeing space by hand: `/tmp`-scoped plan → `--execute` deleted the staged item and reported `removed=1 deleted=1`; a second `--execute /mnt/entropy` cleaned that volume's root (`refused=0`). The cleanup log carries one dated line per run; read it instead of re-deriving what happened.
@@ -0,0 +1,53 @@
---
name: live-checkout-landing-chain
description: "Prove whether a merged change is actually deployed in ~/.omp: walk origin/main vs the checked-out main commit vs the worktree, then fast-forward in place — git refuses fetch into a checked-out branch, and a running harness caches its config."
---
## When to use
The user says a merged/committed change is "not there" (`/switch` can't see a model, a doc is missing, a config edit is invisible) while the repo work itself is green. In a repo where **the checkout is the deployment** (`dank/command-center` at `~/.omp`), "merged" and "deployed" are different commits — measure both before believing either claim.
## The ladder — read all four levels, in order, before answering
```bash
cd ~/.omp && git rev-parse HEAD origin/main # 1+2: deployment vs forge
git ls-tree --name-only HEAD # is the file even in HEAD?
git worktree list # 3: which worktree am I in vs main?
```
1. `origin/main` — the forge truth (`gh`/`tea` absent; Gitea API or a fetch).
2. `/home/blank/.omp` HEAD — **what the running agent actually reads.** Merged PR at `3eb1ada` while this sat at `53025a6` is the whole bug.
3. The feature/session worktree — where the edit was authored; its files can be correct while main is stale, which is why the author sees the change and the user does not.
4. The running process — `models.yml` / `config.yml` are read at startup and cached for the process lifetime; an open session needs a restart even after level 2 is correct.
Belt-and-braces when the answer hinges on file contents, not just commit ids: parse the live file (`python3 -c "import yaml; ..."` for `agent/models.yml`, or grep the doc) rather than inferring from `git log`.
## Landing main without touching the tree
`git fetch origin main:main` **fails**: `fatal: refusing to fetch into branch 'refs/heads/main' checked out at '/home/blank/.omp'`. The fetch-into-ref trick only works for branches that are *not* checked out anywhere.
For a checked-out main, prove then fast-forward:
```bash
git merge-base --is-ancestor main origin/main && echo "ff possible"
git merge --ff-only origin/main # no rebase — Frank dislikes history rewrites
```
`--ff-only` is the safe verb on a dirty main checkout: tracked-file commits (the live-sync tier) and runtime dirt (`agent/memories/*/state.json`, `dotfiles/**`) survive untouched, and it cannot rewrite history. Confirm the dirt is still there afterwards — if it vanished, something else clobbered the tree.
## The heartbeat is not instant
`scripts/live-sync.sh` (systemd user timer, hourly) is what normally advances main. Check its cadence rather than assuming it ran:
```bash
systemctl --user status live-sync.timer --no-pager # "Trigger: …" = next beat
systemctl --user status live-sync.service --no-pager # "since …" = last run
```
A merge made minutes after the last beat waits until the next one; fast-forwarding by hand is legitimate (idempotent with the script) and is the answer when the user wants it now. Never `--no-verify`, never force-push.
## Do not confuse with
- `omp-worktree-path-gotchas` — edits landing in the *wrong* worktree (cwd drift), not a stale main.
- `dotfiles-deployment-truth-audit` — classifies which files are tracked vs runtime state.
- `serverappconfigs-vault-page-registration` / `gitea-merge-form-encoded` — the PR mechanics upstream of this question.
@@ -0,0 +1,86 @@
---
name: omp-provider-row-sources
description: "Add a provider/model row to ~/.omp/agent/models.yml by reading the harness's own catalog tables (provider name, api, window, cost, compat) instead of guessing or web-searching, then prove it with a scratch-HOME call."
---
# Scope
Adding (or correcting) a provider/model row in `~/.omp/agent/models.yml` for a
provider the harness may already know: an upstream vendor API (DeepSeek, OpenAI,
Anthropic, Moonshot, …) or a new local endpoint. The rule this skill exists for:
**almost every field you are about to research on the web already exists in the
installed bundle** — research there first, and the web only confirms it.
Canonical install root (resolve the symlink first if the repo lives under a
worktree): `/home/blank/.bun/install/global/node_modules/@oh-my-pi/`.
# The three tables that own the facts
| You want | Read | Why |
|---|---|---|
| The legal **provider name**, its URL marker, and which quirks the harness gates on it | `pi-catalog/src/hosts.ts` (minified in `dist/cli.js` as the `VIn` map; consumed by `pi-catalog/src/compat/resolve.ts`) | The provider key is a **lookup key, not a display name**. e.g. `deepseekDirect: { providers: ["deepseek"], urlMarkers: ["api.deepseek.com"] }`, `anthropic: { providers: ["anthropic"], urlMarkers: ["api.anthropic.com"] }`. A label matching neither loses the behaviour set silently (the `max_tokens` field name, the thinking `extraBody`, the thinking-disable mode). |
| Model rows: ids, `contextWindow`, `maxTokens`, `cost` rates, per-model `compat` | `pi-catalog/src/provider-models/<api>.ts` — `openai-compat.ts`, `anthropic-messages.ts`, …; factory helpers `openAiCompletionsDescriptor(...)` / `anthropicMessagesDescriptor(...)` | Rows are fetched from the provider's models-dev key, so the bundle carries the same numbers the vendor publishes (and often rates the vendor page hides). The descriptor's `compat:` block is the authored truth for that provider's transport shape — copy it, never simplify it. |
| The **schema** itself (which keys are legal on a provider vs a model, and which are required) | `pi-coding-agent/src/config/models-config-schema-bundle.ts` (and the identical `dist/` copy) | Provider level and model level have *different* field sets. Cheaper and more correct than reading the minified bundle. |
Also in `pi-coding-agent/src/config/`: `model-registry.ts` (how `models.yml`
merges onto the built-in catalog — `#mergeCustomModels` → `mergeByModelKey`),
`resolve-config-value.ts` (how `apiKey` values resolve).
# Two traps this table pays for
- **Provider-level `api` only accepts chat/judge transports.** Naming a runner API
(`openai-speech`, `openai-images`, `openai-embeddings`) at provider level fails
whole-file validation and disables *every* custom provider, not just yours
(`omp models` prints "models.yml validation failed", lists only built-ins).
Runner api belongs on the **model** entry; see the managed skill
`omp-models-yml-runner-api-placement`.
- **A `models.yml` entry is an overlay, not a replacement.** Adding one model to a
provider the catalog already ships does not remove the provider's other rows —
`omp models <provider>` will show yours (yours wins on shared fields, including
`cost` and `contextWindow`) plus the catalog rows. Same-id rows patch with the
`replace` transport policy; a row that omits `contextWindow` inherits the
bundled row's window and its policy tiers.
# Secret handling (non-prose tracked file)
Never inline the credential. `apiKey` resolution order is: leading `!` → command
(`stdout` trimmed, cached for the process lifetime) → exact env-var name → literal.
So write `apiKey: "!cat ~/.config/secrets/<name>_api_key"` with mode 600.
- A bare `sk-…` trips the pre-commit **shape** gate (every file type).
- An unquoted `!` is a YAML tag — quote it.
- A quoted command value also dodges the *assignment* gate: the gate's value
charset rejects `~` and spaces, so only a literal key matches.
# Procedure
1. Read the three tables above for the provider you are adding. Take the exact
provider name, base URL, api, and compat from them; take `contextWindow` /
`maxTokens` / `cost` from the model row and cite the source in a comment.
2. Write the secret to `~/.config/secrets/<name>_api_key`, `chmod 600`.
3. Edit `models.yml`; for a repo-tracked config, follow that repo's own
branch/log/index rules (in `dank/command-center`: branch → issue → PR).
4. Verify with a **scratch HOME**, because `models.yml` loads at process start
and a live session keeps the old registry:
```bash
mkdir -p /tmp/ompx/.omp/agent /tmp/ompx/.config
cp agent/models.yml agent/config.yml /tmp/ompx/.omp/agent/
ln -sfn ~/.config/secrets /tmp/ompx/.config/secrets # ~ in a !cmd value = $HOME
HOME=/tmp/ompx omp models --no-extensions | grep -c 'validation failed' # must be 0
HOME=/tmp/ompx omp --model '<provider>/<id>:off' --tools bash --no-session \
--no-skills --no-rules --no-extensions --no-pty --no-lsp --no-title \
--cwd /tmp -p 'Reply with exactly the word OK and nothing else.'
```
Then the **negative control**, the step that separates a real auth success from
a cached/built-in answer: `sed` the `baseUrl` to a dead port in the scratch copy
and repeat — it must hang or error, not answer. Then `rm -rf /tmp/ompx` (it
holds endpoint URLs).
# Voice notes
- Prefer the bundled schema file over grepping `dist/cli.js`; grep the bundle only
when the source and the bundle disagree, or for a runtime default the schema does
not state (e.g. `reasoningDisableMode` per provider family).
- `cost` is optional in the schema. Omitting it makes the status line report `$0`
forever; one rate per direction is all the schema models, so a peak/off-peak
vendor table can only carry the peak value.
@@ -0,0 +1,91 @@
---
name: quant-pipeline-adversarial-review
description: "Review or audit the serverAppConfigs quantization campaign pipeline (convert-calib-kldref-score plus corpus builder/rechunker/pusher, fp8 stacker) for defects that silently waste GPU-hours or produce a number that is not the claimed number."
---
# Adversarial review of the quant campaign pipeline
Friction-earned 2026-10-29 across `scripts/convert-qwen38-w5nl64.sh`,
`run-qwen38-bf16.sh`, `make-calib-corpus.py`, `rechunk-kld-corpus.py`,
`calib-push.py`, `stack-qwen38-official-fp8.py`. Every class below was a real
defect in that tree. Review code, not vibes: quote the line, name the cost.
## Files in scope (one campaign = this set)
| Stage | File | Owns |
|---|---|---|
| corpus | `make-calib-corpus.py` | calib + kld JSONL, manifest |
| corpus | `rechunk-kld-corpus.py` | windows that fit the container's KV window |
| convert | `convert-<model>.sh` | plan → artifact per arm |
| tape | `run-qwen38-bf16.sh` | the rig's argv, mount, trap, gates |
| drive | `calib-push.py` | prefill traffic into the tap |
| fp8 path | `stack-qwen38-official-fp8.py` | expert stacking + PLE merge |
## Defect classes to hunt, in cost order
1. **Claimed identity that is only a length check.** A tool that writes TEXT back
out (`tok.decode(window)`) and asserts only counts is not proving token
identity — decode→encode can merge/split. Fix: re-encode each window and
compare id lists; refuse on mismatch. Same shape: an assertion on "N tokens
in, N tokens out" that counts windows rather than re-encoded text.
2. **A loop that cannot finish and cannot notice.** Drivers that re-push a corpus
`--max-rounds` times without measuring the sink. Compare the tap/output dir's
byte total per round and abort on zero growth; abort on zero successes; abort
on zero prompts. Without this the loss is silent for hours.
3. **`set -u` argument holes.** `--flag) VAR="$2"; shift 2` dies with "unbound
variable" when the flag is last. Guard `[ "$#" -ge 2 ]` in every such arm.
4. **Restore paths that leave the GPU holder alive.** `--rm` only fires when
podman itself exits, so a killed run leaves conmon holding the cards while
`systemctl is-active` says `active`. `restore()` MUST `podman rm -f "$C"`
before starting production. Prove with `podman ps` after a `kill`.
5. **Gates that run after the expensive phase.** A leftover container name, an
existing output, a non-writable tap dir, an unset figure — check them before
the plan/boot, not when podman/sed fails mid-run.
6. **Zero-exit with no artifact.** After `rc=0`, assert the output file exists
before `stat`/arithmetic on it.
7. **`assert` in a verifier.** `python -O` deletes it silently. Use `raise`.
8. **Warning where the code's own assumption breaks.** e.g. a 128-lane pack fed
an HD that is not a multiple of 128 — warning + continue writes a wrong table.
9. **Re-tokenization in loops.** A pool cycled 40× re-tokenizing each fragment
per pass; and a manifest recomputing a total by re-encoding the whole corpus.
Cache counts once, accumulate during generation.
10. **Substring tests standing in for key tests.** `"content" in line` before
`json.loads(line)["content"]` crashes on any list-shaped `.json`.
11. **Path guards compared as strings.** `A/../A/x` defeats an equality check;
add `realpath -m` on both sides.
12. **Shellcheck's honest false positive**: single-quoted `$CALIB` passed to a
converter that expands it from the container env is correct (SC2016). Say so
in the PR; do not "fix" it.
## Procedure
1. Inventory: `wc -l` the scripts + recipes; read each **completely** (ranges,
not whole files, but every line). Recipes carry the numbers scripts assert.
2. Read the tests that pin the argv (`tests/qwen38_reference.bats`'s `SHAPE`
fixture, `tests/qwen38_campaign.bats`) — they are the contract you must not
break, and the first place a stale number hides.
3. Mechanical gates:
```bash
python3 -m py_compile scripts/*.py
bash -n scripts/<each>.sh
shellcheck -S warning scripts/<each>.sh # never pass .py to it
./tests/run.sh
```
4. Fix the real ones only; keep diffs surgical; re-run the suite after the batch.
5. Smoke the changed path for real — smoke beats the suite:
```bash
# run a local script on the server's data via the corpus venv, no checkout needed
ssh dankserver '/home/dank/venvs/radcorpus/bin/python - --corpus <corpus> --out /tmp/<x> --force' \
< scripts/rechunk-kld-corpus.py
```
Import-level helpers can be exercised with
`importlib.util.spec_from_file_location` (e.g. a new `tap_bytes()`: empty→0,
one file→its size, absent→0).
6. Stacked PRs: a review commit goes on its own branch off the open one; a diff
that includes the parent commit is a stacked PR, not a mistake — say which PR
merges first in the body.
## Report
One table of defect → cost → fix. Name the failure the defect produced in
practice (hours wasted, wrong number), because that is what makes it reviewable.
@@ -0,0 +1,49 @@
---
name: radiance-calib-tap-produce-latest
description: "Produce the radiance calibration tap (layer Hessians for GPTQ calib=) on released radiance: which image actually arms it, the only shape that fits 2x R9700, the floor, the seal contract, and the two silent traps. Use when a w4-MoE quant needs a tap, not when merely verifying a build writes."
---
Friction-earned 2026-10-30 on dankserver (2x R9700 32 GiB, qwen3.8-flash-next bf16 container, campaign `/home/dank/campaign/qwen38-w5nl64`).
## The image decides whether the tap exists
`RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS` arm the tap in the **source** (`mc.calib = calib_dir() != nullptr`, identical in v1.8.1 and v1.14.0 trees), but a boot either prints
```
I[0] recorded passes: off, the program accumulates calibration Grams (gram_accum)
```
or it does not — and if it does not, the tap writes **zero bytes for the whole run** while `/health`, `/stats` and the pusher all look healthy. Measured: production pin 1.8.1 = 17 hours, 0 bytes; `:latest` (v1.14.0, revision `4a61d7296bc5`, digest `69add58f7de1…`) on the same argv/env = sealed in minutes.
**Procedure:** boot the candidate image with the calib env and grep its log for `gram_accum` **before** pushing anything. Absence = wrong build; do not spend a window discovering it.
## The vendor calibration shape does not fit two 32 GiB cards
`docs/MOE-W4.md` prescribes `--max-num-batched-tokens 65536 --max-num-seqs 64` (~930 tok/s). On 2×32 GiB R9700s (27.95 GiB claimable) it dies in the VRAM resolver — arena alone 12.83 GiB; variants came up short by 3.19 GiB, 279 MiB, 59 MiB, 86 MiB, then a starved cache. Wider steps are refused by the arch before memory even matters:
```
radiance: qwen4exp: the QSA score plane would be 3125 MiB -- 16384 rows (--max-num-batched-tokens 16384)
by --max-model-len 200000 / compress ratio 4. Lower either.
```
What boots and seals: **4 slots, `--max-num-batched-tokens 2048`** (the quantised file tier's ceiling), `--kv-cache-dtype fp8`, `--weights-disk-tier`, `--placement expert_tiered`, `--ngram-placement disk`, `--max-model-len 200000` (pool 199,948 tokens: `kv cache 1.50 GiB … 1.0 full-length`). Prefill lands at 50–70 tok/s; the tap's cost is expert re-streaming per step, not the engine.
## Concurrency: 4 stalls, 2 survives
With `--parallel 4` the pusher sends four ~7.8 K-token prefills; **two complete (7,335 and 8,054 tokens), then a step never returns**: log frozen, `W step: nothing has been logged for 21 s` repeating, all threads in `futex`/`kfd_wait_on_events`, 0 MiB/s of reads, `/health` still 200 and `/stats` still `running: 4` with the pool 98 % idle. Reproduced on both a 32,768 and a 200,000 window, so it is not the pool. Production restarts fine afterwards (`chat 200` in 0.49 s) — the card is not wedged. Treat a frozen engine log with live requests as a hang, not a slow rig, and drive the pusher at `--parallel 2`.
## Floor and seal contract
- `RADIANCE_CALIB_TOKENS=1500000` is enough: the vendor's own measurement is that a pooled Gram from 10 M tokens beats one from 1 M by under 1 %; 6 M cost 29 h on this rig.
- One seal writes **~40 GiB** — 24,768 files `gram.r<rank>.blk.<l>.ffn_down_exps.<e>.bin`, 42.9 GB. Give the tap dir ≥60 GiB; a 37 GiB filesystem filled to zero and every later write came back short.
- The engine **exits by itself when the tap seals**, so a mid-flight client gets `429 rate_limit_error: the server is shutting down`. That is success.
- Grams are written only at the seal: a restart before it loses the accumulation but no artifact, so a self-healing driver (kill rig, reboot, re-push) is safe.
## Two traps that read as successes
- **A port that answers is not your rig.** After the runner stops production, the dying process still answers `:7770` for seconds; a health-only gate "passes" at 10 s on a container that is not up, and the pusher then refuses on container name (or worse, pushes into production). Gate on `podman ps` **naming the container**, then health.
- **A container-based GPU tool must own the cards.** An orphaned engine keeps `/dev/kfd` and the next boot dies with `E startup stopped: device error` (also produced by `kill -9`-ing an engine: let the container teardown run, or kill by exact pid from `sudo fuser /dev/kfd` — never `pkill -f`, which also matches your own command line).
## Downstream
The tap is consumed by `rad-convert` via `gptq:calib=<dir>`; the file contract (`RADGRAM2`, `gram.r<rank>.<stem>.bin`) is version-stable, so a latest-written tap converts on latest. `scripts/grams-to-imatrix.py <calib> -o imatrix.gguf` turns it into an imatrix. Arm output size is ~the served container's size (114 G here) — check free space before converting, not after.
@@ -0,0 +1,71 @@
---
name: radiance-calibration-tap-shape
description: "Run and measure a radiance GPTQ calibration tap on a disk-tiered BF16 rig — n-gram tier, step sized to the document, tap marker gates, and reading the real rate from /stats instead of the log."
---
# Running and measuring a radiance calibration tap
A calibration tap (`RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS`) boots a BF16 reference container
on the serving cards under `--weights-disk-tier` and prefills a corpus until the pooled token floor
is reached, writing per-expert Grams for GPTQ. It is the one campaign phase whose cost is dominated
by **disk read volume**, not by compute, and the shape that serves production is the wrong shape for
it.
## The four things that decide the tap's wall time
1. **N-gram table tier.** On the BF16 payload the table is tens of GiB and `--ngram-placement ram`
PINS it. Beside a full host pool that leaves a few GiB of RAM and the boot dies at
`startup stopped: out of memory` while the same pool and slot count had already booted the
reference recording. Calib only needs the routed weights prefilled, so pass
`--ngram-placement disk` for calib and keep `ram` for the KL modes — their numbers are only
comparable at an identical shape.
2. **Step ≥ document length.** A document cut into N steps re-streams each layer's routed expert
union N times from the container. With `--max-num-batched-tokens` below the document length the
tap runs several times slower than with one wide step, and the KV pool does not change at all
(`kv_tokens_total` is unaffected). This is legal for calib and NOT for a scoring boot: scoring
numbers are only comparable at one shape, so the step is calib's own knob.
3. **KV pool, not slots.** When the pool holds barely one document, extra `--max-num-seqs` only
queue (and preempt) requests; raising slots does not raise throughput.
4. **The token floor is pooled, not per round.** One corpus pass may be smaller than the floor, so a
round that seals no bytes is NORMAL — a driver that treats a flat round as failure aborts one
round early. Fail on a tap that *loses* bytes; let "never sealed" fail at max rounds.
## Never push a tap that belongs to no rig
A tap dir is a directory; production is also an endpoint on the same port. Without proof, a pusher
will happily feed millions of tokens into whatever answers and record nothing.
- The rig writes a marker into the tap dir at boot (`container=`, `image=`, `pid=`), retires it
when its boot exits non-zero, and the pusher refuses unless the marker exists, names the
container it targets, and that container is running.
- The window script stops on a failed phase instead of falling through its health wait, and banks a
large reference artifact only when both phases exit 0 and the tap holds files.
## Measuring the rate (do not read it off the log)
The campaign log is `tee`'d and block-buffered, so its request count lags by minutes.
```bash
curl -s http://127.0.0.1:7770/stats | python3 -c 'import json,sys; d=json.load(sys.stdin); \
print({k: d[k] for k in ("uptime_s","prefill_tps","steps_prefill","total_requests","preemptions","kv_tokens_total")})'
```
`prefill_tps` is CUMULATIVE over uptime — `tokens ≈ prefill_tps × uptime_s`, so take two samples a
couple of minutes apart and difference them. `steps_prefill / total_requests` proves whether a
document is one step or many. Discard the first minutes after boot (warmup and the first requests
are not the steady state).
## Sizing the tap before paying for it
Probe one 5-minute sample at the candidate step, compute the ratio against the inherited shape, and
choose the floor from the measured rate — not from an earlier campaign's runtime on a different
corpus and shape. Then set `RADIANCE_CALIB_TOKENS` (via the runner's env knob) and launch ONE run;
if the probe already beat the bar, that probe run IS the real run and needs no restart.
## Deliverables that make it unattended
- a gated window script: boot → health with a hard timeout → push → stop → restore the service →
bank the reference artifact only on success;
- a pusher that is marker-gated and progress-aware;
- a ledger line recording the shape used and the target chosen, so the next session does not
re-derive it.
@@ -0,0 +1,68 @@
---
name: radiance-disk-tier-throughput-census
description: "Measure a radiance rig's real prefill rate and prove whether the bottleneck is disk, CPU or shape — for calibration taps, KLD recording or score boots; use when a rate looks wrong, an ETA must be quoted, or someone asks about CPU allocation."
---
# Census a radiance rig's throughput without lying to yourself
Ask three questions in order: what is the rate, what is the binding resource, and does the shape
explain it. Every number below comes from a kernel or engine counter — never from a gauge.
## 1. The rate: counters, never `prefill_tps`
`/stats` on :7770 exposes `prefill_tps`, and it is a **rolling gauge**. Reading 69.5 → 149.1 across
120 s and multiplying by `uptime_s` produced a bogus "320 tok/s / 10.4 h" verdict that had to be
retracted (measured 2026-10-29). Derive the rate from counters over a ≥240 s window:
```bash
python3 - <<'PY'
import json, time, urllib.request
def stats(): return json.load(urllib.request.urlopen("http://127.0.0.1:7770/stats", timeout=10))
a = stats(); t0 = time.time(); time.sleep(240); b = stats(); t1 = time.time(); dt = t1 - t0
docs = b["total_requests"] - a["total_requests"]
steps = b["steps_prefill"] - a["steps_prefill"]
MEAN = 4238.0 # tokens/doc, from the corpus: total_tokens / documents
print("docs %d in %.0fs -> %.0f tok/s | steps/doc %.2f" % (docs, dt, docs * MEAN / dt, steps / docs))
PY
```
`steps/doc` is the read-amplification signal: a document cut into N prefill steps re-streams each
layer's routed expert union N times when the weights are on the disk tier, so N > 1 is leaked time.
## 2. The binding resource: diskstats and the idle/iowait split
```bash
# per-device read bandwidth (parent device, not the partition)
awk '{print $3, $6*512}' /proc/diskstats | grep -E "^nvme"
# host busy = total - idle - iowait (summing /proc/stat's first 8 fields WITHOUT subtracting is
# the classic error: the sum already covers idle, so it reads ~800% on an 8-core box)
```
Then split CPU by process: sample `/proc/<pid>/stat` fields 14+15 (utimes + stimes) twice.
Two hard readings that decide the argument:
- `radiance` compute at ~0.26 cores while the disk delivers ~3.2 GB/s and `iowait` is ~29 % of the
box ⇒ **disk-bound**. More cores or a `--cpuset-cpus` pin cannot add bytes; pinning only caps it.
- A container with `cpu_quota=0` and an empty `CpusetCpus` (check `podman inspect -f
'{{.HostConfig.CpuQuota}} {{.HostConfig.CpusetCpus}}'`) is already unrestricted — "allocate more
CPU" is a no-op, and "allocate 6 of 8" is a ceiling.
Sanity relation: tok/s × (GiB streamed per document ÷ tokens per document) should match the measured
device bandwidth. If it does not, the shape, not the hardware, is the story.
## 3. The shape: what may legally be widened
- `kldref` and `score` must keep production's `--max-num-batched-tokens` and `--max-num-seqs`: their
KL numbers are only comparable across boots at an identical shape (the audit law).
- A **calibration tap** is exempt — its output is layer Hessians, nothing compares it. Give it a step
that covers a whole document (`Q38_CALIB_MNBT` ≥ the corpus window) and read `steps/doc`: on a
7,168-token corpus, 2,048 → 8,192 moved 2.5 → 1.00 steps/doc and 94 → 113 tok/s (**1.2×**, not the
4× the step-count ratio suggests — total bytes are unchanged, so do not promise more).
- The KV pool, not the step, bounds concurrency: when `kv_tokens_total` ≈ one document, requests queue
and preempt; extra slots buy nothing.
## Reporting
Quote tok/s, steps/doc, device GB/s and the busy/iowait split together, plus the shape each number
was taken at. A rate without its shape is not a measurement; a gauge without its window is not a rate.
@@ -0,0 +1,81 @@
---
name: radiance-tap-arm-and-seal-verify
description: "Prove quickly whether a radiance build writes calibration Grams (arming line, 8k-floor seal, engine exits on seal, 43 GB seal, GPU-exclusivity preflight, pgrep/health gate traps)."
---
# Verify a radiance calibration tap in 15 minutes
Prove whether a radiance build writes calibration Grams (`RADIANCE_CALIB_DIR`) before
committing hours of drained serving to it. The whole answer is two log lines and a
directory size.
## The two lines that decide it
1. **Arming** — on the boot ledger, rank 0:
`recorded passes: off, the program accumulates calibration Grams (gram_accum)`
Absent = the tap is not in the program: nothing will ever be written, and
`/health`, `/stats` and a healthy push prove nothing. A 17-hour run on the
production pin (radiance 1.8.1) wrote **zero bytes** with no arming line.
2. **Seal** — when the pooled token floor is reached:
`radiance: calibration wrote layer Hessians after N tokens to /data/calib`
printed once per rank. Then the directory holds `gram.r<rank>.<block>.<stem>.bin`
files (~24,768 files, ~43 GB). Check `du -sb`; a plausible-looking empty dir is
the failure mode.
## The seal ends the server
The engine **shuts itself down** once it seals. In-flight requests then receive
`429 rate_limit_error: the server is shutting down`. That is SUCCESS, not an error:
wait for the seal line, and treat a post-seal 429 as done. Do not retry into it.
## Minimum proof run
- Boot the candidate image with `RADIANCE_CALIB_DIR=/data/calib` and
`RADIANCE_CALIB_TOKENS=8192` (a small floor, not the campaign floor).
- Push three or four ~7k-token documents concurrently; at ~64 tok/s aggregate the
seal lands in ~2 minutes.
- One seal costs ~43 GB: point the tap at a filesystem with ≥60 GB free and check
before booting.
- Consumer side is version-stable: `rad-convert` takes `gptq:calib=<dir>` and reads
`RADGRAM2` grams written by any pin, so the producer and the converter may be
different builds.
## Shape: the vendor recipe does not fit two 32 GiB cards
Vendor `docs/MOE-W4.md` prescribes `--max-num-batched-tokens 65536 --max-num-seqs 64`
(~930 tok/s). On two 32 GiB R9700s the activation arena alone is 12.83 GiB of the
27.95 GiB claimable and every variant stops in the VRAM resolver.
The qwen4exp arch prices a **QSA score plane** of `rows × window / compress`:
`the QSA score plane would be 3125 MiB -- 16384 rows by --max-model-len 200000 / 4.
Lower either.` → 16,384 rows is refused outright; 8,192 rows costs ~1.56 GiB.
The shape that fits and seals: production's own — 4 slots, `--max-num-batched-tokens
2048`, fp8 KV, `expert_tiered` + weights disk tier, n-gram table on disk.
## Exclusivity is not optional
A leftover calibration container silently owns the cards and poisons every later
boot with `E startup stopped: device error`. Before any boot:
```
sudo fuser /dev/kfd # must print nothing
podman ps --format '{{.Names}}'
```
After stopping a rig, `podman rm -f <name>` **and** kill any `radiance` pid still
holding `/dev/kfd` (`kill -9` by the pid from `fuser`, never `pkill -f`). Then verify
`fuser` is empty before booting the next shape — a 21-second `device error` at boot
is a dirty device, not a shape refusal.
Two gate traps, both measured:
- `curl /health` right after stopping the service can be answered by the **dying**
production process, so "up at 10 s" is a lie. Gate on the container name in
`podman ps` **and** the marker file the rig writes, not on the port alone.
- `pgrep -f "script.sh"` matches the polling shell's own command line; a
`while pgrep -f` wait loop then never exits. Gate on a file, a container name, or
the pid you captured at launch.
## Report
Write the outcome into the page/README that owns the campaign, not a diary: which
build arms the tap, the fitting shape, the floor and its warrant, and the seal size.
Numbers from a shape that is not running now are dead weight — keep the law.
@@ -1,18 +1,39 @@
---
name: radiance-w4-campaign
description: "Run a radiance GPTQ-w4 quantization campaign on dankserver: calibration → protected experts → convert → KLD gate → deploy, with the engine-pin and MNBT laws."
description: "Run a radiance GPTQ-w4 quantization campaign on dankserver: calibration tap pin and engagement proof, protected experts, convert, KLD gate, deploy — plus the tap-disk and orphan-engine laws."
---
Pipeline for quantizing a BF16 Qwen3.8-Flash-Next checkpoint into a servable `qwen4exp` `.rad` on dankserver (frank-primary). Reference run: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`; scripts in `scripts/` (`run-swift15-bf16.sh`, `calib-push.py`, `derive-protected-experts.py`, `apply-protected-cast.sh`, `convert-swift15-w4dn8.sh`, `kld-gate-swift15.sh`, `fleet-probes-swift15.sh`).
Pipeline for quantizing a BF16 MoE checkpoint into a servable `.rad` on dankserver (frank-primary). Reference run: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`; scripts in `scripts/` (`run-swift15-bf16.sh`, `run-<model>-bf16.sh`, `calib-push.py`, `derive-protected-experts.py`, `apply-protected-cast.sh`, `convert-<model>.sh`, `kld-gate-<model>.sh`, `fleet-probes-<model>.sh`).
Sequence:
1. **Fetch** checkpoint with per-shard LFS sha256 verification (`-f`, byte counts, no token on argv).
2. **BF16 container**: `rad-convert` no-recipe (pure copy) — serves as the KLD reference engine.
3. **KLD reference**: record logits on the SERVE pin (1.0.13) at MNBT 32768, `--max-model-len 16384`, corpus ≤16k prompts. Writes atomically at completion (`rows:` line = receipt). Never scp a running bash script (lazy-read law).
4. **Calibration**: engine **1.2.0** (only pin with `RADIANCE_CALIB_TOKENS`); push ~10M tokens of the model's own distribution; per-expert down-proj Grams land as RADGRAM2 files (`gram.r<rank>.blk.L.ffn_down_exps.<e>.bin`). Grams are engine-invariant across pins — the version boundary is by design.
3. **KLD reference**: record logits on the SERVE pin at the production geometry, `--max-model-len` under the pool, corpus ≤ that window. Writes atomically at completion (`rows:` line = receipt).
4. **Calibration tap** — see the pin law and the engagement test below.
5. **Protected experts**: derive from gram energy (share ≥3 % AND ≥3× routing share), splice between `# PROTECTED-BEGIN/END` markers of the recipe. Plugin law: ONE encoding per expert table; only whole bf16 experts legal beside it — no gate_up/down split.
6. **Final convert** on the SERVE pin (CPU GPTQ, hours; disk gate ≥146 GiB free first).
7. **KLD gate** on the serve pin: **MNBT ≤ 2048** (w4 streams in place only there; 32768 → `DID NOT FIT ... stored form is not the container's`), pool 40960, fp8 KV matching the production KV dtype. Gate: mean KL ≤ 0.03, top-1 ≥ 95 %.
8. **Deploy** `./scripts/deploy.sh dankserver --service frank-primary`; verify `/v1/models` id = served-model-name (alias independent of file — harness models.yml untouched), boot ledger, streamed ≥128-token decode probe, fleet probes (vision needs ≥256-token budget + `reasoning_content` fallback).
7. **KLD gate** on the serve pin at the production MNBT (a w4 does not stream in place outside the shape it was measured at), pool and KV dtype matching production. Gate: mean KL ≤ 0.03, top-1 ≥ 95 %.
8. **Deploy** `./scripts/deploy.sh dankserver --service frank-archon`; verify `/v1/models` id = served-model-name, boot ledger, streamed ≥128-token decode probe, fleet probes (vision needs ≥512-token budget + `reasoning_content` fallback).
Laws: stop frank-primary for every GPU phase (relinked unit: stop, mask fails); verify docs against `podman inspect <name> --format {{join .Config.Cmd " "}}` (rootless, as dank) before writing shapes; ff-pull fails on untracked files the merge tracks — trash first.
## The tap pin law (measured 2026-10-30, cost: one 17-hour night)
`RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS` arms the per-expert tap on the **1.2.x** line only.
- Every plugin from 1.2.4 through 1.8.1 still **contains** the machinery (`CalibExperts`, `RADGRAM2`, `calib_write_host`, the seal line `calibration wrote layer Hessians after %lld tokens`), so `strings` proves nothing about the pin.
- A runner that boots `FRANK_ARCHON_IMAGE` (1.8.1) for `calib` serves perfectly — `/health` 200, 1,700 documents at 118 tok/s, `.rig-boot` written, no error anywhere — and writes **zero bytes**: the tap never arms.
- The engine exposes **no tap counter** in `/stats` and prints **no arming line** in the boot ledger, so a dead tap is indistinguishable from a slow one for hours.
- Default `calib` to the 1.2.0 digest (`stilldeadcode/radiance@sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa` on dankserver as of 2026-10-30); override only by naming `--image` on purpose.
**Engagement test — 3 minutes, run it before any long tap:** boot with `RADIANCE_CALIB_TOKENS=8192`, push 2–3 real corpus documents, then require the seal line *and* bytes in the tap dir. On 1.2.0 that reads `calibration wrote layer Hessians after 10207 tokens to /data/calib`. No seal line in ten minutes = the pin or the arming is wrong; stop.
## The tap's disk and process laws
- **One seal writes ~40 GB** (39.4 GiB measured for an 8,192-token floor): the output is expert-shaped, not token-shaped. Put the tap dir on a volume with ≥60 GiB free and gate it before boot (`Q38_CALIB_MIN_FREE_MIB`). On a 37 GiB root fs it filled `/` to **0 bytes**; the engine's later writes came back `cannot write` / `write was short`, and *other* tooling on the host then failed too.
- **Removing the container does not kill its engine.** `podman rm -f <rig>` leaves `podman-init -- radiance …` alive holding `/dev/kfd`; production then fails `ar_oneshot_2rank_wht6 … device error` at kernel init, exits **0**, and `Restart=always` loops it (counter reached 100) while `/health` never answers. Diagnose with `sudo fuser -v /dev/kfd`; kill **by pid** from `fuser /dev/kfd` + `/proc/<pid>/comm == radiance`, never `pkill -f` (the pattern matches the caller's own argv). Put that gate in the runner's restore trap.
- The service stop/restore path needs `sudo systemctl` — polkit denies the unprivileged call intermittently (`Access denied … interactive authentication`), and a failed stop means the rig boots *beside* production.
## Rate measurement
`/stats` `prefill_tps` is a **rolling gauge**, never a cumulative counter: `gauge × uptime` invents rates (it once produced a bogus 4.5×). Derive tok/s from `total_requests` deltas × the corpus mean tokens/document, or from `steps_prefill`.
Laws: stop production for every GPU phase; verify shapes against `podman inspect <name> --format {{join .Config.Cmd " "}}` (rootless, as dank) before writing them; ff-pull fails on untracked files the merge tracks — trash first.
@@ -0,0 +1,77 @@
---
name: script-refactor-parity-vs-previous-revision
description: "Prove a refactor of sibling data-pipeline scripts is behavior-identical: run the pre-refactor revision from git HEAD beside the new one on the host that owns the venv/data, compare parsed records rather than bytes, and bootstrap the shared module so file, runpy and container paths all import it."
---
Refactoring a pipeline of sibling scripts (corpus builders, re-chunkers, pushers) whose only
tests are smokes. The test suite cannot see a moved record boundary, so parity has to be
*measured against the previous revision*.
## 1. Keep the old revision runnable
Never trust the diff. Materialize both sides where the data lives (the host with the venv,
tokenizer, parquet, corpus):
```sh
git show HEAD:scripts/<tool>.py > /tmp/qc-old/<tool>.py # pre-refactor
cp scripts/<tool>.py scripts/<newmodule>.py /tmp/qc-new/ # post, module beside it
```
`ssh host 'cat > /tmp/...'` with the bytes on stdin is enough; no repo on the target needed.
Run both with **identical seeds and flags**, into separate output dirs.
## 2. Compare records, not bytes — unless the escaping is unchanged
- If the refactor funnels writes through one helper, that helper's `ensure_ascii` choice moves
every non-ASCII character. A byte diff then reports a false regression while the rows are
identical.
- Prove it correctly: parse both sides and compare in order —
```python
a = [json.loads(l) for l in open(old) if l.strip()]
b = [json.loads(l) for l in open(new) if l.strip()]
print(a == b, hashlib.sha256(json.dumps(a, sort_keys=True).encode()).hexdigest()[:16])
```
- Where the refactor does not touch serialization (a tool that already wrote literal UTF-8),
demand the stronger proof: `cmp` + md5 equality of the artifact.
- Order matters: shuffled output with an unchanged seed proves the RNG draw sequence survived
the hoist of nested closures. Compare the *sorted* file only as a second check — permutation
equality alone hides a reordered `rng.choice`.
- Hashing the record set with `sort_keys=True` also catches dict-key drift in manifests.
## 3. Sibling-module bootstrap that survives every invocation
A new shared module must import when the tool is run three ways: `python3 scripts/tool.py`,
`runpy.run_path('/work/tool.py')` from a container mount (`-v scripts:/work:ro`), and from any
cwd. Standard preamble:
```python
_HERE = os.path.dirname(os.path.abspath(__file__)) if "__file__" in globals() else os.getcwd()
if _HERE not in sys.path:
sys.path.insert(0, _HERE)
import <module> # noqa: E402 - sibling module, same directory
```
Piping the script through `python3 -` has no `__file__` and defeats this — run tools by path and
say so in the module docstring. Verify both forms explicitly before claiming it works.
## 4. Unit-test the refusals without the heavy dependency
Hoist validation into a function so it can be driven with a stub, in-process:
```python
spec = importlib.util.spec_from_file_location("<module>", ".../<module>.py")
m = importlib.util.module_from_spec(spec); sys.modules["<module>"] = m; spec.loader.exec_module(m)
```
Registering the module in `sys.modules` under its real name *before* loading the dependent
script is required, or the script's own `import <module>` re-imports it and your stub is invisible.
A stub object with `encode`/`decode` that mangles ids past a threshold exercises a
"token identity" check in one page of output, no tokenizer file needed.
## 5. Gate order
`py_compile` every touched file → run the tool's `--help` through both invocation forms →
old-vs-new parity on the host → suite → commit. A suite that enumerates shell seams will not
notice a corpus-tool regression; the parity run is the only evidence there.
@@ -0,0 +1,100 @@
---
name: sftp-client-unreachable-triage
description: "Diagnose \"SFTP/SSH from box A to box B doesn't work\" when box A is unreachable from the fleet: exonerate the server with a log+key census, then discriminate sleep vs firewall vs name-resolution vs missing key from outside."
---
# Triage a client-side SFTP/SSH failure (client box unreachable)
Use when Frank reports "FileZilla/SFTP/scp from <client> to <server> isn't working" and the
client box is not reachable from the fleet. Produces a ranked, evidence-backed verdict
without access to the client. **Read-only probes only**; no restarts, no config writes.
## Order of operations (each step can end the investigation)
### 1. Server-side exoneration — gather before theorising
```bash
ssh -o BatchMode=yes <server> 'sudo -n sshd -T | grep -Ei "^(port|listenaddress|pubkeyauth|passwordauth|kbdinteractive|allowusers|subsystem|authorizedkeysfile)"'
ssh -o BatchMode=yes <server> 'ls -l /usr/lib/openssh/sftp-server; sudo -n ls -l /etc/ssh/ssh_host_*key.pub' # host-key mtime = known-hosts churn check
ssh -o BatchMode=yes <server> 'sudo -n iptables -S | head; sudo -n nft list ruleset | head -20' # INPUT ACCEPT = no firewall
```
Host keys unchanged for years ⇒ FileZilla cannot be tripping a host-key warning.
### 2. Censuses that turn "maybe" into "never" (the payload of this skill)
```bash
# keys trusted by the server, by comment
ssh <server> 'ssh-keygen -lf ~/.ssh/authorized_keys'
# every IP that ever authenticated, and with which key
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
| grep -oE "Accepted publickey for [a-z]+ from [0-9.]+" | awk "{print \$NF}" | sort | uniq -c'
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
| grep -oE "ED25519 SHA256:[A-Za-z0-9+/=]+" | sort | uniq -c'
# did the client IP EVER appear, anywhere?
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-45 days" | grep -c "<client-ip>"'
ssh <server> 'sudo -n zgrep -h "<client-ip>" /var/log/auth.log* | grep -v sudo | tail'
```
Count 0 across journal **and** rotated logs ⇒ no TCP ever established ⇒ the failure is
DNS, client egress, or a sleeping host — **not** credentials. Say so explicitly, because
"permission denied" and "connection timed out" have different fixes.
### 3. Is the client even at the IP everyone believes? (DHCP drift)
```bash
avahi-resolve -n <Client>.local # mDNS truth
ip neigh | grep -E '\.(old|new) ' # FAILED vs REACHABLE
nmap -sn -n 192.168.0.0/24 # works unprivileged; arp-scan needs root
```
macOS **Private Wi-Fi Address** rotation changes the MAC ⇒ new DHCP lease ⇒ every
hardcoded IP for that box rots. Check both old and new addresses for mentions in
`/var/log/auth.log*` and `journalctl`.
### 4. Discriminate sleep vs firewall (ARP/mDNS alive, everything else dead)
- ARP replies + live mDNS but **zero** ICMP and **zero** TCP on a spread of ports
(22, 80, 445, 548, 5900, 8080, 5000, 7000, 3283) from **multiple** hosts including a
wired one: probe from a wired box on the same L2 — that also rules out Wi-Fi AP isolation.
- Same result ⇒ either the host is asleep (Bonjour sleep proxy / AP answers ARP+mDNS on its
behalf) **or** a host packet filter (macOS "Block all incoming", LuLu/Little Snitch,
ProtonVPN kill switch) is dropping it.
- **Do not overclaim which.** State both, and note the built-in macOS firewall only blocks
*inbound* — so if Frank is actively using that Mac, a built-in-firewall verdict cannot
explain an *outbound* SFTP failure; a third-party filter/VPN or DNS is then the cause.
### 5. Name resolution (the silent failure)
```bash
getent hosts <server> <server>.local <server>.<domain>
dig +short <server> @<pihole-ip>
```
Traps seen on this fleet:
- Bare `dankserver` resolves **nowhere** — no Pi-hole custom record, gateway DNS refuses
client queries. Only mDNS `<host>.local`, which may advertise a **link-local IPv6 only**.
- `dankserver.franksdank.cloud` resolves to **Caddy (192.168.0.143)**, not the SSH box —
a wildcard vhost. Never suggest a `*.franksdank.cloud` name for SSH/SFTP.
- **FileZilla ignores `~/.ssh/config`** — `Host dankserver` aliases do not apply; the Host
field needs a literal IP that resolves.
### 6. Fix recipe handed to the user (client side)
```bash
ping -c2 <server-ip> && nc -vz <server-ip> 22
ls ~/.ssh/id_ed25519.pub || ssh-keygen -t ed25519 -C "<client-name>"
ssh-copy-id -i ~/.ssh/id_ed25519.pub dank@<server-ip>
sftp -i ~/.ssh/id_ed25519 dank@<server-ip>
```
FileZilla: Protocol **SFTP**, Host literal IP, Port 22, Logon Type **Key file** →
`/Users/dank/.ssh/id_ed25519` (or Normal + password). The "Encryption" dropdown is FTP-only
noise for SFTP sites.
### 7. Report collateral you proved, don't guess it
Dead client IP ⇒ grep the fleet for hardcoded references and *observe* the breakage:
```bash
grep -rn --exclude-dir=.git "<old-ip>" ~/Projects/serverAppConfig | wc -l
curl -sk -o /dev/null -w '%{http_code}\n' --resolve "radarr.franksdank.cloud:443:<caddy-ip>" https://radarr.franksdank.cloud/
```
(`-H Host:` alone gives `000` — curl sends the IP as TLS SNI and the handshake fails. Use
`--resolve` to keep SNI correct before concluding anything about a vhost.)
## Laws
- Zero packets at the server beats any credential theory; log census first, key census second.
- ARP/mDNS answering is **not** proof a host is awake; the AP or a sleep proxy can answer.
- Probe from a wired host before blaming Wi-Fi isolation.
- A rotating private Wi-Fi MAC will re-break every hardcoded IP: recommend a DHCP
reservation as the fix, not 40 file edits.
@@ -0,0 +1,34 @@
---
name: upstream-engine-reference-into-vault
description: "Author a multi-page reference for an upstream engine into the serverAppConfigs OKF vault, passing the docs seams."
---
# Upstream engine → vault reference set
Use when asked to "deeply read and document" an upstream inference engine (radiance, llama.cpp, halogen) into `serverAppConfigs-docs`. Distill a repo's docs + ABI + option registry into progressive-disclosure pages, not a transcript.
## Read order (source beats docs)
1. `README.md`, then `docs/` TOC; read the serving guide, tools reference, plugin guide, architecture guide, op vocabulary.
2. Find the **authoritative option table** in source (`core/config.cpp` for serve flags, `tools/<tool>.cpp` `usage()` for tools, `abi/*.h` for plugin fields). Docs lag: radiance's `TOOLS.md` omitted `--chat-template`, `--lora`, `--lora-map`.
3. Version/ABI: `CMakeLists.txt` project VERSION and `#define RAD_ABI_VERSION`; these are independent.
4. Read one complete real plugin/example end to end (e.g. `plugin/spechub/exports.cpp`, `arch/llama_fp8/llama_fp8.cpp`) before writing "how to write a plugin".
## Page set (one owner per fact)
Hub + Options + HTTP API + Conversion + Model Families + Plugins + Architecture Plugins + Tools + Operations. The engine pages own engine defaults/flags/ABI; the fleet endpoint page (e.g. `LLM/frank-archon.md`) keeps live shape, Eras, measured laws, rollback. Link, never restate.
State the upstream commit + release + ABI, and separately the fleet pin; a flag newer than the pin is a boot refusal, not a no-op.
## Vault mechanics that fail the seams
- Frontmatter: `type` ∈ reference/howto/tutorial/explanation, `description` non-blank, ≤120 chars, not the title, no fleet quantity; ISO `timestamp` not future.
- Every page ≤500 lines. Split rather than exempt.
- Tables: escape `|` inside cells as `\|` (pipes inside backticks still split GFM tables).
- Register pages: edit the directory `index.md` rows with the **exact** description, then `python3 scripts/gen-vault-index.py serverAppConfigs-docs` (must be a no-op on a second run).
- Run: `check-okf.py`, `check-vault-links.py`, `bats tests/docs_completeness.bats`; then `./tests/run.sh`.
- Do not add a second log/status file; the vault's `log.md` policy is owned by the repo.
## Commit
Branch `docs/<slug>` off `origin/main`; commit docs + generated indices together. If main moved, merge it (never rebase) and re-run the full suite. Report commit hash, page list, test evidence; ask before opening a PR.
@@ -1,6 +1,6 @@
{
"v": 1,
"lastConsolidatedAt": 1793308587833,
"lastConsolidatedAt": 1793314235443,
"lastResult": {
"at": 1791599865775,
"sessions": 1,
@@ -1,4 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793215693649
"lastConsolidatedAt": 1793396927634
}
@@ -1,3 +1,4 @@
## Structural Standards
Quant quality must hold regardless of implementation choice.
Quant quality must hold regardless of implementation choice.
Container versions must be digest-pinned to latest snapshot.
@@ -1,7 +1,5 @@
## Product Quality Standards
Quant quality must hold regardless of implementation choice.
## Behavioral Updates
Hourly updates on a deterministic timer.
Merge any fixes immediately.
@@ -1,8 +1,8 @@
{
"v": 1,
"lastConsolidatedAt": 1793309257223,
"lastConsolidatedAt": 1793398614775,
"lastResult": {
"at": 1793218183375,
"at": 1793397014344,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
@@ -0,0 +1,4 @@
## Models
- Models settings include /switch to switch to it
- hdmi monitor to the left and the other to the right of eDP-1
@@ -1,4 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793309241992
"lastConsolidatedAt": 1793312076890,
"lastResult": {
"at": 1793312076890,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
}
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793310990286
}
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793315256444,
"lastResult": {
"at": 1793314927653,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
}
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793322540660
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793398691951
}
@@ -0,0 +1,3 @@
## Runtime Decisions
Avoid subagent spawns when GPU saturation occurs on Dankserver.
@@ -0,0 +1,3 @@
## Reliability
No reboots on deployed containers.
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793398788553,
"lastResult": {
"at": 1793397235249,
"sessions": 1,
"deltas": 14,
"model": "frank-zealot"
}
}
@@ -107,11 +107,11 @@ widget_order = [
[lockscreen_widgets.widget."lockscreen-login-box@DP-2"]
box_height = 196.0
box_width = 810.0
cx = 1719.999755859375
cy = 1221.60009765625
cx = 599.9998779296875
cy = 1628.8001708984375
output = "DP-2"
placement_height = 1440.0
placement_width = 3440.0
placement_height = 1920.0
placement_width = 1200.0
rotation = 0.0
type = "login_box"
@@ -279,22 +279,22 @@ transition = [ "fade" ]
interval_seconds = 1200
[wallpaper.default]
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-mlomkm.png"
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gw26v3.png"
[wallpaper.last]
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-mlomkm.png"
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gw26v3.png"
[wallpaper.monitors.DP-1]
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-mlomkm.png"
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gjqmel.png"
[wallpaper.monitors.DP-2]
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gjqmel.png"
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gw26v3.png"
[wallpaper.monitors.HDMI-A-1]
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gjqmel.png"
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gw26v3.png"
[wallpaper.monitors.eDP-1]
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-mlomkm.png"
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gw26v3.png"
[weather]
unit = "imperial"
@@ -1,18 +1,50 @@
// ────────────── Desk Profile: portable (skeleton) ──────────────
// SKELETON — portable monitor EDID not captured yet. On first use at the
// portable desk: run `desk.sh capture`, replace PORTABLE-EDID-TBD below and
// in the requires list with the printed long name, adjust position to taste.
// ────────────── Desk Profile: portable (second desk) ──────────────
// Two PM1601 16" panels, landscape, flanking the laptop panel.
// Applied by symlinking this file to cfg/display.kdl (see scripts/desk.sh).
// Panels are keyed by EDID identity, NOT by connector name — connector names
// reshuffle between docks; identity travels with the panel. Long names copied
// verbatim from `niri msg outputs` (2026-10-30).
//
// requires: Lenovo Group Limited 0x40A9 Unknown
// requires: PORTABLE-EDID-TBD
// requires: PNP(XYM) PM1601 0000000000000
// requires: PNP(XYM) PM1601 demoset-1
//
// Layout (logical, y grows downward; three 1920-wide columns):
//
// ┌──────────┬──────────────┬──────────┐
// │ PM1601 │ │ PM1601 │
// │ 1920x1200│ eDP-1 │ 1920x1200│
// │(HDMI-A-1)│ 1920x1080 │ (DP-2) │
// │ │ │ │
// └──────────┴──────────────┴──────────┘
// x=0,y=0 x=1920,y=60 x=3840,y=0
//
// HDMI-A-1 sits to the LEFT of the laptop by request (2026-10-30); DP-2 stays
// on its right. The laptop is 60 px down so it is centred against the taller
// 1200 px panels — no dead corner where the cursor crosses.
// Modes are the panels' reported preferred modes, copied verbatim: niri
// matches mode strings exactly and silently falls back to preferred on a
// miss, so a typo here costs the refresh rate without an error.
// Reasoning: omp-docs/tools/niri/displays.md ("The config pattern").
output "Lenovo Group Limited 0x40A9 Unknown" {
mode "1920x1080@60.001"
scale 1.0
position x=0 y=0
transform "normal"
position x=1920 y=60
focus-at-startup
}
// Assumed: 1920x1080 landscape panel right of the laptop panel.
output "PORTABLE-EDID-TBD" {
output "PNP(XYM) PM1601 demoset-1" {
mode "1920x1200@59.950"
scale 1.0
position x=1920 y=0
transform "normal"
position x=0 y=0
}
output "PNP(XYM) PM1601 0000000000000" {
mode "1920x1200@59.950"
scale 1.0
transform "normal"
position x=3840 y=0
}
+16
View File
@@ -22,6 +22,22 @@ not restated here — entry, then pointer. That is what the 2026-10-21 audit pas
newest twelve entries: the facts stayed, the duplicates went, `git log --follow` keeps the
long form.
## 2026-10-30
### A desk with no matching profile is silently misconfigured
`desk-2-portable.kdl` stopped being a skeleton: the two PM1601 16" panels are
identity-keyed (`PNP(XYM) PM1601 0000000000000`, `… demoset-1`), laid out
landscape either side of the laptop, and `display.kdl` points at it. Law: a rig
with no eligible profile does not run "unchanged" — it runs whatever the *last*
profile said. Here that was the home profile, whose Sceptre identity block never
matched and whose connectors still carried `3440x1440@180.000`, so niri fell back
to preferred and every panel in force had a mode written for another desk.
`desk.sh match` reporting "no desk profile matches connected outputs" is the
defect signature, and `blankpad-desk-autoswitch` exits at its initial match —
check `is-active` after any profile edit. Page:
[tools/niri/displays.md](tools/niri/displays.md).
## 2026-10-29
### A vendor-cloud provider must be named the way the harness names it
+51 -4
View File
@@ -8,7 +8,7 @@ tags:
- monitors
- rotation
- desk-profiles
verified: 2026-10-21
verified: 2026-10-30
---
# Display Configuration
@@ -49,7 +49,7 @@ the dock and the cable, which is why the config keys on the EDID long name inste
| Panel | EDID long name | Keyed by | Notes |
|-------|----------------|----------|-------|
| Sceptre M34 ultrawide | `Sceptre Tech Inc Sceptre M34 Unknown` | identity | `mode "3440x1440@180.000"`, **deliberate** — the panel also offers `3440x1440@239.998`, `144.001` and `120.000`; 180 is driven while the link is degraded (see the law below) — `focus-at-startup` |
| PM1601 portrait | *not captured* | connector `HDMI-A-1` | dock-dependent; run `desk.sh capture` when plugged in, then re-key |
| PM1601 portrait | captured 2026-10-30 — the two panels are `PNP(XYM) PM1601 0000000000000` and `PNP(XYM) PM1601 demoset-1` | connector `HDMI-A-1` in *this* profile; identity on the portable desk | dock-dependent; the identity key is known now, so re-key the block when the panel next rotates through a dock |
| eDP-1 laptop panel | `Lenovo Group Limited 0x40A9 Unknown` | identity | scale **1.0** (deliberate; Ghostty grid at 1.25 rejected 2026-10-09), second `focus-at-startup` |
VRR: the M34 supports it (its connector today is `DP-1`); **left disabled by
@@ -58,6 +58,48 @@ decision** (2026-10-09). Add `variable-refresh-rate` per output if that ever cha
`focus-at-startup` on both M34 and eDP-1: niri focuses in config order, so
the ultrawide wins when present and the laptop panel is the fallback.
## The desk-2 rig (portable)
Laptop flanked by the two 16" PM1601 panels, all three landscape. Profile:
`cfg/desk-2-portable.kdl` (`display.kdl` symlinks to it).
```
┌──────────┬──────────────┬──────────┐
│ PM1601 │ │ PM1601 │ HDMI-A-1 1920x1200 normal x=0,y=0
│ 1920x1200│ eDP-1 │ 1920x1200│ eDP-1 1920x1080 normal x=1920,y=60
│(HDMI-A-1)│ 1920x1080 │ (DP-2) │ DP-2 1920x1200 normal x=3840,y=0
│ │ │ │
└──────────┴──────────────┴──────────┘ The laptop is 60 px down so it is
x=0,y=0 x=1920,y=60 x=3840,y=0 centred against the taller panels.
HDMI-A-1 sits to the LEFT of the laptop by request (2026-10-30); DP-2 keeps the
right. Earlier the same rig ran both panels in portrait (rot 90, 1200x1920) with
the laptop and both panels to its right — the rotation is `normal` now, so the
panels are 1920 wide and every x moved.
```
Positions are absolute in logical space, so rotating a panel changes its logical
width (1200 → 1920) and every x to its right; recompute the whole row, never just
the rotated block.
All three blocks are identity-keyed and pin their mode string verbatim. The
`requires:` list names all three panels, so the matcher only claims this profile
when the whole rig is present — unplug one 16" panel and the watcher leaves the
current layout alone rather than shuffling the survivors.
**The stale-profile trap this fixed (2026-10-30).** Before this profile existed,
the portable rig ran with `display.kdl` → `desk-1-home.kdl`, whose identity
blocks belong to the Sceptre ultrawide that is *not* connected and whose
connector-keyed fallbacks carried `mode "3440x1440@180.000"`. niri found no such
mode on a 1920x1200 panel, logged the fallback, and drove it at preferred — so
the panels looked fine while every position and refresh value in force had been
written for a different desk. `desk.sh match` reported "no desk profile matches
connected outputs" and `blankpad-desk-autoswitch.service` therefore died at its
initial match, leaving the laptop pinned to the wrong profile indefinitely.
Law: **a desk with no matching profile is not "unchanged", it is silently
misconfigured** — `desk.sh match` saying "no match" is a defect to fix before
anything else on that desk behaves strangely.
## Desk switching
```mermaid
@@ -80,8 +122,13 @@ graph LR
- **Manual override:** `Mod+Shift+D` cycles profiles (wrong auto-pick, or
same panels, new layout).
- **New desk:** `desk.sh capture` prints connected long names → copy into a
new `desk-N-<name>.kdl` (see `desk-2-portable.kdl` skeleton, portable
monitor EDID pending first capture).
new `desk-N-<name>.kdl` with one identity block per panel (template:
[the desk-2 rig](#the-desk-2-rig-portable)).
- **A dead watcher is a signal, not a crash to ignore:** the unit exits at
startup when `desk.sh match` finds no eligible profile, so "no match" on a
desk that clearly has panels means a missing profile, not an idle daemon.
Check `systemctl --user is-active blankpad-desk-autoswitch` after touching
desk profiles.
- Unit: `blankpad-desk-autoswitch.service` (systemd user,
`ConditionHost=blankpad`).
+1 -1
View File
@@ -18,7 +18,7 @@ Niri is a scrollable-tiling Wayland compositor. Windows are arranged in columns
├── autostart.kdl # Startup applications (noctalia, handy --start-hidden)
├── display.kdl # SYMLINK to the active desk-*.kdl profile (see displays)
├── desk-1-home.kdl # Desk profile: home rig (EDID-keyed)
├── desk-2-portable.kdl # Desk profile: portable monitor (skeleton)
├── desk-2-portable.kdl # Desk profile: portable rig (2x PM1601 portrait)
├── input.kdl # Keyboard, touchpad, mouse
├── keybinds.kdl # All key bindings
├── layout.kdl # Gaps, column widths, struts