diff --git a/agent/managed-skills/deploy-target-dirty-blob-audit/SKILL.md b/agent/managed-skills/deploy-target-dirty-blob-audit/SKILL.md new file mode 100644 index 0000000..7b32183 --- /dev/null +++ b/agent/managed-skills/deploy-target-dirty-blob-audit/SKILL.md @@ -0,0 +1,64 @@ +--- +name: deploy-target-dirty-blob-audit +description: "When scripts/deploy.sh on serverAppConfigs aborts with \"Your local changes to the following files would be overwritten by merge\", prove whether the target's dirty file is already-merged content before restoring it, then deploy from the main checkout." +--- + +# Deploy target dirty-file audit + +`deploy.sh ` pushes the local branch, then ssh-pulls on the target. The pull is +`git pull --ff-only origin main` on the TARGET checkout, so any uncommitted edit there +aborts the whole deploy with: + +``` +error: Your local changes to the following files would be overwritten by merge: + scripts/run-qwen38-bf16.sh +Aborting +``` + +Nothing is dispatched, so the service is untouched (no wasted outage) — but the deploy +cannot proceed until that path is reconciled. This happens when an agent edited the file +directly on the host (the repo's "never edit on the remote" law) and committed the same +bytes later from a worktree, leaving the host's HEAD behind its own working tree. + +## 1. Classify before touching anything + +```bash +ssh 'cd && git status --porcelain' # every dirty path +ssh 'cd && git hash-object + && git rev-parse :' # compare blobs +``` + +- Hash equal to the blob the pull wants (`origin/main:`, or a commit main contains) + → the edit is already merged; restoring is lossless, no trash copy needed. +- Hash matches neither HEAD nor the incoming blob → real unmerged content: `cp` it to + `.AGENTTRASH/YYYYMMDD-topic/` first, then decide. +- `D` entries under `.AGENTTRASH/` (deletions, not modifications) do not block the pull + unless those paths changed between HEAD and main. Leave them — only Frank purges trash. + +## 2. Restore, then deploy from the MAIN checkout + +```bash +ssh 'cd && git checkout -- ' +cd ~/Projects/serverAppConfig && ./scripts/deploy.sh --service +``` + +`deploy.sh` refuses with `FAIL: deploy.sh must run from the main branch.` when run from a +feature worktree — use the host's `~/Projects/serverAppConfig` main checkout, and pull it +`--ff-only` first so `[1/2] Pushing main to origin` reports `Everything up-to-date`. +A missing server-side path (`/home/dank/models`) plus an `rm -f` in the chain means the +`.rad` was already retired; confirm before relaunching anything that needs it. + +## 3. Long deploys never fit an async bash slot + +An LLM-service deploy waits up to 20 min for model load plus the Playwright gate, but an +async `bash` call is killed at 600 s. Launch it as a named service instead, then watch the +pid with a deadline-free watcher: + +```bash +bash command="cd ~/Projects/serverAppConfig && ./scripts/deploy.sh --service " \ + name=deploy- ready.log="Deployment plan" +# then: while kill -0 2>/dev/null; do sleep 20; done (timeout 0, async) +``` + +`ready.log="Deployment plan"` matches the plan banner, so readiness reports the deploy +started rather than finished; the real result arrives when the watcher completes. diff --git a/agent/managed-skills/qwen38-chain-rerun-dankserver/SKILL.md b/agent/managed-skills/qwen38-chain-rerun-dankserver/SKILL.md new file mode 100644 index 0000000..be86cf8 --- /dev/null +++ b/agent/managed-skills/qwen38-chain-rerun-dankserver/SKILL.md @@ -0,0 +1,72 @@ +--- +name: qwen38-chain-rerun-dankserver +description: "Rerun the qwen38-w5nl64 campaign on dankserver: re-chunk the KLD corpus to the real KV pool, verify the rig's gate with --dry-run, deploy, and relaunch run-chained.sh detached — plus the three state-reading traps (root vs rootless podman, unit failed during an outage, the .rc sentinel)." +--- + +# Qwen38 campaign rerun on dankserver + +Use when rerunning the `qwen38-w5nl64` campaign chain (kldref -> calib -> retire the reference `.rad`) on Dankserver, or when the chain died with `E kld: document N is tokens and the KV pool holds ` and needs its corpus fixed. + +Order is fixed: **corpus -> gate -> deploy -> chain.** Skipping the gate burns the outage window. + +## 1. Corpus must fit the pool, not the flag + +The rig's `--max-model-len` is decorative: the recorder refuses any document longer than the **KV pool the boot actually got** (printed by the boot ledger, e.g. `7,436`), so a corpus authored at a wider window scores nothing. + +```bash +ssh dankserver '/home/dank/venvs/radcorpus/bin/python \ + /home/dank/campaign/qwen38-w5nl64/rechunk-kld-corpus.py \ + --corpus /home/dank/corpora/out/kld.jsonl \ + --out /home/dank/corpora/out/kld-7168.jsonl' +``` + +Read the summary, don't trust the run: `tokens_in` must equal `tokens_out` (that equality is the whole invariant — the tool exists to prove it), `longest_document` must equal the ceiling, and `scored_positions` grows because every window now carries its own `score_from`. + +Chars-per-token is payload-specific: this file is chars/3.6, another corpus measured chars/4.97. Never size a window from chars alone. + +## 2. Gate the rig before spending the outage + +`scripts/run-qwen38-bf16.sh kldref --dry-run` runs the corpus gate and prints the argv **without touching the production unit**: + +```bash +ssh dankserver 'cd /home/dank/serverAppConfigs && \ + FRANK_QWEN38_KLD_CORPUS=/home/dank/corpora/out/kld-7168.jsonl \ + bash scripts/run-qwen38-bf16.sh kldref --dry-run' +``` + +Check the printed line `corpus fits: longest record ... inside the ` **and** that the `-v ...:/data/kld.jsonl:ro` mount names the file you intended — the rig mounts whatever `FRANK_QWEN38_KLD_CORPUS` points at, and its default is the old file. + +## 3. Deploy from a clean MAIN checkout, not a feature worktree + +`deploy.sh` refuses off-main (`FAIL: deploy.sh must run from the main branch.`). Run it from the local main checkout and name the service — a bare target is a fleet-wide drain: + +```bash +cd ~/Projects/serverAppConfig && ./scripts/deploy.sh dankserver --service frank-archon +``` + +If the remote pull aborts with `Your local changes ... would be overwritten by merge`, the remote copy is a committed blob written directly on the host. Prove it before discarding: + +```bash +git -C /home/dank/serverAppConfigs hash-object # working copy +git -C /home/dank/serverAppConfigs rev-parse : # candidate commit +``` + +Equal -> `git checkout -- ` is lossless (the blob is in the object store), then re-deploy. Unequal -> archive to `.AGENTTRASH/-/` first, then port into the repo. + +## 4. Launch the chain detached, record the pid + +```bash +ssh dankserver 'cd /home/dank/campaign/qwen38-w5nl64 && \ + nohup bash run-chained.sh >> run-chained.log 2>&1 < /dev/null & \ + echo "chain_pid=$!" > run-chained.pid' +``` + +- `run-chained.sh` is a server-side campaign artifact (not tracked in the repo); it chains the phases and must export `FRANK_QWEN38_KLD_CORPUS` for the re-chunked file. +- Move a prior `kld-ref/` into `.AGENTTRASH/-qwen38-kldref-attemptN/` first, or the new recording's artefacts mix with the old ones. +- The chain writes `/home/dank/logs/qwen38-bf16-run.rc` as its completion sentinel; watch the pid, don't poll the log. + +## 5. Reading state during the outage (three traps) + +- **Container storage is split**: `frank-archon` is a root unit -> its container appears only under `sudo podman ps`; the media stack (romm, copyparty, exporters) is rootless -> only plain `podman ps`. Each view legitimately shows "nothing" for the other half. `pgrep -af radiance` settles which runtime is live. +- **`frank-archon` reads `failed` while the rig holds the card.** The rig stops the unit; the container exits status=1 on stop, so systemd marks `failed` until the phase restores it. Confirm with `journalctl -u frank-archon -n 6` — `Stopping...` then `Failed with result 'exit-code'` means the outage, not a crash. `systemctl show -p ActiveEnterTimestamp` dates it. +- **Engine stdout lives in `/home/dank/logs/`, not the journal** (for halogen); for radiance read the chain log. diff --git a/agent/memories/sharpshooter/wt-20261027-142335-6d1fc1f-1hpiehxnq73s0/state.json b/agent/memories/sharpshooter/wt-20261027-142335-6d1fc1f-1hpiehxnq73s0/state.json index 74bdb6f..dbf1d3d 100644 --- a/agent/memories/sharpshooter/wt-20261027-142335-6d1fc1f-1hpiehxnq73s0/state.json +++ b/agent/memories/sharpshooter/wt-20261027-142335-6d1fc1f-1hpiehxnq73s0/state.json @@ -1,6 +1,6 @@ { "v": 1, - "lastConsolidatedAt": 1793208636702, + "lastConsolidatedAt": 1793212656722, "lastResult": { "at": 1793170367714, "sessions": 1, diff --git a/dotfiles/noctalia/.local/state/noctalia/settings.toml b/dotfiles/noctalia/.local/state/noctalia/settings.toml index a025eed..3b2cd4b 100644 --- a/dotfiles/noctalia/.local/state/noctalia/settings.toml +++ b/dotfiles/noctalia/.local/state/noctalia/settings.toml @@ -285,7 +285,7 @@ transition = [ "fade" ] path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gw26v3.png" [wallpaper.monitors.DP-1] - path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-mlomkm.png" + path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gw26v3.png" [wallpaper.monitors.DP-2] path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven-gjqmel.png"