Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
07768ce8b5 | ||
|
|
d845160e62 | ||
|
|
3ab48bf1f2 | ||
|
|
51f9846ac6 | ||
|
|
4085ee403f | ||
|
|
1dc01bd006 | ||
|
|
336201e814 |
@@ -1,61 +1,41 @@
|
|||||||
---
|
---
|
||||||
name: audit-subagent-deliverables
|
name: audit-subagent-deliverables
|
||||||
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), adversarial-review-report triage, resource-safety pass (fd/memory), and cutover with running-process restarts."
|
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), merge shared-function collisions, fix scope creep, verify env var propagation."
|
||||||
---
|
---
|
||||||
|
|
||||||
# Audit subagent deliverables
|
Integrating code written by subagents (fan-out implementation). The code is usually correct; the failures are structural — collisions between agents, scope creep, and assumptions that never reach the runtime.
|
||||||
|
|
||||||
A subagent that wrote code is a source of plausible output, not a verified change. Integrate it the way you'd integrate an untrusted PR — with extra suspicion, because the author is an agent that shares your blind spots and its prose is more confident than its evidence.
|
## Workflow
|
||||||
|
|
||||||
Order of operations: read everything → gates → tests (as evidence, not proof) → resource-safety pass → fix in-place → cutover.
|
1. **Read every deliverable line by line** before integrating — not the summary.
|
||||||
|
2. **Run the mechanical gates immediately**, in parallel: `bash -n` on every script; `bats tests/ 2>&1 | grep "not ok"` for the failure list, never `tail` (tail shows a passing tail and hides the red in the middle).
|
||||||
|
3. **Merge shared-function collisions.** When two agents add different versions of the same function, pick the superset and re-point the other's callers at it. (Real case: one agent's `fleet_base()` carried `-b`, the other's `resolve_fleet_base()` carried `--base-url`; resolve to the one flag set the tests use.)
|
||||||
|
4. **Delete scope creep.** Code added for a goal nobody asked for — a new env var, a new CLI flag, a new output file — goes, with its docs, unless the operator asked.
|
||||||
|
5. **Verify env var propagation.** A var the orchestrator computes must be exported before the `bash -c` / subshell that reads it, or the child silently takes a default. Check the export site, not the doc.
|
||||||
|
|
||||||
## 1. Read every produced file, line by line
|
## Audit checks
|
||||||
Full file, not a diff skim. You are the reviewer who must catch the line the author never thought about.
|
|
||||||
|
|
||||||
## 2. Mechanical gates before logic review
|
| Check | How |
|
||||||
```bash
|
|---|---|
|
||||||
bash -n <script> # syntax (scripts)
|
| Flag names real | `bash script.sh --help` |
|
||||||
bunx tsc --noEmit # types (TS)
|
| Paths exist | `test -e` each literal in the diff |
|
||||||
bun test # suite
|
| Env reaches the runtime | grep the export/inline-assignment site, then the read |
|
||||||
```
|
| Tests assert behaviour | not `[[ -n "$X" ]]`, not `test -f <new file>` |
|
||||||
Run the suite for information, not as the stopping point — it only proves the author's own scenarios.
|
| Tests not weakened | `git diff HEAD~ -- tests/` — a deleted assertion is a red flag |
|
||||||
|
|
||||||
## 3. Hunt the three classic agent defects
|
## Red flags
|
||||||
- **Quoting bugs:** `set -e` + `"${@/$p/$r}"` = substitution regex containing user input (bash can't escape `/` in a `${var/re/pat}` pattern: `/` reads as the pattern/replacement separator). Fix with a separator absent from inputs (e.g. `%`) and `printf '%q'` for shell-interpolated values.
|
|
||||||
- **`cd` assumptions:** parallel `runShell` calls share one CWD (pool mode). Any command sequence that assumes CWD (e.g. `git -C .` after a sibling's `cd`) must use explicit absolute paths.
|
|
||||||
- **Unbounded resources:** every fd opened must be closed on ALL paths (success, throw, validation fail) — `using` for fs.sinks (closes on throw; sinks are `AsyncDisposable`, not `fs.FileHandle`, no `.close()`); bounded `Map` for caches (`MAX` + FIFO eviction); explicit caps (lines read, file sizes).
|
|
||||||
|
|
||||||
## 4. Check the test file is real
|
- Test asserting a new file exists rather than behaviour.
|
||||||
Tests that assert internal state (`expect(wcwd).toBe(...)`) over output contracts; tests of mock echo; tautologies. Delete them per house rules, keep behavior contracts.
|
- Doc claiming a flag the script never defines.
|
||||||
|
- Function added with no caller.
|
||||||
|
- Doc example that contradicts the code it documents.
|
||||||
|
- Same helper defined twice with different flag spelling.
|
||||||
|
- New env var read but never exported.
|
||||||
|
|
||||||
## 5. Cutover audit (repo-wide)
|
## Re-verify after the operator's own edits
|
||||||
`grep -r` the symbol namespace the subagent touched — `wcwd`-style globals are the classic blast radius. Grep every `cd`/`process.chdir` site and verify each paired command sequence uses explicit paths. Stale comments describing the old design must go — comments that lie are time bombs for the next reader.
|
|
||||||
|
|
||||||
## 6. Cutover restarts running processes
|
Run the gates again — their edit may undo an integration (a re-typed helper, a reverted flag).
|
||||||
A code fix on disk is NOT live in any process that loaded the file at spawn (bun/node load-at-spawn). After shipping fixes: restart every long-running supervisor/daemon/UI-server that carries the affected code path; prefer takeover protocols (stale-pid detection) that make restarts cheap. Symptom of forgetting: "fixed" code producing old-code symptoms in production (observed: 812 poisoned ledger rows).
|
|
||||||
|
|
||||||
## 7. Triaging an adversarial REVIEW report (not code)
|
## Report
|
||||||
When a subagent produced findings, treat each finding as a hypothesis with a base rate ~40% (measured over 3 rounds, 2026-10):
|
|
||||||
1. **Walk every finding end-to-end against the CURRENT tree** before accepting — cite file:line yourself. Reviewers frequently read a mid-edit snapshot; many findings are stale by construction.
|
|
||||||
2. **Classification that kills most findings:**
|
|
||||||
- *Stale read* — code changed during review; verify against live lines.
|
|
||||||
- *Inverted logic* — "failed rows are skipped" when the filter makes failed rows RETRY. Re-read the filter.
|
|
||||||
- *Unreachable* — trigger requires breaking a stated design law (single-worker, append-only). Not a defect.
|
|
||||||
- *By-design* — the reviewer labels a spec law a bug (serial execution, non-stream client). Check the spec, not the reviewer's confidence.
|
|
||||||
- *Constant-trust* — "XSS via manifest field" where the field is git-pinned config; "path traversal via REMOTE_TMP" where it's a hardcoded const. Ask: what untrusted input reaches this sink?
|
|
||||||
- *Self-refuting* — the reviewer's own probe output disproves its claim (raw==dedup probe in a "dedup" finding). Quote it back.
|
|
||||||
3. **Acceptance bar:** concrete trigger with real inputs + full path walked. Real accepts cluster in: resource caps (pids/timeout/awaited-cleanup), honest labels (default signatures), escaping discipline, self-healing probes.
|
|
||||||
4. **One-line verdicts + evidence per finding**, shipped in the commit message; calibration section ("genuinely solid") accepted as free verification of past fixes.
|
|
||||||
|
|
||||||
## Verification (the non-negotiable)
|
Per deliverable: correct / changed / removed, with file:line, plus the gate output that proves it (`bats` failure list before and after).
|
||||||
Build/run the real artifact with its real inputs. Observe behavior at the seams:
|
|
||||||
- concurrent calls → no CWD races;
|
|
||||||
- resource ceilings → caps enforced;
|
|
||||||
- cleanup → no fd/pid/dir leaks;
|
|
||||||
- security → injection probes inert (quotes, slashes, metachars);
|
|
||||||
- normal case → output identical to pre-audit.
|
|
||||||
|
|
||||||
## Evidence (2026-10-07 + 2026-10-10/11 sessions)
|
|
||||||
- Shell-engine subagent: 426 LOC, `bun test` 39/39 green. Line-audit found: 5 leaked fds per call (`fd` sink never closed), CWD-race in `git -C .`, `set -e` + `${@/\/g\//}` slash-escaping bug, unbounded 10k-entry cache, 45s timeout on git-status.
|
|
||||||
- Review-triage census over 3 blind adversarial rounds (same repo, ~41 findings): accepted 9, rejected ~32 — rejects dominated by stale reads (protocol rewritten mid-review), inverted filter logic, unreachable-under-law triggers, base64-alphabet "quote injection" (charset has no quotes), SIGPIPE non-leaks, "no memory cap" quoting the `--memory 4g` line above the claim. Real accepts: fork-bomb pids cap, stalled-ssh upload cap, stale-flag self-heal probe, honest census label, freshness gate on pid reuse.
|
|
||||||
- Poison-law incident: sandbox fix shipped while supervisor ran old build → 812 rows graded by the dead protocol; takeover made recovery cheap. The restart step (#6) exists because of this.
|
|
||||||
|
|||||||
@@ -1,49 +1,44 @@
|
|||||||
---
|
---
|
||||||
name: docker-pin-latest-census
|
name: docker-pin-latest-census
|
||||||
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag census plus digest-equals-latest verification, no docker CLI needed."
|
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag enumeration, the rolling-latest trap, digest pinning, and rollback proof."
|
||||||
---
|
---
|
||||||
|
|
||||||
# Is my pinned image actually the latest?
|
Use when asked whether a Docker Hub / lshr.io image pin is the latest version before bumping it, or when a manifest claims an image is stale. Registry API is the answer; the `latest` tag is not.
|
||||||
|
|
||||||
Use when bumping a pinned container image tag (e.g. `lscr.io/linuxserver/*` quadlet `Image=` lines) and "latest" must be PROVEN, not assumed. Works from any box with curl — no podman/docker daemon required. `lscr.io` fronts Docker Hub; its tags are NOT on quay.io (querying quay returns nothing and reads like "no newer tags exist" — false).
|
## The API
|
||||||
|
|
||||||
## 1. Full tag census via the registry API
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:<owner>/<repo>:pull" \
|
# hub: token then paginate (the list is 100 per page)
|
||||||
| python3 -c "import json,sys;print(json.load(sys.stdin)['token'])")
|
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" | jq -r .token)
|
||||||
curl -s "https://registry-1.docker.io/v2/<owner>/<repo>/tags/list?n=2000" \
|
for off in 0 100 200 300 400 500; do
|
||||||
-H "Authorization: Bearer $tok" | python3 - <<'PY'
|
curl -s -H "Authorization: Bearer $tok" \
|
||||||
import json,sys,re
|
"https://hub.docker.com/v2/repositories/${repo}/tags?page_size=100&ordering=last_updated&page=${page}" \
|
||||||
tags=json.load(sys.stdin).get('tags') or []
|
| jq -r '.results[] | "\(.name)\t\(.last_updated)"'
|
||||||
# linuxserver form: <version>-<sha>-ls<N>; group by version, keep newest ls per version
|
done | sort
|
||||||
v={}
|
|
||||||
for t in tags:
|
|
||||||
m=re.match(r'^(\d+(?:\.\d+)+)-([0-9a-f]+)-ls(\d+)$',t)
|
|
||||||
if m: v.setdefault(tuple(map(int,m.group(1).split('.'))),[]).append((int(m.group(3)),t))
|
|
||||||
for k in sorted(v)[-5:]:
|
|
||||||
print('.'.join(map(str,k)),'-> newest:',max(v[k])[1]) # max on (ls#, tag) tuple
|
|
||||||
PY
|
|
||||||
```
|
```
|
||||||
|
|
||||||
- Sort versions as **tuples of ints**, never strings (`1.4.10 < 1.4.9` lexicographically).
|
|
||||||
- Keep the max `ls` build per version, then the newest version — both can lag independently.
|
|
||||||
- Trap already eaten: joining the tuple key to a string mid-loop then re-looking-it-up → `KeyError`. Keep keys as tuples; format only when printing.
|
|
||||||
- Some channels have NO `latest` tag in `/tags/list` (or the endpoint truncates) — absence there is not evidence.
|
|
||||||
|
|
||||||
## 2. Digest-proof the candidate against `latest`
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
for t in latest <candidate-tag>; do
|
# lscr.io (linuxserver): tags are semver, `latest` is a rolling alias
|
||||||
curl -sI "https://registry-1.docker.io/v2/<owner>/<repo>/manifests/$t" \
|
curl -s https://images-api.github.com/v2/linuxserver/${repo}/tags/list # wrong host for lscr
|
||||||
-H "Authorization: Bearer $tok" \
|
curl -s "https://hub.docker.com/v2/repositories/linuxserver/${repo}/tags?page_size=100" | jq -r '.results[].name'
|
||||||
-H "Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.docker.distribution.manifest.v2+json" \
|
|
||||||
| grep -i '^docker-content-digest'
|
|
||||||
done
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Equal digests ⇒ the pinned tag IS what `latest` serves right now — the strongest "ensure latest" evidence, and recordable in the endpoint page's pin note (the dank-fleet law: *pinned; verified == the registry's `latest` manifest digest at pin time*). A multi-arch index digest matches only if both requests send the same Accept set — send all three types above.
|
Docker Hub's own tag list (`hub.docker.com/v2/repositories/<repo>/tags`, ordered by `last_updated`) is the practical source. The registry API (`/v2/<repo>/tags/list`) needs the bearer token flow above; both return tags, neither sorts by semver.
|
||||||
|
|
||||||
## 3. Bump through git, not by hand
|
## The `latest` trap
|
||||||
|
|
||||||
Edit the quadlet/unit `Image=` line + the vault page's pin row (one owner each), run the repo suite, PR, deploy `--service <svc>` only. Rollback stays "revert one line".
|
- `latest` is a **rolling alias**, not a version. "Pin is older than `latest`" says nothing about whether a *versioned* tag exists.
|
||||||
|
- `latest` can lag or be rebuilt without a version bump; comparing a digest pin against it proves nothing.
|
||||||
|
- For linuxserver images: `stable` is the rolling branch and version tags (`2026-09-30`) are what you pin; multi-arch manifest lists mean the same tag has per-arch digests — the pin must be the manifest-list digest (`RepoDigests[0]`) you actually pull, or a re-pull on another arch silently changes the image.
|
||||||
|
|
||||||
|
## Verification recipe (before bumping a pin)
|
||||||
|
|
||||||
|
1. Read the pinned tag and digest from the manifest/quadlet.
|
||||||
|
2. Enumerate tags from the registry API (paginate; `last_updated` is not semver order).
|
||||||
|
3. Compare **versions**, not recency: does a version greater than the pinned one exist?
|
||||||
|
4. If bumping: pull the new tag and record `docker/podman inspect --format '{{index .RepoDigests 0}}'` as the new digest — pin the digest, not the tag.
|
||||||
|
5. Prove the service: it starts, version endpoint answers the new version, no crash-loop, and the rollback is the previous digest.
|
||||||
|
|
||||||
|
## Report
|
||||||
|
|
||||||
|
"pinned: X (digest …), latest available: Y, upstream changelog: …" — or "pin is current: no version tag newer than X exists". Never claim staleness from the `latest` tag alone.
|
||||||
|
|||||||
@@ -1,35 +1,60 @@
|
|||||||
---
|
---
|
||||||
name: docs-dedupe-verify-campaign
|
name: docs-dedupe-verify-campaign
|
||||||
description: "Run an exhaustive docs dedupe + fact-verification campaign in dank/command-center: scout cluster ledgers, override scouts with evidence, gate every commit, and sweep scaffolding before PR."
|
description: "Run an exhaustive docs dedupe + fact-verification campaign (command-center or any OKF vault, e.g. franksign): scout cluster ledgers, override scouts with evidence, gate every commit, and sweep scaffolding before PR."
|
||||||
---
|
---
|
||||||
|
|
||||||
# Exhaustive docs dedupe + verification campaign (dank/command-center)
|
# Exhaustive docs dedupe + verification campaign (dank/command-center; transplants to any OKF vault — proven on franksign 2026-10-21, #64 2026-10-21)
|
||||||
|
|
||||||
## Frame
|
## Frame
|
||||||
- Read `omp-docs/references/okf-spec.md` + root `omp-docs/index.md` first; inventory with `find omp-docs -name '*.md' | wc -l`.
|
- Read `omp-docs/references/okf-spec.md` (or the vault's vendored OKF contract, e.g. `franksign-docs/read-only/okf-v0.2-spec.md`) + root `index.md` first; inventory with `find <vault> -name '*.md' | wc -l`.
|
||||||
- Run `scripts/kb-lint.sh omp-docs` baseline; index-coverage gate is `scripts/kb-coverage.py omp-docs` (each dir's `index.md` must link every sibling file/subdir).
|
- Run `scripts/kb-lint.sh omp-docs` baseline; index-coverage gate is `scripts/kb-coverage.py omp-docs` (each dir's `index.md` must link every sibling file/subdir). On franksign the whole mechanical gate is one seam: `bun tests/lib/okf-lint.mjs` (frontmatter, links, index coverage, version-once) — run it before and after.
|
||||||
- Branch off fresh `origin/main` in a NEW worktree (`git -C ~/.omp worktree add ~/.omp/wt/<name> -b docs/<issue>-slug origin/main`) — workspace policy blocks `git checkout -b` in session worktrees.
|
- Branch off fresh `origin/main` in a NEW worktree (`git -C ~/.omp worktree add ~/.omp/wt/<name> -b docs/<issue>-slug origin/main`) — workspace policy blocks `git checkout -b` in session worktrees. In a session worktree, `git checkout -b` is intercepted anyway: commit on the session branch, then named-ref push `HEAD:refs/heads/docs/<n>-slug` (verify `git log origin/main..HEAD` shows ONLY your commit first).
|
||||||
- Open the Gitea issue first (multi-moving-parts law); branch name `docs/<issue#>-<slug>`.
|
- Open the Gitea issue first (multi-moving-parts law); branch name `docs/<issue#>-<slug>`.
|
||||||
|
|
||||||
## Scout fleet (overlap detection)
|
## Scout fleet (overlap detection)
|
||||||
- One `task` call, `tasks[]` batch, agent `scout`, read-only; one cluster per task (skills trees, llama trees, pi vs packages, niri, sleep/power, core, gitea, gitea+cross-cutting sweep). Cap 2 concurrent; queue rest.
|
- One `task` call, `tasks[]` batch, agent `scout`, read-only; one cluster per task. **Fan out 2-3 clusters at a time, never seven**: `task.maxConcurrency` is 2 on this fleet, so the excess only queues, and `proc://` reports queued tasks as `running` — identical uptimes with zero deliveries means thrash, not progress (#64: seven scouts, nothing in 17 min; killing the five redundant ones let the two survivors land within minutes).
|
||||||
- Ledger format per cluster → `local://dedupe/<cluster>.md`: inventory table (file | lines | verdict KEEP|MERGE|DELETE|VENDOR|UPDATE), overlaps, facts-to-keep, inbound links, staleness.
|
- Do NOT scout a cluster you have already audited inline. Kill the redundant jobs early with `write proc://<id>/kill` — the endpoint is the scarce resource, and your own session shares it.
|
||||||
- **Scouts lie.** Verified misses in the 2026-10-19 campaign: fabricated ledger entries ("peer-reviewed BrowseComp" claim that exists in no file), a DELETE verdict for a generated artifact that is checked in by design AND under test (`tools/niri/keybindings.md`, see `scripts/gen-keybinds.sh` + `tests/gen-keybinds.bats`), and a KEEP for a leftover the scout misread its own changelog about. NEVER apply a scout verdict without re-checking the file, the tests, and the changelog yourself.
|
- Ledger format per cluster: inventory rows (file | verdict KEEP|MERGE|DELETE|VENDOR|UPDATE | evidence line/command). **Demand the ledger as the yield TEXT, never a `local://` file reference**: #64 produced one 0-byte ledger plus three files whose names contained the scout's own serialized parameter block (`gitea-caddy-inventory.md` + newline + a parameter tag), recoverable only by `ls`-ing `~/.omp/agent/sessions/<session>/local/<dir>/`.
|
||||||
|
- **Scouts lie.** Verified misses in the 2026-10-19 campaign: fabricated ledger entries ("peer-reviewed BrowseComp" claim that exists in no file), a DELETE verdict for a generated artifact that is checked in by design AND under test (`tools/niri/keybindings.md`, see `scripts/gen-keybinds.sh` + `tests/gen-keybinds.bats`), and a KEEP for a leftover the scout misread its own changelog about. Franksign 2026-10-21: 2 of 3 cluster scouts fabricated — "7 missing component stories" refuted by the pairing-law seam that runs green on every push, invented entry/page counts, DELETE verdicts on live-box census numbers the purpose test KEEPS; a third hit the 30-min runtime cap producing zero output. #64 same day: three pages "missing an H1" that have one on their first heading line; a 215-line `omp-docs/AGENTS.md` that does not exist; a "wikilinks + 700-char wall" defect in `core/settings.md` that a commit from that same morning had already removed; `core/subagent-limits.md` called superseded while `agent/config.yml` carries the cited `maxRuntimeMs` / `softRequestBudget` / `maxRecursionDepth` values verbatim.
|
||||||
|
- **Four pre-checks before a scout row enters the diff**: (1) the file exists; (2) the named heading/marker is actually absent (`grep -c '^# '`, `grep -c '^# Citations'`); (3) `git log --oneline -3 -- <file>` — a same-day commit may already have fixed it; (4) if the claim is "config supersedes the doc", grep the config for the cited keys.
|
||||||
|
- Cluster sizing: for a vault under ~20 pages, scout fleets are net-negative — audit inline yourself (reading 5 pages beats babysitting a scout). Reserve scouts for multi-hundred-page trees.
|
||||||
|
|
||||||
|
## Mechanical sweeps that pay (run on every vault)
|
||||||
|
- Empty-text links: `grep -rnE '(^|[ ,])\]\(' <vault>` — a generated "See Also" pass left five `- ](index.md)` links (#64); rendered, an empty bullet reads as style, never as a broken reference.
|
||||||
|
- Citation convention census BEFORE adding anything: count `grep -rl '^resource:'` against `grep -rl '^# Citations'`. command-center's convention is `resource:` frontmatter (12 vs 3) — "add Citations to six pages" is a convention change, not a fix; only the pages missing `resource:` were real hits.
|
||||||
|
- Long-line walls: `awk 'length>300 {print FILENAME":"NR"("length")"}'` — reflow prose to ~100 columns, wording intact; leave tables (they cannot wrap) and the text of verbatim blockquotes alone.
|
||||||
|
- Dated log headings: three `## YYYY-MM-DD` sections for one day is three logs wearing one date. Collapse to one `##` plus one `###` per campaign, and write the shape into the spec page so nobody has to infer it.
|
||||||
|
|
||||||
## Fact verification
|
## Fact verification
|
||||||
- Fleet facts: read-only probes over `ssh -o BatchMode=yes dankclaw|dankserver` (podman ps/inspect/top, stat, /etc/os-release, curl localhost APIs, `podman exec <c> <tool> --version`). Live state beats docs even when docs claim recent verification — the 2026-10-18 "audit" still drifted (zealot 27B→9B, build 10754→10777, container census 14→32).
|
- Fleet facts: read-only probes over `ssh -o BatchMode=yes dankclaw|dankserver` (podman ps/inspect/top, stat, /etc/os-release, curl localhost APIs, `podman exec <c> <tool> --version`). Live state beats docs even when docs claim recent verification — the 2026-10-18 "audit" still drifted (zealot 27B→9B, build 10754→10777, container census 14→32; #64 then found `8C/16T` where SMT is off, a "masked" unit that is simply absent, Pi-hole missing from a DNS list, and a weekly-drifting engine count that became a size class with `/config` named as the only authority).
|
||||||
|
- Local box facts too: `pacman -Qs <pkg>` / `command -v` before trusting a page's "installed tool" claim (#64: `wlr-randr` documented as available, never installed).
|
||||||
|
- Unit health is not endpoint health: an `:7770` that answers `/v1/models` while its systemd unit sits inactive `Result=exit-code` means a container escaped its unit. Report the drift to the operator; never record transient state as docs, and never touch a production unit without a per-run OK.
|
||||||
|
- Code-bound facts (franksign-style): every doc claim about routes/columns/shipped status gets grepped against `apps/*/src`, `packages/db/migrations/` — docs narrating features that never shipped ("page ships in the slice" while only the API landed) get truthed with a status pointer to the owning issue.
|
||||||
- External claims: web_search/web read yourself (no subagents per Frank's rule). Kill fabrications outright (invented distro `FreeToken`, nonexistent `ornith-llama-cpp-setup` tree, unattributable "44.5% peer-reviewed CUA") — delete/drop with an explicit note, never keep "plausible" numbers.
|
- External claims: web_search/web read yourself (no subagents per Frank's rule). Kill fabrications outright (invented distro `FreeToken`, nonexistent `ornith-llama-cpp-setup` tree, unattributable "44.5% peer-reviewed CUA") — delete/drop with an explicit note, never keep "plausible" numbers.
|
||||||
- Policy drift (live ≠ tuned docs): flag in the doc + PR body as an operator decision; NEVER silently re-apply policy.
|
- Policy drift (live ≠ tuned docs): flag in the doc + PR body as an operator decision; NEVER silently re-apply policy.
|
||||||
|
- Deletion proposals from scouts die by default: archived trees carrying an honest archive banner (`pi/**`, banner 2026-10-18) stay, a directory `index.md` cannot be deleted (index-coverage gate), and incident logs are laws, not bookkeeping. Deletion is a pruning session with Frank present.
|
||||||
|
|
||||||
|
## OKF §9 / frontmatter audit (cheap wins, check every vault)
|
||||||
|
- Log entries filed under the wrong ISO-date heading (crossed midnight): cut the entry (`CUT N.=M @reg`), insert the new date heading above the old one (newest-first headings), paste the entry under it.
|
||||||
|
- Missing trust family (`generated` on machine-written pages) — lint may not catch it; grep every frontmatter block.
|
||||||
|
- Laws living only in `log.md`: promote each to its owner page (mechanism law → architecture page owning the thing), leave the log entry as the incident record; the HANDOFF/README boundary then points at the vault instead of restating.
|
||||||
|
- A law restated in two tiers (vault spec page + always-loaded `AGENTS.md`) is not a duplicate to cut: name the co-ownership in the spec page ("change one, change the other") instead of silently dropping one copy.
|
||||||
|
|
||||||
## Log compaction (log.md)
|
## Log compaction (log.md)
|
||||||
- Entry law: dated entry = what changed + durable lesson + concept-page pointer; narratives recoverable via `git log --follow`. Record the law at the top of the log. Tag historical-era entries.
|
- Entry law: dated entry = what changed + durable lesson + concept-page pointer; narratives recoverable via `git log --follow`. Record the law at the top of the log. Tag historical-era entries.
|
||||||
|
- One `## YYYY-MM-DD` per day, `###` per campaign. Put the fan-out and ledger-verification lesson in the entry — that is what stops the next campaign from trusting a scout.
|
||||||
|
|
||||||
## Gates + landing
|
## Gates + landing
|
||||||
- Gate every commit: `bash scripts/kb-lint.sh omp-docs && python3 scripts/kb-coverage.py omp-docs && bats tests/`. Pre-commit hook prints `error: command not found: caddy` on Caddyfile-touching commits — warning only, commit still lands.
|
- Gate every commit: `bash scripts/kb-lint.sh omp-docs && python3 scripts/kb-coverage.py omp-docs && bats tests/`. Pre-commit hook prints `error: command not found: caddy` on Caddyfile-touching commits — warning only, commit still lands. On franksign: kill the :3000 demo stack first (a live API hangs the gate ~10 min; law lives in `franksign-docs/architecture/testing.md`), then `bun tests/lib/okf-lint.mjs && tests/run.sh docs`.
|
||||||
- Before opening the PR: `git diff --stat origin/main..HEAD | grep -i 'tmp\|scratch\|\.bak'` — scaffolding merged under a `.tmp-` name in the 2026-10-19 campaign and needed a follow-up PR to promote.
|
- Before opening the PR: `git diff --stat origin/main..HEAD | grep -i 'tmp\|scratch\|\.bak'` — scaffolding merged under a `.tmp-` name in the 2026-10-19 campaign and needed a follow-up PR to promote.
|
||||||
- PR via API (skill `gitea-api-pr-operations` for token-extraction + merge payload); `xdg-open` the PR the moment it opens; comment the campaign ledger on the issue.
|
- PR via API (skills `gitea-api-pr-operations`, `gitea-merge-form-encoded`): token through a 600-perm `-K` curl config, never `-H` (`/proc/<pid>/cmdline` is world-readable); create/PATCH = JSON WITH Content-Type, merge = form body with NO Content-Type, `do=merge&merge_style=no-ff`, success = HTTP 200 empty body verified by landed state, not by re-reading the PR. `shred -u` the cfg and payload afterwards. `xdg-open` the PR the moment it opens; comment the campaign ledger on the issue — ownership-move table, re-read facts with the command that read them, and rejected findings with their evidence.
|
||||||
- After Frank's merge OK: propagate deployment `git -C ~/.omp fetch && git merge --ff-only origin/main`; reap campaign worktree (`git worktree remove --force`) + delete branch local/remote; close issue with landing comment.
|
- After Frank's merge OK: propagate the deployment with `git -C ~/.omp fetch && git merge --ff-only origin/main`, but first prove the FF cannot touch live machine-tier state: `comm -12 <(git -C ~/.omp status --porcelain | grep -v '^??' | awk '{print $2}' | sort) <(git diff --name-only HEAD..origin/main | sort)` must print nothing, and re-check the dirty count after the merge.
|
||||||
|
- Worktree reaping is the live-sync heartbeat's job (`wt/*` older than 7 days) unless Frank says otherwise — no `rm`, no deleting a session branch you did not create.
|
||||||
|
|
||||||
## Line-edit op syntax trap (bit three times)
|
## Line-edit op syntax trap (bit five times)
|
||||||
- Valid ops: `PUT N.=M:` (body `+text` lines), `CUT N.=M` / `CUT N.=M @reg`, `PUT <N @reg`, `PUT >N:` / `PUT <N:`. There is NO `PUT N+=:` form — it lands in the file as a literal garbage line and shifts the insert to the wrong position. For gap-insert use a lone `PUT >N:` or `PUT <N:` per call; pre-compute line numbers against the file map (ops reference original lines within one call, but a failed op poisons positions).
|
- Valid ops: `PUT N.=M:` (body `+text` lines), `CUT N.=M` / `CUT N.=M @reg`, `PUT <N @reg`, `PUT >N:` / `PUT <N:`. There is NO `PUT N+=:` form — it lands in the file as a literal garbage line and shifts the insert to the wrong position. For gap-insert use a lone `PUT >N:` or `PUT <N:` per call; pre-compute line numbers against the file map (ops reference original lines within one call, but a failed op poisons positions).
|
||||||
|
- **PUT with an empty body DELETES the range** (franksign 2026-10-21: a typo-fix `PUT 7-7:` with empty body deleted the whole entry). Bodyless deletes: use `CUT N.=M`.
|
||||||
|
- A `CUT`/`PUT` range outside the file errors cleanly (`out of range (file has N lines)`) — that is the cheapest way to learn a body was shorter than assumed; re-read the tail rather than guessing offsets.
|
||||||
|
- Bare body rows get auto-prefixed with `+` and a warning — harmless, but never rely on it: prefix every body row yourself.
|
||||||
|
- Editing a PUT body that ends adjacent to the next original line can silently consume the closing frontmatter `---` — after any frontmatter-range PUT, re-read lines 1–10 of the result.
|
||||||
- Recovery recipe when a literal op line lands: read exact bounds, `CUT N.=N` (junk), `CUT A.=B @block` (misplaced section), then `PUT <M @block` at the true anchor, re-read tail, add spacing blank if a `---` now abuts a list line.
|
- Recovery recipe when a literal op line lands: read exact bounds, `CUT N.=N` (junk), `CUT A.=B @block` (misplaced section), then `PUT <M @block` at the true anchor, re-read tail, add spacing blank if a `---` now abuts a list line.
|
||||||
|
|||||||
@@ -1,62 +1,29 @@
|
|||||||
---
|
---
|
||||||
name: fleet-service-runtime-census
|
name: fleet-service-runtime-census
|
||||||
description: "Answer \"what is service running right now?\" for a dank-fleet service: live endpoint probe plus host units, image ids, live-vs-repo env, and drop-ins."
|
description: "Answer \"what is service running right now?\" for a dank-fleet service: unit state, container image/digest, endpoint contract, loaded env — machine over repo, in that order."
|
||||||
---
|
---
|
||||||
|
|
||||||
# Fleet service runtime census
|
Answer "what is service running right now?" for a dank-fleet service without trusting the repo. Repo state is intent; the machine is truth, and they differ by however long since the last deploy.
|
||||||
|
|
||||||
Answer the question "what is `<service>` running currently?" with evidence, not from docs. Docs say what *should* run; the host says what *does*. Run all five probes, then report one table.
|
## Order of operations (cheap → expensive)
|
||||||
|
|
||||||
## 1. Probe the published port (works from any LAN box, no ssh)
|
1. **Unit state on the host** (rootless quadlet ⇒ `--user`; some units are root):
|
||||||
|
`ssh -o BatchMode=yes host 'systemctl --user show S -p ActiveState -p SubState -p ExecMainStartTimestamp -p NRestarts'`
|
||||||
|
Boot-loop tell: `NRestarts` climbing with a short `ExecMainStartTimestamp`.
|
||||||
|
2. **The container's own view**: `podman ps --filter name=S --format '{{.Image}} {{.Status}} {{.Names}}'`, then `podman inspect` for `Image` (tag **and** `RepoDigests`) and `Env`.
|
||||||
|
3. **The endpoint contract, from the service itself** — this is the only answer that matters to a consumer:
|
||||||
|
`curl -s host:port/v1/models` (served model id), `curl -s host:port/props` (`n_ctx`, `n_ctx_train`, caps), `curl -s -o /dev/null -w '%{http_code}' host:port/health`.
|
||||||
|
4. **Effective flags**: the live process is the truth. `systemctl --user cat S` shows `EnvironmentFile=`; read that file. A repo `scripts/S.env` can be ahead of, or behind, what the running unit loaded — and a unit restart is what re-reads it.
|
||||||
|
5. **Logs last**: `journalctl --user -u S -n 80 --no-pager` for the boot ledger / refusal messages, after you know whether it is even up.
|
||||||
|
|
||||||
```bash
|
## Facts that bite
|
||||||
curl -s -m 10 http://<host-ip>:<port>/health; echo
|
|
||||||
curl -s -m 10 http://<host-ip>:<port>/v1/audio/voices # per-endpoint extras: /v1/models, /metrics
|
|
||||||
```
|
|
||||||
|
|
||||||
The extra route matters more than `/health` for identity claims:
|
- **A service is up ≠ configured how you think.** llama.cpp's `/v1/models` reports `max_model_len`, not the engine's per-slot ceiling; halogen reports `n_ctx` per slot. Read `/props` before quoting any window number.
|
||||||
- llama.cpp and radiance **echo the requested model name** — a 200 proves nothing; read `/v1/models`.
|
- **`systemctl is-enabled` answers enablement, not activity.** A timerless service reports `linked`/`static`; `deploy.sh` that prints `ENABLE` for it is lying. Read `systemctl is-enabled --quiet` exit status, not the text.
|
||||||
- halogen 404s `/completion`, `/ping`, `/props`, `/tokenize`.
|
- **`NRestarts: 0` with `ActiveState=activating` is a crash-loop only if the timestamps move**; check `ExecMainStartTimestamp` twice, 5 s apart.
|
||||||
- The frank-tts façade answers `/health` **without dialing the engine**, so "façade healthy, engine down" is a real state; the voice list in that same response is the honest census.
|
- **Read-only is always allowed** from the dev box (`ssh -o BatchMode=yes`); state changes on remote boxes are not, without a per-run OK. Production services (`frank-archon`, `frank-dragoon`, `frank-zealot`) are never stopped/restarted to "see what happens".
|
||||||
|
- **Digest-pinned images**: "when was this image updated?" is answered by `podman inspect` `RepoDigests` + the registry API, not by the unit's start time (a restart re-uses the local digest).
|
||||||
|
|
||||||
## 2. Units + containers on the host (ssh as the deploy user, no sudo)
|
## Report
|
||||||
|
|
||||||
```bash
|
One line per layer with the evidence: unit (`active (running)`, since, restarts), container (image + digest), endpoint (served id, `n_ctx`, health code), loaded env (file path + the knobs that matter), and explicitly which of those came from the machine versus the repo — that gap is the answer to most "why is it behaving like that" questions.
|
||||||
ssh <host> 'systemctl --user status <svc>.service <engine>.service --no-pager \
|
|
||||||
| grep -E "^(●| Active| Loaded)"; \
|
|
||||||
podman images --format "{{.Repository}}:{{.Tag}} {{.ID}} {{.Created}}"; \
|
|
||||||
podman ps -a --format "{{.Names}}\t{{.Status}}\t{{.Image}}"'
|
|
||||||
```
|
|
||||||
|
|
||||||
- `active (running) since …` + `Up N hours (healthy)` is the answer; `activating (auto-restart)` with a ~50 ms exit is a **preflight refusal**, not an engine fault.
|
|
||||||
- **NEVER prefix these podman calls with `sudo`.** Containers live in the deploy user's rootless storage; `sudo podman inspect frank-tts-engine` returns `no such object` while the plain form succeeds. Treating that as "container missing" is a false finding.
|
|
||||||
|
|
||||||
## 3. Drop-ins — the override that no health gate sees
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ssh <host> 'systemctl --user show <svc>.service -p DropInPaths; \
|
|
||||||
systemctl --user show <engine>.service -p DropInPaths'
|
|
||||||
```
|
|
||||||
|
|
||||||
`DropInPaths=` must be empty. A leftover bench drop-in (`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`) makes the next deploy boot a different model at a different shape while `/health` still answers 200 and the e2e gate passes.
|
|
||||||
|
|
||||||
## 4. Live env vs repo env — the only truth test
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# repo side (local checkout)
|
|
||||||
grep -vE '^\s*#|^\s*$' scripts/<svc>.env
|
|
||||||
# host side
|
|
||||||
ssh <host> 'sudo cat /home/dank/serverAppConfigs/scripts/<svc>.env' | grep -vE '^\s*#|^\s*$'
|
|
||||||
```
|
|
||||||
|
|
||||||
Diff them. Per repo law the env file beside the serve script **is** the authority; a doc page that disagrees with it is the bug, and host-side drift means an uncommitted edit. Also check `git log --oneline -3 -- scripts/<svc>.env` to confirm the host isn't ahead of git.
|
|
||||||
|
|
||||||
## 5. Report
|
|
||||||
|
|
||||||
Table of: unit state + since, image repo:tag@short-ID + age, backend/alias actually served (from step 1, not from the name), drop-ins, and whether live env == repo env. State explicitly when a fact came from docs rather than from a probe.
|
|
||||||
|
|
||||||
## Traps that produce false findings
|
|
||||||
|
|
||||||
- Digest-pinned images: the local image ID's *creation date* answers "is this stale" better than the tag; `latest` + `Pull=never` is a locally built image, not a registry pull.
|
|
||||||
- `ssh` user context changes what podman sees (root vs rootless) — see step 2.
|
|
||||||
- Two ports on one host are two providers in `models.yml`; a port mismatch is a rename or drift, not a config typo.
|
|
||||||
|
|||||||
@@ -1,35 +1,37 @@
|
|||||||
---
|
---
|
||||||
name: frank-job-hunt-maintenance
|
name: frank-job-hunt-maintenance
|
||||||
description: "Maintain frank-job-hunt agents: live-census gate hardening loop, registry locator verification, and the workstation/Dankserver/Dankclaw deploy-sync ritual."
|
description: "Maintaining frank-job-hunt: the census gate's fail-closed rules, ATS locator token derivation, and the two-host deploy-sync verification."
|
||||||
---
|
---
|
||||||
|
|
||||||
# frank-job-hunt maintenance
|
Use when maintaining `frank-job-hunt`: the live-census gate, the registry locator, or Dankserver/Dankclaw deploy-sync. The repo is built and deployed; agents live on Dankserver, ntfy on Dankclaw.
|
||||||
|
|
||||||
Repo `~/Projects/frank-job-hunt` (Gitea `dank/frank-job-hunt` on Dankclaw 192.168.0.143:3000). Production: Dankserver user timers `frank-job-sweep.timer` 07:30, `frank-job-digest.timer` 08:15 (`Linger=yes`). ntfy relay: Dankclaw quadlet `ntfy.service`, route `ntfy.franksdank.cloud` (no anubis), LAN record in `hosts/custom.list` (Pi-hole v6 file at `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list`; v6 API paths are hostile — edit the file, then `podman exec pihole pihole restartdns`). Secrets: `~/.config/frank-job-hunt.env` (600, workstation + Dankserver).
|
## The two rules that caused every bug here
|
||||||
|
|
||||||
## Law (validated 2026-10-11)
|
1. **A URL is data.** The census gate and the registry locator treat portal URLs as opaque strings; nothing may template a URL from a company name. `jobs.dankclaw.example` is a fixture host (RFC 2606) — never derive a real domain from a company name, and never click a link the pipeline constructed.
|
||||||
- The fleet judge (`frank-zealot`, dankclaw:7771) is a **ranker, never a gate** — it inflates anything reaching it (non-technical roles scored 88-91). All discrimination lives in deterministic gates (`agents/lib/gates.ts`); zealot only ranks survivors.
|
2. **A stale row is worse than a missing row.** Every gate must fail closed: an unverifiable posting is reported as unverifiable, not kept as live. Deleting is allowed; asserting is not.
|
||||||
- Thinking-model fault: verdict lands in `reasoning_content` with `content:""` → judge sends `chat_template_kwargs:{enable_thinking:false}` (0.7s clean JSON).
|
|
||||||
- **Family beats topic**: role family (recruiter/sales/finance/PM/designer/people-ops/revops/…) gates `gate:scope` unconditionally — no infra-token override ("DevOps Recruiter" is recruiting). Compound technical titles keep the `INFRA_TITLE` override via `NEG_TITLE` branch.
|
|
||||||
- **gate:region**: remote is US-only (visa reality). "Remote - Bangalore" = onsite abroad → gate; `US` regex pass wins over `NONUS` match ("U.S. & Canada" passes).
|
|
||||||
|
|
||||||
## Gate hardening loop (live-census, TDD)
|
## Census gate
|
||||||
1. `bun agents/sweep.ts --dry` (full lane ≈ 30–90 s; 852 postings ≈ 62 pass / 790 filed).
|
|
||||||
2. Classify leaks in the top-10: family gap / region gap / scoring tail. Accept long tail (Director/Founder edge cases) — digest + human curation is the design.
|
|
||||||
3. Token fix in `gates.ts` regexes (`NEG_FAMILY` = no override; `METRO`/`US`/`NONUS` for arrangement). NEVER dedupe-check by memory — read file first; the file has bitten line-offset edits repeatedly.
|
|
||||||
4. Pin every leak with a named regression test in `agents/test/gates.test.ts` before/with the fix.
|
|
||||||
5. `bun test agents/test/ && bun x tsc --noEmit` — hooks re-run these at commit.
|
|
||||||
6. Ship: `git commit` (pre-commit = tests + tsc + docs-link + secret scan) → `git push` → `ssh dankserver 'cd ~/Projects/frank-job-hunt && git pull'` (timers pull from master; Dankserver drift = tomorrow's run is stale).
|
|
||||||
|
|
||||||
## Registry verification (monthly, `agents/verify-registry.ts`)
|
`scripts/census-gate.sh` decides keep / stale / unverifiable per row.
|
||||||
Scouts' `ats`/`ats_identifier` are guesses (14/84 were wrong). Ritual:
|
|
||||||
1. `bun agents/verify-registry.ts` (dry, ~70 s serial) → REPAIR census.
|
|
||||||
2. `--write` applies; repairs with live jobs preferred over empty boards (0-job boards 404 later).
|
|
||||||
3. Unresolved = honest manual lane (vanity Workdays, self-built ATS incl. majors, neogov) — NOT a code bug; per-entry error census in sweep output is the same signal.
|
|
||||||
4. Update `targets/README.md` census lines + `docs/log.md`; commit/push/pull (same ship step).
|
|
||||||
|
|
||||||
## Pitfalls
|
- Live proof = HTTP GET on the posting URL against the ATS API or the page itself; `404`/`410` ⇒ stale, non-200 with no body ⇒ unverifiable. Never infer from age alone.
|
||||||
- `--ats=<x>` smoke filters the census line — "N API-sweepable" reflects the filter, not the registry; don't diagnose a phantom bug.
|
- The gate runs against the *live* registry, so a row that passes here and then fails in production means the registry drifted — re-run before blaming the gate.
|
||||||
- Workday vanity careers URLs (`careers.uw.edu`) are NOT CXS-API-resolvable; tenant-only identifiers are (site path required).
|
- Test seams: fixture ATS server + `…_BASE_URL` env. Tests assert the row's fate (kept / dropped / flagged), never internal state.
|
||||||
- Root `README.md` vs `targets/README.md` name collision in edit anchors: use full path headers.
|
|
||||||
- Zealot endpoint probe: `curl http://192.168.0.143:7771/v1/models`.
|
## Registry locator
|
||||||
|
|
||||||
|
Resolves which ATS a company uses (Greenhouse `boards-api.greenhouse.io/v1/boards/<token>/jobs`, Lever `api.lever.co/v0/postings/<token>`, Ashby `api.ashbyhq.com/posting-api/job-board/<token>`, Workable, SmartRecruiters).
|
||||||
|
|
||||||
|
- Token ≠ company name: it comes from the discovered posting URL, so a locator change must keep the derivation (URL → token), not substitute a slug.
|
||||||
|
- Unknown ATS ⇒ unverifiable, not "no jobs". "Zero jobs" is a strong claim; only a successful API answer with an empty list may say it.
|
||||||
|
- Rate limits: 429 ⇒ back off and mark unverifiable for this pass, never stale.
|
||||||
|
|
||||||
|
## Deploy-sync (Dankserver agents, Dankclaw ntfy)
|
||||||
|
|
||||||
|
- Two hosts, two roles: inference/agents on Dankserver, notification sink on Dankclaw. A deploy that only touches one leaves the pair inconsistent — the agent can post to a sink that no longer accepts its token, silently.
|
||||||
|
- Deploy is `scripts/deploy.sh` from the correct checkout (dev side, clean tree); it pushes the branch first, so a stale local `main` makes the push fail — `git merge --ff-only origin/main` before deploying after any externally merged PR.
|
||||||
|
- Verify after deploy on the *served* surface: the agent answers a probe, and a test ntfy message arrives on the phone. "The unit is active" is not the proof.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
Black-box, fixtures over network: fake ATS servers on localhost, fake ntfy sink, temp registry file. Assert output rows and exit status. Never assert that an env var is set or that a helper was called.
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
---
|
||||||
|
name: gitea-silent-ignore-patch-trap
|
||||||
|
description: "Gitea API answers success codes while silently ignoring mutation bodies: labels in PATCH /issues/N, and any JSON body sent without Content-Type: application/json. Verify mutations with a follow-up GET, never the status code."
|
||||||
|
---
|
||||||
|
|
||||||
|
# Gitea silent-ignore family (issue mutations)
|
||||||
|
|
||||||
|
Two verified failure modes on the Dankclaw Gitea API (1.27.3, verified live 2026-10-21 on `dank/franksign`); both **return a success code and change nothing**:
|
||||||
|
|
||||||
|
1. **`PATCH /repos/{o}/{r}/issues/{n}` with `{"labels":[...]}`** — 200, labels untouched, even *with* a proper JSON Content-Type. Labels move only via the dedicated endpoints:
|
||||||
|
- add: `POST /issues/{n}/labels` with `{"labels":[<id>]}`
|
||||||
|
- replace: `PUT /issues/{n}/labels` with `{"labels":[ids]}` (works without Content-Type)
|
||||||
|
- remove: `DELETE /issues/{n}/labels/{id}`
|
||||||
|
2. **Any JSON body without `header = "Content-Type: application/json"` in the curl config** — e.g. closing with `PATCH {"state":"closed"}` returns **201 and no-ops**; identical request with the header closes. curl's default `-d` type (`application/x-www-form-urlencoded`) makes this silent.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
- Mutating JSON calls: always send `Content-Type: application/json` alongside `Authorization: token` in the same `-K` config file.
|
||||||
|
- **Never trust the status code for mutations — follow up with `GET` and read the field** (`labels`, `state`). Both traps look like success otherwise.
|
||||||
|
- Token stays out of `-H` args (world-readable `/proc/<pid>/cmdline`): extract from `~/.config/secrets/gitea` into a 600-perm curl config, use `-K`.
|
||||||
|
|
||||||
|
## Quick verification idiom
|
||||||
|
```bash
|
||||||
|
curl -sK "$cfg" "http://192.168.0.143:3000/api/v1/repos/dank/franksign/issues/22" \
|
||||||
|
| jq '{state, labels: [.labels[].name]}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Franksign's vault page `franksign-docs/agents/issue-tracker.md` owns the repo-local copy of this law; this skill exists so other repos (serverAppConfigs, command-center) don't re-burn the cycle.
|
||||||
@@ -13,7 +13,10 @@ http://dank:<40-char-token>@192.168.0.143%3a3000
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
tok=$(sed -nE 's#^[a-z]+://[^:]+:([^@]+)@.*#\1#p' ~/.config/secrets/gitea | head -1)
|
tok=$(sed -nE 's#^[a-z]+://[^:]+:([^@]+)@.*#\1#p' ~/.config/secrets/gitea | head -1)
|
||||||
curl -s -K <600-perm cfg file> http://192.168.0.143:3000/api/v1/<path>
|
echo "tok len ${#tok}" # must be 40 — assert before any call
|
||||||
|
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"
|
||||||
|
curl -s -K "$cfg" http://192.168.0.143:3000/api/v1/<path>
|
||||||
|
rm -f "$cfg" # in the SAME session: the file holds a live token
|
||||||
```
|
```
|
||||||
|
|
||||||
## `git credential-store --file … fill` answers EMPTY — do not debug it
|
## `git credential-store --file … fill` answers EMPTY — do not debug it
|
||||||
@@ -24,20 +27,21 @@ Tried 2026-10-21 while opening `dank/command-center` PR #61: the obvious extract
|
|||||||
printf 'protocol=http\nhost=192.168.0.143\nport=3000\n\n' | git credential-store --file ~/.config/secrets/gitea fill
|
printf 'protocol=http\nhost=192.168.0.143\nport=3000\n\n' | git credential-store --file ~/.config/secrets/gitea fill
|
||||||
```
|
```
|
||||||
|
|
||||||
prints **nothing**, with no error. It is not a permissions problem and not a wrong path: the stored line encodes the port inside the host field (`@192.168.0.143%3a3000`), so a `host=`+`port=` query can never match it. Add `path=` or drop `port=` and it still answers empty. Read the file with `sed` (recipe above) and move on — the chain `TOKEN=$(…) && [ -n "$TOKEN" ] && …` fails silently at the test, which reads like a curl failure if the `&&` hides it.
|
prints **nothing**, with no error. Not permissions, not a wrong path: the stored line encodes the port inside the host field (`@192.168.0.143%3a3000`), so a `host=`+`port=` query can never match it. Add `path=` or drop `port=` and it still answers empty. Read the file with `sed` (recipe above) and move on — the chain `TOKEN=$(…) && [ -n "$TOKEN" ] && …` fails silently at the test, which reads like a curl failure because the `&&` hides which link broke.
|
||||||
|
|
||||||
## The silent-empty-token trap (bit 2026-10-21, cost a debug detour)
|
## The silent-empty-token trap (bit 2026-10-21, cost a debug detour)
|
||||||
|
|
||||||
An `awk -F= '/^username=/{...}'` parse of this file yields `tok=""` **silently** — and reads still appear to work when the repo is **public**: `dank/serverAppConfigs` and `dank/command-center` return real issue/label data unauthenticated, so every census "passes" until the first **write** (issue comment, label, PR create/merge) fails with `{"message":"token is always required"}`-style errors or 404s (private repos 404 on an empty token — that is an auth problem, NOT a missing repo).
|
An `awk -F= '/^username=/{...}'` parse of this file yields `tok=""` **silently** — and reads still appear to work when the repo is **public**: `dank/serverAppConfigs` and `dank/command-center` return real issue/label data unauthenticated, so every census "passes" until the first **write** (issue comment, label, PR create/merge) fails with `{"message":"token is always required"}`-style errors or 404s (private repos 404 on an empty token — that is an auth problem, NOT a missing repo).
|
||||||
|
|
||||||
- `echo "tok len ${#tok}"` (must be 40) before believing ANY API result — reads included.
|
- `echo "tok len ${#tok}"` (must be 40) before believing ANY API result — reads included.
|
||||||
- `curl -s -K <600-perm file>` carrying `header = "Authorization: token $tok"` — never `-H` on argv (`/proc/<pid>/cmdline` is world-readable; this leaked an HF token once).
|
|
||||||
- Prove auth once per session: `GET /api/v1/user` → `"login":"dank"`.
|
- Prove auth once per session: `GET /api/v1/user` → `"login":"dank"`.
|
||||||
- The temp `-K` config holds a live token: `rm -f` it in the same session that made the calls, and never `cat` it.
|
- `curl -s -K <600-perm file>` carrying `header = "Authorization: token $tok"` — never `-H` on argv (`/proc/<pid>/cmdline` is world-readable; this leaked an HF token once).
|
||||||
|
- `jq -n --arg t … --arg h … --arg b "$(/bin/cat /tmp/pr.md)" '{…}'` — `$(cat file)` gets optimised away in this harness, so use `--rawfile b /tmp/pr.md` instead.
|
||||||
|
|
||||||
## Other notes
|
## Other notes
|
||||||
|
|
||||||
- `git credential fill` fails headless here (`could not read Username … terminal prompts disabled`); read the store file directly instead.
|
- `git credential fill` fails headless here (`could not read Username … terminal prompts disabled`); read the store file directly instead.
|
||||||
- Never print the token: mask when inspecting the file —
|
- Never print the token: mask when inspecting the file —
|
||||||
`sed -E 's#(://[^:]+:)[^@]+@#\1REDACTED@#' ~/.config/secrets/gitea`
|
`sed -E 's#(://[^:]+:)[^@]+@#\1REDACTED@#' ~/.config/secrets/gitea`
|
||||||
|
- Merge writes need form-encoded fields, not JSON — see `gitea-pr-api-cycle`.
|
||||||
- Issue/PR numbers are **per repo**: `dank/dankvoice` #8–#11 (refactor, hardening, AbortError fix, menu defects) are unrelated to `dank/command-center` #8–#11 (SDLC, trunk-law). Read the body from the repo the work targets before referencing an issue in a commit message.
|
- Issue/PR numbers are **per repo**: `dank/dankvoice` #8–#11 (refactor, hardening, AbortError fix, menu defects) are unrelated to `dank/command-center` #8–#11 (SDLC, trunk-law). Read the body from the repo the work targets before referencing an issue in a commit message.
|
||||||
|
|||||||
@@ -1,41 +1,48 @@
|
|||||||
---
|
---
|
||||||
name: gitignore-trailing-slash-symlink-contamination
|
name: gitignore-trailing-slash-symlink-contamination
|
||||||
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get committed; detect with git ls-tree mode 120000, fix with git rm --cached."
|
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get tracked and pushed as 1-line blobs; same-name rules silently match worktree dirs, and .deploy-stale pollutes. Three fixes with tests."
|
||||||
---
|
---
|
||||||
|
|
||||||
A `.gitignore` entry written with a **trailing slash matches directories only**. A symlink of the same name is not a directory, so it slips past the rule, and `git add -A` in a `git worktree` checkout silently **tracks** it. Cost measured 2026-10-19: a machine-specific symlink (`node_modules -> /home/blank/Projects/dankvoice/node_modules`) landed in `main` with a merged PR.
|
Gitignore semantics trap with three real-world consequences (serverAppConfigs, 2026-10-19):
|
||||||
|
|
||||||
Why worktrees are the trigger: a worktree has no `node_modules` (or other installed-deps dir) of its own, so the cheap way to make build/test work there is a symlink into the primary checkout. With `node_modules/` ignored, nothing complains.
|
## The rule
|
||||||
|
|
||||||
## Detect
|
`.gitignore` patterns ending in `/` match directories only. Worktree-relative: `**/node_modules/` is `**/node_modules/` → a symlink named `node_modules` does NOT match, so it is "untracked" no more — it gets added, committed, and pushed as a 1-line blob containing its target path.
|
||||||
|
|
||||||
|
## Consequence 1: worktree symlink contamination
|
||||||
|
|
||||||
|
- `git add -A` then commit then push in a worktree whose root holds `node_modules -> ...` (the worktree convention symlink) commits it.
|
||||||
|
- The repo then holds `package/frontend/node_modules` as a text file containing the absolute path on the author's box.
|
||||||
|
- The receiving worktree gets that 1-line file instead of the symlink → "module not found" and "why is this a 30-byte file?".
|
||||||
|
- Detection: `git ls-tree -r origin/main | awk '$1=="120000"'` (mode 120000 = symlink) or `git ls-files -s | grep 120000`.
|
||||||
|
- Fix: ignore rules WITHOUT trailing slash (`**/node_modules`, `**/.venv`) so symlinks match too.
|
||||||
|
|
||||||
|
## Consequence 2: same-name worktree rules silently matching worktree dirs
|
||||||
|
|
||||||
|
A root rule like `wt-*` (meant to ignore local worktree checkouts) matches every path whose last segment starts with `wt-` — so legitimate content under such a directory goes untracked. `git status --ignored` is the only visible trace; detection: `git check-ignore -v <path>` prints the owning rule + line number. (Repo-specific: `dank/command-center`'s `.gitignore` carries a "per-worktree memory buckets" block ignoring `agent/memories/sharpshooter/wt-*/` and `agent/memories/summaries/wt-*/` — intentional there.)
|
||||||
|
|
||||||
|
## Consequence 3: `.deploy-stale`
|
||||||
|
|
||||||
|
`deploy.sh` renames a stale tree to `<path>.deploy-stale` before recreating it. Without an ignore rule those directories land in `git status` as untracked and get swept into `git add -A` commits; they accumulate on every server.
|
||||||
|
|
||||||
|
## Fixes with tests
|
||||||
|
|
||||||
|
```gitignore
|
||||||
|
**/node_modules
|
||||||
|
**/.venv
|
||||||
|
.deploy-stale/
|
||||||
|
```
|
||||||
|
|
||||||
|
Black-box test against the real rules, not a fixture: `git check-ignore -v <symlink-target>` must name the rule. `git check-ignore` takes hypothetical paths — nothing needs to exist on disk:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git ls-tree HEAD -- node_modules # mode 120000 == tracked symlink
|
git check-ignore -v natives/x.wasm dotfiles/x/.config/noctalia/Noctalia.json
|
||||||
git ls-files -s | awk '$1==120000' # every tracked symlink in the index
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Fix (branch, never on main directly)
|
Two ignore surfaces must be kept in sync (deploy exclusions vs ignore rules): if a deployer excludes `*/Noctalia*` but the ignore list has only the lowercase class, generated files become tracked-but-never-deployed — test both halves with `git check-ignore`.
|
||||||
|
|
||||||
```bash
|
## Recovery (if contamination already pushed)
|
||||||
git rm -q --cached node_modules # untrack; leaves the file on disk
|
|
||||||
```
|
|
||||||
|
|
||||||
plus relax the ignore rule to the slash-free form, with the reason inline so nobody re-adds the slash:
|
1. `git rm --cached <path>` on the symlink blob, commit, push.
|
||||||
|
2. Fix the ignore rule first (else the next `git add -A` re-adds it).
|
||||||
```
|
3. Untracked ≠ deleted from history: the blob stays in every old commit. Only a rewrite removes it; that is the operator's call, not a cleanup run alone.
|
||||||
# Trailing slash would only match directories: a node_modules symlink (the
|
|
||||||
# shape `git worktree` checkouts end up with) slips past it and gets tracked.
|
|
||||||
node_modules
|
|
||||||
```
|
|
||||||
|
|
||||||
Then `git check-ignore -v node_modules` must print the rule, and the suite runs after (the symlink still resolves, it is just ignored).
|
|
||||||
|
|
||||||
## Prevent
|
|
||||||
|
|
||||||
In a fresh worktree, create the deps symlink **after** confirming the ignore rule has no trailing slash:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git check-ignore -v node_modules || sed -i 's|^node_modules/$|node_modules|' .gitignore
|
|
||||||
```
|
|
||||||
|
|
||||||
Same class for any ignored-by-convention dir that a worktree might provide as a link: `node_modules`, `vendor`, `.venv`, `target`, `dist` when built artifacts are ignored.
|
|
||||||
|
|||||||
@@ -0,0 +1,29 @@
|
|||||||
|
---
|
||||||
|
name: halogen-npu-lane-boot-loop-triage
|
||||||
|
description: "Triage a frank-dragoon (halogen-flash-server) unit crash-loop caused by the NPU lane: signature classification, restart-debris trap, XRT mount-path census, and the transient env-file proof boot."
|
||||||
|
---
|
||||||
|
|
||||||
|
Triage when `frank-dragoon` (or any halogen-flash-server unit with `HALOGEN_NPU_MODELS` armed) sits in `activating (auto-restart)` with `ExecStart … code=exited, status=1/FAILURE` seconds after start, while `/health` stays dark.
|
||||||
|
|
||||||
|
## 1. Classify the exit BEFORE touching the config
|
||||||
|
- `NRestarts` climbing + exit in <5 s = refusal, not an engine fault. Read `journalctl -u frank-dragoon --since "-30m"` AND the engine log `/home/dank/logs/frank_dragoon.log` (engine stdout never goes to the journal).
|
||||||
|
- Lane signature: log ends with `halogen npu: … <name>: N files checked against this image's record` then `halogen-npu: No such library '<path>'` + `the NPU engine exited before it was ready (rc=1)` → **the NPU lane exit is fatal to the whole container** (supervisor kills the Flash engine with it). Chat-down is the lane's fault even though chat code never changed.
|
||||||
|
- KV/pre-flight lines present but no `locked in RAM` line = died before weight pinning.
|
||||||
|
|
||||||
|
## 2. The restart-debris trap
|
||||||
|
`Restart=always` re-runs the failing boot every ~10 s; each attempt leaves GPU/NPU handles being released, and the next boot logs `GTT in use before this start: N GiB … held by 1 process(es)`. That NOTE is debris from your own crash-loop — do not chase it as a foreign co-tenant (real co-tenants: ComfyUI `python` on `/dev/kfd`, find with `sudo fuser -v /dev/dri/renderD128 /dev/kfd`). `sudo systemctl stop <unit>` FIRST, then any diagnosis.
|
||||||
|
|
||||||
|
## 3. XRT path census (the `No such library '/opt/xilinx/lib/<multiarch>/libxrt_*.so.2'` case)
|
||||||
|
- The image ships NO XRT at all (`find / -xdev -name "libxrt*"` inside the image = empty); the NPU child `dlopen`s the host's libs through the serve script's mounts, which land at TWO destinations only: `/opt/xilinx/xrt/lib/<L>` and `$XRT_DIR/<L>`. A child searching `/opt/xilinx/lib/<multiarch>/` names a THIRD prefix nobody mounts.
|
||||||
|
- The serve loop resolves `REAL="$(readlink -f …)"` and mounts resolved files — if a host package upgrade retargets the symlinks (check `grep libxrt /var/log/dpkg.log` + `ldconfig -p | grep xrt` mtimes vs the last good boot), the file set the child expects can change under byte-identical mounts.
|
||||||
|
- Rootless gotcha: `podman mount` fails outside `podman unshare` and leaves `$M` EMPTY — `grep -r pat "$M/opt"` then greps your CWD = HOST tree (false hits like `/opt/fastflowlm`). Always `podman unshare bash -c 'M=$(podman mount $C); …'` or `--entrypoint ls|cat` probes instead, and distrust hits that name host-only paths.
|
||||||
|
|
||||||
|
## 4. Proof boot without touching production
|
||||||
|
The serve script honors `FRANK_DRAGOON_ENV_FILE` (sanctioned seam). Throwaway script on the host:
|
||||||
|
1. scratch env: `source /home/dank/serverAppConfigs/scripts/frank-dragoon.env` + the knob under test.
|
||||||
|
2. `sed` the serve script to inject the candidate extra mounts into the `NPU_ARGS+=(--device …` line — mount RESOLVED real files (`.so.2.25.00`), never symlinks (vendor: dangling links inside the container).
|
||||||
|
3. `export FRANK_DRAGOON_ENV_FILE=/tmp/… FRANK_DRAGOON_LOG=/tmp/…; FRANK_DRAGOON_CONTAINER=<scratch-name> timeout 1200 /tmp/serve-test.sh` — the container still publishes 7770, so `systemctl stop` the unit first (port binds at CREATE, exit 126 on race).
|
||||||
|
4. Retire same session: `podman rm -f <scratch-name>`, trash `/tmp` scripts; the unit stays stopped until the real fix deploys.
|
||||||
|
|
||||||
|
## 5. Restore chat while the lane is unfixed
|
||||||
|
`HALOGEN_NPU_MODELS=` (blank) is the documented one-line lane-off switch — lane absent = endpoint exactly as before the lane existed. Land through git + `deploy.sh dankclaw --service frank-dragoon`; never hand-edit the remote env.
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
---
|
||||||
|
name: halogen-npu-mount-prefix-proof
|
||||||
|
description: "Prove and fix a halogen frank-dragoon NPU-lane mount failure: crash-loop where a dead lane kills the whole container, dlopen-prefix census, and the sanctioned FRANK_DRAGOON_ENV_FILE proof-boot."
|
||||||
|
---
|
||||||
|
|
||||||
|
# Halogen NPU lane: dead lane kills the container — prove the mount prefix, don't guess
|
||||||
|
|
||||||
|
Applies when a `frank-dragoon` unit crash-loops in seconds (`exit 1` in ~200 ms,
|
||||||
|
`NRestarts` climbing) and the engine log's last lines are:
|
||||||
|
|
||||||
|
```
|
||||||
|
halogen npu: <model>: 15 files checked against this image's record
|
||||||
|
halogen-npu: No such library '<some/prefix>/libxrt_core.so.2'
|
||||||
|
halogen npu: the NPU engine exited before it was ready (rc=1)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Laws this incident wrote (2026-10-21, frank-dragoon, 88-restart loop)
|
||||||
|
- **A dead NPU lane is fatal to the whole container**: the flash-server supervisor
|
||||||
|
exits with the lane. Chat dies beside a healthy GPU preflight — do not read the
|
||||||
|
GTT/weights lines as "engine fine, side feature broken".
|
||||||
|
- The NPU child `dlopen`s distro XRT from **three** container prefixes per library:
|
||||||
|
`/opt/xilinx/xrt/lib/`, the library's own install path, **and
|
||||||
|
`/opt/xilinx/lib/<multiarch>/`**. The missing one is exactly what the error names.
|
||||||
|
- Host state can flip this under a byte-identical container: the lane ran for days
|
||||||
|
with two prefixes, then a host `libxrt2`/`libxrt-npu2` reconfigure (check
|
||||||
|
`grep libxrt /var/log/dpkg.log`) made the child demand the third. Image digest,
|
||||||
|
serve script, payload all unchanged ⇒ suspect HOST package state, not the deploy.
|
||||||
|
- Fastest discriminator vs engine faults: exit in <1 s with the engine's preflight
|
||||||
|
lines present = serve/lane gate failure, not model load. Model-load failures take
|
||||||
|
minutes and print pin/KV lines.
|
||||||
|
|
||||||
|
## Proof procedure (sanctioned seam, no scratch ports)
|
||||||
|
1. Read the error's path verbatim — it names the missing prefix.
|
||||||
|
2. `sudo systemctl stop frank-dragoon` (unit must be down: the proof binds :7770
|
||||||
|
at container CREATE).
|
||||||
|
3. Scratch env file on the host using the bench-sanctioned seam:
|
||||||
|
```
|
||||||
|
/tmp/dragoon-nputest.env:
|
||||||
|
source /home/dank/serverAppConfigs/scripts/frank-dragoon.env
|
||||||
|
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
|
||||||
|
```
|
||||||
|
4. Scratch serve copy that appends the candidate mount to the `NPU_ARGS+=(--device`
|
||||||
|
line (`sed`), REAL files only (`readlink -f`; vendor forbids dangling symlinks
|
||||||
|
under /opt/xilinx), and run with:
|
||||||
|
`FRANK_DRAGOON_ENV_FILE=/tmp/... FRANK_DRAGOON_LOG=/tmp/... timeout 1200 /tmp/serve-nputest.sh`
|
||||||
|
5. Success = boot ledger (`… in all`) **plus** `npu_api: <model> … buckets u0…u9`
|
||||||
|
plus `/health` `npu.status ok`. Then codify the mount in the serve script, pin
|
||||||
|
it in `tests/frank_dragoon_unit.bats` (the fixture XRT dir's basename IS the
|
||||||
|
multiarch name — name it `x86_64-linux-gnu`), update the endpoint page's NPU
|
||||||
|
prerequisite to the three-prefix law, and cut over via PR +
|
||||||
|
`deploy.sh dankclaw --service frank-dragoon`.
|
||||||
|
6. Retire scratch: the `timeout` kills the proof container; `podman rm -f`,
|
||||||
|
move /tmp scripts to `~/.AGENTTRASH/<date>-<topic>/`. A proof instance
|
||||||
|
serving production while you wait is drift — deploy the merged fix promptly.
|
||||||
|
|
||||||
|
## Traps
|
||||||
|
- Greping image contents rootless: `podman mount` fails (needs `podman unshare`)
|
||||||
|
and leaves `$M` EMPTY, so `grep -r $M/opt` silently greps the HOST cwd — that
|
||||||
|
is how host `/opt/fastflowlm` masqueraded as image content. Always check `$M`
|
||||||
|
non-empty, or use `podman unshare bash -c 'M=$(podman mount C); …'`.
|
||||||
|
- The entrypoint swallows `podman run IMG sh -c …` as a mode name; use
|
||||||
|
`--entrypoint sh/cat/ls`.
|
||||||
|
- The proof container inherits production name/port unless the override is set
|
||||||
|
AFTER sourcing the env (the env file re-exports `FRANK_DRAGOON_CONTAINER`).
|
||||||
|
- `Restart=always` re-runs the lane into its own GPU/NPU debris; stop the unit
|
||||||
|
before any manual boot, don't fight the loop.
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
---
|
||||||
|
name: halogen-npu-xrt-third-prefix
|
||||||
|
description: "Triage a halogen frank-dragoon (flash-server) unit crash-loop caused by the NPU lane: signature \"No such library /opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2\", dead-lane-kills-container law, third-prefix mount fix, and the FRANK_DRAGOON_ENV_FILE scratch-boot proof recipe."
|
||||||
|
---
|
||||||
|
|
||||||
|
# Halogen NPU lane: dead lane kills the container; XRT needs THREE mount prefixes
|
||||||
|
|
||||||
|
Friction-earned 2026-10-21 on `frank-dragoon` (Dankclaw): unit boot-looped 88× (`activating`, exit 1 in ~200 ms) with the GPU/serve_api lines looking healthy; chat dead beside a fine-looking ledger.
|
||||||
|
|
||||||
|
## Signatures (in order of appearance)
|
||||||
|
|
||||||
|
```
|
||||||
|
halogen-npu: No such library '/opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2'
|
||||||
|
halogen npu: the NPU engine exited before it was ready (rc=1; ...)
|
||||||
|
```
|
||||||
|
then `systemctl show <svc> -p ExecMainStatus` = 1, `NRestarts` climbing, and `/home/dank/logs/frank_dragoon.log` truncated at ~10 lines each boot (serve script truncates the log per start).
|
||||||
|
|
||||||
|
**Laws:** a dead NPU lane is NOT survivable — the engine supervisor takes the WHOLE container down on it. And `Restart=always` makes each restart race its own GPU/NPU debris: the FIRST failure's message is the only real evidence; later loop entries are noise. Stop the unit immediately (`sudo systemctl stop frank-dragoon`) before probing.
|
||||||
|
|
||||||
|
## Root cause class
|
||||||
|
|
||||||
|
The NPU child `dlopen`s XRT from `/opt/xilinx/lib/<multiarch>/` IN ADDITION to `/opt/xilinx/xrt/lib` and the install path. Host package churn (dpkg.log reconfigure of `libxrt2` the previous day) made the child demand the third prefix; our mounts filled only two. A lane that worked for days with byte-identical mounts can die purely from host package changes — diff the mounts last, check `grep libxrt /var/log/dpkg.log` first.
|
||||||
|
|
||||||
|
Fix: mount each REAL file (never a symlink) at all three prefixes; derive the multiarch name from `FRANK_DRAGOON_XRT_DIR`'s basename (`MULTIARCH="${XRT_DIR##*/}"`).
|
||||||
|
|
||||||
|
## Proof recipe (production-shaped, no scratch port)
|
||||||
|
|
||||||
|
One card holds one server — never bench on a side port; use the sanctioned env-file seam:
|
||||||
|
|
||||||
|
1. Scratch env on the host: `source /home/dank/serverAppConfigs/scripts/frank-dragoon.env` + knob overrides (e.g. `HALOGEN_NPU_MODELS=...`).
|
||||||
|
2. Copy the serve script with the candidate change applied via `sed` into `/tmp` (or edit + `FRANK_DRAGOON_ENV_FILE`); export `FRANK_DRAGOON_ENV_FILE=<scratch>` and `FRANK_DRAGOON_LOG=/tmp/<name>.log`; run under `timeout 1200`. The seam exists precisely for this.
|
||||||
|
3. Judge from `/tmp/<name>.log`: `npu_api: <model> ... buckets u0…u9` = lane alive; `No such library` = still broken. Then `curl :7770/health` → `npu.status`.
|
||||||
|
4. Retire the same session: `podman stop -t 60` (weights unpin slowly), `podman rm -f`, kill wrapper procs BY PID (`pgrep -f` the scratch serve script — `pkill -f` a pattern containing your own ssh line kills the session), move scratch files to `~/.AGENTTRASH/<date>-<topic>/`, then prove `pgrep -f serve-<scratch>` is empty. Leftover scratch state silently hijacks the next deploy (same class as leftover bench drop-ins).
|
||||||
|
|
||||||
|
## Traps hit en route
|
||||||
|
|
||||||
|
- Rootless `podman mount` fails ("cannot run in rootless mode") and leaves `$M` EMPTY — a later `grep -r "$M/..."` then reads the HOST cwd and "finds" the answer in Loki WAL files. Never grep an unverified mount var; `podman unshare` or `--entrypoint` probe instead.
|
||||||
|
- Image ships NO `/opt/xilinx` at all: all XRT comes from host mounts; `docs/NPU.md` is not inside the image.
|
||||||
|
- `ssh host 'tail <log>'` runs on the DEV box if you forgot `ssh` — a missing-file error is not "the engine wrote nothing".
|
||||||
|
- Health `context` field echoes the RUNNING engine's ceiling — use it to prove which shape is live, unlike `/v1/models` name echo.
|
||||||
|
|
||||||
|
## Verification that the fix landed
|
||||||
|
|
||||||
|
`systemctl show frank-dragoon -p DropInPaths` empty; boot ledger shows `reserving the KV pool … (about 11.0 GiB)`, `… in all`, an `npu_api:` line, no clamp line; deployer prints `NPU lane verified — <model> embedded`.
|
||||||
@@ -1,60 +1,62 @@
|
|||||||
---
|
---
|
||||||
name: harden-precommit-secret-gate
|
name: harden-precommit-secret-gate
|
||||||
description: "Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer."
|
description: "Fix or build a pre-commit secret-scan hook that actually blocks: the -z path hole, line-wide placeholder exemptions, inert doc placeholders, glued report lines, self-test fixture paths — each with a red-first bats case."
|
||||||
---
|
---
|
||||||
|
|
||||||
Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer. Use when a secret gate is wrong (misses a leak) or too right (rejects an inert example and bricks the heartbeat).
|
A pre-commit secret gate that fails open is worse than no gate: it teaches the operator that scanning happened. Four defect classes, each with a black-box test that goes red against the broken version (serverAppConfigs PR #70, command-center #59 — 4 holes closed same night, all red first).
|
||||||
|
|
||||||
## The two failure directions
|
## 1. Non-ASCII paths fail open
|
||||||
|
|
||||||
A gate can be **too weak** (misses a real key) or **too strong** (rejects its own documentation and takes down an automated committer). Both are outages; the second is silent.
|
`git diff --cached --name-only` quotes non-ASCII paths (`"docs/\303\251.md"`), so `git diff` on that name returns **empty**, and a loop that `continue`s on an empty diff skips the file silently. Fail closed with NUL delimiting:
|
||||||
|
|
||||||
## Audit recipe for an existing gate
|
|
||||||
|
|
||||||
1. Read the hook and list what it actually inspects. The classic scope hole: `git diff --cached -- '*.env' '*.pem'` — an extension whitelist. A credential in `agent/config.yml`, a quadlet unit, or a SKILL.md is then invisible to the gate. The rule that survives: scan **the entire staged diff**, every path, and let only binary content out.
|
|
||||||
2. Test fail-closed behaviour: can the hook be made to print nothing and still exit 0? Common escapes — running outside a git repo, unreadable staged diff, a hook that pipes through `grep` and inherits `grep`'s exit code from the wrong stage, or `set -e` swallowing the verdict. Verdict must be: unknown ⇒ **block**.
|
|
||||||
3. Quarantine list, never tracked: `.env`, `auth.json`, `trust.json`, `*.pem`, `*.key`, `identity*` — and the `.gitignore` policy must quarantine env files **at any depth under any name** (`**/.env*`, not `.env`), or `config/prod.env` slips past.
|
|
||||||
4. Escape hatch must be explicit, per-commit, and loud: `ALLOW_SECRETS=1 git commit …`, and the hook says so when it honours it.
|
|
||||||
5. Gate the gate: a test that commits a fixture repo through the hook and asserts it blocks. `tests/secret_gate.bats` here covers: shape in an unexpected extension, bcrypt hash, JWT, placeholder/indirection allowed, prose allowed, binary ignored, unreadable diff refused, deletion of a credential never blocked, `ALLOW_SECRETS` override.
|
|
||||||
|
|
||||||
## Self-trip: the gate rejects its own example (live 2026-10-21)
|
|
||||||
|
|
||||||
Symptom: a systemd timer that commits machine-written files fails **every hour**, and the journal shows the gate's own refusal:
|
|
||||||
|
|
||||||
```
|
|
||||||
SECRET GATE: staged diff contains 1 secret-shaped string(s):
|
|
||||||
agent/managed-skills/git-secret-gate-hook/SKILL.md
|
|
||||||
Fake secrets must be inert (RFC 2606 domains, all-zero ids, `AKIA0000000000000EXAMPLE`) AND must not
|
|
||||||
```
|
|
||||||
|
|
||||||
The offender is the AWS **documentation** example key — entirely inert — and `AKIA[0-9A-Z]{16}` cannot tell it from a leak. So the gate's own advice about writing an inert fake is uncommittable by the gate. Auto-learned skills describe credential shapes for a living, so this recurs by construction, and an automated committer cannot use `ALLOW_SECRETS=1`: that would blind the gate for the *other* 64 staged files.
|
|
||||||
|
|
||||||
Diagnose, in this order:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
journalctl --user -u live-sync.service -b --no-pager | tail -25 # which file, which line
|
git diff --cached -z --name-only | while IFS= read -r -d '' f; do ...
|
||||||
git -C ~/.omp status --short | head # how much is queued behind it
|
|
||||||
# then scan the WHOLE staged queue, not just the reported file:
|
|
||||||
git diff --cached -U0 | awk '/^diff --git/{f=$3;sub("^a/","",f)} /^\+/ && !/^\+\+\+/ {
|
|
||||||
l=substr($0,2)
|
|
||||||
if (l ~ /sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|glpat-[A-Za-z0-9_-]{16,}|AKIA[0-9A-Z]{16}|xox[baprs]-[A-Za-z0-9-]{10,}|eyJ[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{10,}\.|-----BEGIN [A-Z ]*PRIVATE KEY-----/)
|
|
||||||
print f": "substr(l,1,90) }'
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The second count is the decision input: one hit ⇒ fix the line; many ⇒ the gate needs an exemption channel before anything lands.
|
Test: a tracked file whose name contains a non-ASCII character, holding a fake key on an added line, must be blocked.
|
||||||
|
|
||||||
Fixes, in order of preference:
|
## 2. Placeholder filters must be value-scoped
|
||||||
|
|
||||||
1. **Describe the shape instead of printing a matching literal** — `AKIA` followed by zeroes and `EXAMPLE`, spelled out in prose, cannot match. One line, the gate stays strict, the next timer run drains the queue.
|
`if echo "$line" | grep -q "$placeholders"` exempts the whole line, so a comment word vouches for the credential beside it:
|
||||||
2. **Per-line allow marker** honoured by the hook (e.g. a trailing `secret-gate: allow` on the flagged line) — durable, keeps documentation honest, still fail-closed everywhere else. Needs a test both ways: a marked line passes, an unmarked one still blocks. Note the risk honestly: an attacker-adjacent string can self-authorise, so the marker belongs in review-able prose tiers, not in a diff that also touches secrets config.
|
|
||||||
3. Never widen the regex to skip markdown, never run the automation with the override env var set.
|
|
||||||
|
|
||||||
Inert-fake rule for anything that must *show* a credential shape: RFC 2606 domains, all-zero ids, the word `EXAMPLE` — and if the shape itself trips the gate, describe it in prose rather than print it.
|
```python
|
||||||
|
token = "sk-…" # example config for the docs
|
||||||
|
```
|
||||||
|
|
||||||
## Prose must not be collaterally blocked
|
Filter only the captured value (the `sk-…` group), never the line. Two directions to test: `key = "example"` stays allowed; `token = "sk-<real>" # example` is blocked.
|
||||||
|
|
||||||
Markdown explaining credential handling is the bulk of what a knowledge repo commits. `xox` inside a sentence about Slack token prefixes, or a `sk-` prefix discussed abstractly, must pass; the shapes are anchored long-suffix patterns for exactly that reason. When a gate fires on prose, that is a bug in the gate — but if the prose contains a *literal* matching token, the fix is the prose (option 1 above), not the pattern.
|
## 3. Inert placeholders in documentation
|
||||||
|
|
||||||
## Verifying a gate change
|
A skill/README that *documents* a secret gate quotes the vendor's placeholder ids (`AKIA0000000000000EXAMPLE`, all-zero JWTs). Exempt them by value shape (all-zero / `EXAMPLE` suffix / RFC 2606 domain), or the machine tier stops committing — and the refusal usually lands on a heartbeat commit, where it looks like infrastructure failure, not policy. Also note: an unlanded fix means the checkout runs the **old** hook, so a landing can be blocked by the very bug the PR corrects; audit the staged diff and use the documented override (`ALLOW_SECRETS=1`, never `--no-verify`), recording the audit in the commit message.
|
||||||
|
|
||||||
Write the fixture repo in a temp dir, commit through the hook, assert block/allow for each case. Do not test the regex by grepping the hook's source for the pattern — a copy of the pattern in a comment is not a gate that runs.
|
## 4. Report-line assembly
|
||||||
|
|
||||||
|
`report+=(" $file: $line")` inside a loop that keeps only the last hit per file glues the next filename onto the previous line when a second hit exists — the report attributes the finding to the wrong file. Test a two-file staged set where both match, and assert each report line names exactly one existing path.
|
||||||
|
|
||||||
|
## 5. Self-test fixtures in the hook itself
|
||||||
|
|
||||||
|
The hook's own fixtures contain fake keys by necessity, so the gate would refuse any commit touching the hook or its tests (or a docs page quoting a key as documentation). Pin the bypass to specific paths and to added lines only, and print what was exempted:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
case "$f" in .githooks/pre-commit|tests/secret-gate.bats)
|
||||||
|
echo "secret-gate: self-test fixtures exempt: $f" >&2; continue;; esac
|
||||||
|
```
|
||||||
|
|
||||||
|
## The scan rule
|
||||||
|
|
||||||
|
Scan **added content only** — `git diff --cached -U0 | grep '^+[^+]'` — or every commit touching a file with an old key in context gets blocked, and the operator's response to a gate that blocks everything is to override it always.
|
||||||
|
|
||||||
|
## Test harness (fixture repo, never the real checkout)
|
||||||
|
|
||||||
|
`git init` a temp repo, `git config core.hooksPath .githooks`, copy the hook in, `git add` a payload, then run it with the env overrides the hook honours (point its cache/notify paths at `$TMP` so tests never touch the user's config). Pin, in order:
|
||||||
|
|
||||||
|
| Case | Expect |
|
||||||
|
|---|---|
|
||||||
|
| fake key on an added line | exit 1, message names the file |
|
||||||
|
| `key = "example"` | pass |
|
||||||
|
| real key + `# example` comment | exit 1 |
|
||||||
|
| non-ASCII filename with a key | exit 1 |
|
||||||
|
| two files with hits | report names both |
|
||||||
|
| self-test fixtures | pass, with the exempt line printed |
|
||||||
|
|
||||||
|
`bats -r` after each edit. `--no-verify` is forbidden on Frank's boxes: the hook IS the control.
|
||||||
|
|||||||
@@ -1,48 +1,49 @@
|
|||||||
---
|
---
|
||||||
name: html-report-ground-truth-audit
|
name: html-report-ground-truth-audit
|
||||||
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix, and prove structure + render"
|
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix wrong values, delete filler that restates git, verify the served artifact byte-identical, and flag unverifiable content to the operator."
|
||||||
---
|
---
|
||||||
|
|
||||||
Use after generating a large HTML report (e.g. via `repo-change-html-report`) whose claims were partly reconstructed from session memory. Fabricated numbers, invented code blocks, and silently-swallowed sections are the failure modes; the audit makes each traceable to the repo, a live command, or a recorded run.
|
Ground-truth audit of an HTML report (generated by an agent, for a human). Three defects hide in a plausible report: **wrong numbers, unverifiable claims, and filler that restates git for a reader who has git.**
|
||||||
|
|
||||||
|
## Law
|
||||||
|
|
||||||
|
Every numeric claim must be re-measured against the repo, not the narrative that produced it.
|
||||||
|
|
||||||
## Procedure
|
## Procedure
|
||||||
|
|
||||||
### 1. Enumerate every checkable claim
|
1. **Dump claims with file:line anchors.**
|
||||||
Grep the report for numerics, file paths, symbol names, and code blocks:
|
`sed 's/<[^>]*>//g' report.html | grep -vE '^\s*$' | grep -nE 'commit|file|line|fix|refactor' > /tmp/claims.txt`
|
||||||
- line counts (`wc -l` the real file)
|
Then enumerate the ground truth: `git log --oneline origin/main..HEAD` for commits; `git diff --stat origin/main..HEAD` for the per-file census — **sort it**: `--sort=-deletions` and `--sort=-creations` give the top movers, so the "largest single change" and "largest net code change" are computed facts.
|
||||||
- test counts (`grep -c '@test' tests/<f>.bats` — NOT suite-run totals: conditional skips make raw counts differ; run-observed totals are legitimate run history, keep them labeled as such)
|
|
||||||
- diff stats (`git diff --shortstat main..HEAD` — the footer's files/insertions/deletions)
|
|
||||||
- commit hashes (`git log --oneline`)
|
|
||||||
- "verbatim" code blocks (see 3)
|
|
||||||
- diagram node lists and captions (see 4)
|
|
||||||
|
|
||||||
### 2. Ground numeric claims with one command each
|
2. **Check code blocks compile.** Write every snippet to a temp file and `bash -n` each. Common breakages from HTML generation: `"`/`&` entities leaking into code, and `<br>` line breaks that split a line mid-pipe.
|
||||||
Fix every mismatch in place. Record the was/now pair — the audit table IS the evidence.
|
|
||||||
|
|
||||||
### 3. Verbatim code blocks: prove them
|
3. **Check the diagrams carry information.** A mermaid `flowchart LR` whose nodes merely restate the bullet list above it is decoration — delete it.
|
||||||
For a block labeled verbatim, the exact string must exist:
|
|
||||||
- live code: `grep -n '<distinctive line>' <file>`
|
|
||||||
- historical: `git show <sha> -- <file> | grep -E '^[+-].*(<token1>|<token2>)'`
|
|
||||||
If the block was fabricated (plausible-looking pseudo-code), replace it with the real excerpt. If condensed, the block header must say "condensed" and every shown line must be real.
|
|
||||||
|
|
||||||
### 4. Diagram claims: run the engine
|
4. **Check counts.** Commit and file counts must match `git rev-list --count` and `--name-only | sort -u | wc -l`. A report whose sections were written by different subagents drifts: each counts its own slice ("4 commits / 14 files" vs the true 11 / 25).
|
||||||
Behavioral diagrams (plan traces, closure graphs) must be re-derived by running the actual CLI:
|
|
||||||
`python3 scripts/service-map.py plan --host <h> <path>` — then edit node labels, captions, and the prose count ("four services") to match output. Cross-check derived-set diagrams against the source relation (e.g. `grep -rl 'Network=gluetun.container' */**/*.container` for who rides a tunnel).
|
|
||||||
|
|
||||||
### 5. Check for swallowed sections
|
5. **Flag what cannot be verified.** Test pass claims, external tool output, runtime behaviour — if the repo can't confirm it, say so to the operator instead of asserting it.
|
||||||
Large prior-session PUT ranges silently overwrite neighbors. Grep for headings/captions the plan promised; re-insert any that vanished, anchored on a fresh `read` of the region.
|
|
||||||
|
|
||||||
### 6. Structural balance
|
## Filler test (delete on failure)
|
||||||
Python tag-balance pass over the file:
|
|
||||||
- per-tag open/close counts (`div`, `pre`, `figure`, `svg`, `table`, `span`, `figcaption`)
|
|
||||||
- running-div-depth scan printing per-section balance to localize stray closers
|
|
||||||
Common defect: `.ln`/`.nt` pair emitted without its `.cg` wrapper → stray `</div>`; fix by opening `<div class="cg">` before the pair.
|
|
||||||
|
|
||||||
### 7. Render proof
|
- A `file → line-count → description` table restating `git diff --stat`? Delete — the reader has git.
|
||||||
`file://` is blocked on the shared browser — serve the dir: `bash` service `python3 -m http.server <port> --bind 127.0.0.1 --directory <dir>`, ready on that port. Then:
|
- A "what this means" / "conclusion" / "key takeaways" section? Delete — it is the same facts in softer prose. (Same rule as vault pages: point of use or gone.)
|
||||||
- `browser.open({name, url: 'http://127.0.0.1:<port>/<file>'})`
|
- Keep only what a reader cannot rebuild from `git log` + `git show`.
|
||||||
- DOM assertions via `tab.evaluate(() => …)` — NOT `tab.run` (that runs in Bun; no `document`). Sections are `<h2 id>` + siblings: scope queries to `document.body`/`document.querySelectorAll`, not `#ch5 …`.
|
|
||||||
- screenshot the repaired regions (scroll via `tab.evaluate(el.scrollIntoView())`), then `tab.close()` and kill the server proc (`write proc://<name>/kill`).
|
|
||||||
|
|
||||||
## Deliver
|
## Serve and verify the artifact
|
||||||
A was/now table of every fix with its ground command, plus the list of claims verified intact. State honestly which numbers are run-history and not recomputable.
|
|
||||||
|
**Verify the file that is served, not the source.** If a `public/` copy is byte-identical, edit both or the served artifact stays wrong. Then re-fetch:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
base=http://host:port/reports/x.html
|
||||||
|
for i in $(seq 1 30); do curl -fsS -o /tmp/got.html "$base" && break; sleep 1; done # service may still be activating
|
||||||
|
for i in 1 2; do curl -fsS "$base" | md5sum; done # stable across refetches
|
||||||
|
diff <(curl -fsS "$base") /path/to/source.html && echo "served == source"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Voice drift
|
||||||
|
|
||||||
|
If the operator asked for a tone ("no filler", "terse"), the audit must enforce it too — marketing phrasing ("ships the", "This means:") is a style defect, fix it in the same pass, not a later one.
|
||||||
|
|
||||||
|
## Report
|
||||||
|
|
||||||
|
Fixes grouped by defect class (wrong number / unverifiable / filler / style), then what stays unverifiable and why, with the measured ground truth quoted.
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
---
|
||||||
|
name: infra-repo-security-review
|
||||||
|
description: "Full-repo security review of an infra/config repo (serverAppConfigs-shaped: shell deployers, quadlets, systemd, Caddy edge, bats/Playwright) — slice fan-out, mechanical censuses, and subagent-claim verification."
|
||||||
|
---
|
||||||
|
|
||||||
|
# Whole-repo security review (infra/config repo)
|
||||||
|
|
||||||
|
For "full security review of this repo" on a deployment-config repo (shell scripts + podman quadlets + systemd units + Caddy + tests). NOT for a diff review — that is `code-review`. NOT for a secrets-only question — that is `git-repo-secret-audit`.
|
||||||
|
|
||||||
|
## 1. Slice, then fan out in ONE `task` batch
|
||||||
|
Five `security-reviewer` slices, each with the repo's own laws pre-injected in `context` so no agent re-derives them (AGENTS.md policy text + exclusions: docs prose, `.AGENTTRASH/**`, binaries, `package-lock.json`):
|
||||||
|
|
||||||
|
1. **Shell** — all `*.sh` + `.githooks/` + `tests/run.sh`: injection, quoting, `mktemp` vs `/tmp/fixed`, `set -e`, `curl -f`, secrets in argv, `rm`/`git clean`/`systemctl disable`.
|
||||||
|
2. **Python** — every `*.py`: `subprocess(shell=True)`, `verify=False`, unbounded `read()`, path traversal on request-supplied filenames, fixed `/tmp` writes, **which HTTP servers exist and which are unauthenticated**.
|
||||||
|
3. **Container + systemd posture** — every `*.container`/`.network`/`.service`/Containerfile.
|
||||||
|
4. **Edge + web** — Caddyfile vhost inventory, Anubis/Authentik chain, response headers, static-asset XSS sinks, service configs (`limiter.toml`, `prometheus.yml`, `alertmanager.yml`).
|
||||||
|
5. **Tests** — vacuous assertions + credentials/secrets in fixtures + harness safety.
|
||||||
|
|
||||||
|
**Set a yield deadline in the prompt.** Subagents here get killed at `task.maxRuntimeMs` (30 min) and the dead ones produced **zero** output. When one runs long, write early: `write agent://<id>` "yield now, unconfirmed items prefixed UNVERIFIED" — the resumed agent returns its partial report instead of losing it. Two of four needed this.
|
||||||
|
|
||||||
|
## 2. Verify every citation before reporting
|
||||||
|
`verify-subagent-review-findings` applies in full: re-grep each cited `file:line` and compare text. Expect ~40% of lines to be off (e.g. an agent described a 13-line hook; the file is 6) and several files to be hallucinated entirely (`.githooks/compose/*.container` did not exist). Discard mismatches; publish a **Discarded subagent claims** section. Re-grep your own citations too — two of mine were off by two lines and had to be corrected in an addendum.
|
||||||
|
|
||||||
|
## 3. Mechanical censuses (cheap, high yield — run these yourself)
|
||||||
|
- **Port binding**: every `^PublishPort=` line, classify `127.0.0.1:` / `192.168.0.x:` / **no address prefix** (= implicit `0.0.0.0`). The unprefixed set is the auth-bypass surface: a LAN client hits the container port directly and skips the Caddy→Anubis→Authentik chain that the vhost enforces. Report as one consolidated finding listing all unprefixed lines, and cite the repo's own correct examples (`127.0.0.1:9090`, `192.168.0.143:8080`) as the fix pattern.
|
||||||
|
- **Image pinning**: count `^Image=` lines with vs without `@sha256:` (this repo: 4 of 54 pinned) — floating tags + `Pull=never` = unreproducible edge.
|
||||||
|
- **Secret modes**: grep every `chmod 600` in the deployer and diff against every file it writes from an `.env`; a `>` redirect without the `chmod` is the finding (the Caddy/Porkbun env file).
|
||||||
|
- **Systemd hardening**: `User=`/`ProtectSystem`/`PrivateTmp`/`NoNewPrivileges` per unit, and `ExecStartPre=+` lines — a `+` hook on a root unit running a checkout-writable script is a local privesc.
|
||||||
|
- **Vhost table**: vhost → upstream → guarded? → exemption documented? Cross-check each upstream against the quadlet `PublishPort` list; unguarded vhost with no exemption comment is a policy violation even when the service self-authenticates.
|
||||||
|
- **Hook wiring**: `git config core.hooksPath` + `ls "$(git rev-parse --git-path hooks)"`. A repo can ship `.githooks/pre-commit` that git never executes — and its own test can assert the *file* is executable and still miss that it isn't installed.
|
||||||
|
- **Secret sweep**: `git cat-file --batch-all-objects --batch` (all history + dangling), match known values read from `~/.config/secrets`; report `prefix…(len N, sha 10)` only, never the value. Watch for the **split-concat disguise** (`PW="as""set"`): their own regex can't see it, but `grep -F` on a fragment finds it and history keeps it forever → rotation required.
|
||||||
|
|
||||||
|
## 4. Report
|
||||||
|
Severity-ranked table (Critical→Low, one line each: location + flaw), then prose detail for Critical/High only, then the Discarded-claims section, then remaining coverage gaps, then an ordered fix set and an offer to branch. Do not create a report file unless asked — chat is the deliverable.
|
||||||
@@ -1,50 +1,82 @@
|
|||||||
---
|
---
|
||||||
name: land-merged-pr-with-blocked-tree-ops
|
name: land-merged-pr-with-blocked-tree-ops
|
||||||
description: "Use when a merged PR must land in a checkout whose working tree is dirty and workspace policy blocks checkout/stash — prove the delta content-free, hand the operator the two-liner, verify via live config meanwhile."
|
description: "Use when a merged PR must land in a checkout whose working tree is dirty and workspace policy blocks git checkout/restore/stash: commit-the-dirt pattern, the secret-gate false positive that blocks the landing commit, conflict resolution proven settings-vs-comments, and heartbeat (live-sync) handoff."
|
||||||
---
|
---
|
||||||
|
|
||||||
Landing a merged PR when the deployment checkout is dirty **and** workspace policy blocks `git checkout`/`git stash`. Use when `git merge --ff-only origin/main` aborts with "Please commit your changes or stash them before you merge" in a SHARED workspace where other agents have work in flight. Earned 2026-10-20 on `dank/command-center` (the checkout IS the deployment).
|
Landing a merged trunk into a checkout that IS the deployment (dank/command-center `~/.omp`), when tracked files are dirty and policy forbids `checkout`/`restore`/`stash` on shared state. Re-earned 2026-10-21 landing PRs #61 + #63 into `~/.omp` (27 commits behind, two conflicts, one gate refusal).
|
||||||
|
|
||||||
# Land a merged PR with blocked tree ops
|
## 1. Classify the dirt before touching it
|
||||||
|
|
||||||
## Why it blocks
|
`git status --porcelain` then compare against what the merge brings in — only overlap blocks the merge:
|
||||||
|
|
||||||
`--ff-only` refuses when **any** incoming path is dirty in the worktree — it checks dirtiness against the *old* HEAD, not whether your content matches the target. Proven: overwriting the file so it is **byte-identical to the target commit** (`cmp` clean against `git show origin/main:<path>`) still aborts with the same message. So "make the file match main" is not a way in, and neither is `git merge` from another worktree (the branch is checked out in the dirty one).
|
|
||||||
|
|
||||||
The blocking set is what you must know, exactly:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
comm -12 <(git diff --name-only HEAD..origin/main | LC_ALL=C sort -u) \
|
{ git diff --name-only; git diff --cached --name-only; } | sort -u > /tmp/dirty.txt
|
||||||
<(git status --porcelain | awk '{print $NF}' | LC_ALL=C sort -u)
|
git diff --name-only HEAD origin/main | sort -u > /tmp/incoming.txt
|
||||||
|
comm -12 /tmp/dirty.txt /tmp/incoming.txt # these paths must be clean
|
||||||
```
|
```
|
||||||
|
|
||||||
`LC_ALL=C sort` matters — git's own output order trips `comm: file N is not in sorted order` and you get garbage overlap lists. In the case this was earned on, 12 incoming paths and 57 dirty ones intersected in exactly **one** file.
|
Note the first column of porcelain: `M ` = **staged**, ` M` = unstaged. A staged file shows nothing in `git diff` — look at `--cached` too, or you will "verify" a file is unchanged while it is staged. `comm` needs both inputs sorted; unsorted input prints "file is not in sorted order" and a partial answer.
|
||||||
|
|
||||||
## Procedure
|
## 2. Prove a dirty file content-free before resolving
|
||||||
|
|
||||||
1. **Fetch, don't merge yet.** List incoming commits (`git log --oneline HEAD..origin/main`) so you can show what is waiting.
|
`cmp` it against the trunk blob, never against memory:
|
||||||
2. **Compute the blocking set** with the `comm` above (both inputs `LC_ALL=C sort -u`).
|
|
||||||
3. **Prove the local delta is content-free**, in the direction that matters — lines the working tree has that `main` does not:
|
|
||||||
```bash
|
|
||||||
diff <(git show origin/main:<path>) <path> | grep -c '^>' # local-unique lines
|
|
||||||
```
|
|
||||||
Zero means the working tree is a strict subset of what was merged: discarding loses nothing.
|
|
||||||
4. **Preserve before you touch:** copy the current file to `~/.AGENTTRASH/` (snapshot), and copy anything you had moved into a gitignored `.AGENTTRASH/` inside a build worktree to its permanent home in the deployment — otherwise pruning the worktree destroys the only copy of a quarantined page.
|
|
||||||
5. **Do NOT work around the policy.** `git checkout -- <path>`, `git stash`, and `git stash push -- <path>` are all blocked in a shared workspace and the block is correct: it protects other agents' in-flight work. Do not reach for `git reset --soft` + `git restore --staged` to slide the pointer past it either — that leaves the index holding staged *reverts* of every incoming path, one careless `git commit -a` away from undoing the merge.
|
|
||||||
6. **Hand the operator the two-liner** and keep working on what does not depend on it:
|
|
||||||
```bash
|
|
||||||
git -C ~/.omp checkout -- <path> && git -C ~/.omp merge --ff-only origin/main
|
|
||||||
```
|
|
||||||
Say why it is safe (step 3's number, plus where the snapshots live) and what it unblocks.
|
|
||||||
|
|
||||||
## Why waiting is not free
|
```bash
|
||||||
|
git show origin/main:$f > /tmp/incoming
|
||||||
|
cmp -s "$f" /tmp/incoming && echo "IDENTICAL to trunk" # commit/discard is content-free
|
||||||
|
```
|
||||||
|
|
||||||
`scripts/live-sync.sh` merges with a plain `git merge --no-edit origin/main` (its line ~88), so the hourly heartbeat hits the identical wall and raises its ntfy alert `merge conflict with origin/main — local commit kept; needs you at the keyboard`. Report that consequence: the dirt is not merely untidy, it alerts.
|
If identical, the dirt is a sync artifact, not user work. Say so in the commit message with the `cmp` result — that is what makes a direct commit on `main` defensible for a prose-tier file.
|
||||||
|
|
||||||
Also note for later: `git branch -d <merged-branch>` **refuses until the commits reach `main`** — `-d` checks against the current HEAD, not `origin/main` — so local branch deletion goes on the post-landing list, not the pruning list.
|
## 3. Commit rather than discard (policy-blocked verbs)
|
||||||
|
|
||||||
## What you CAN do while it waits
|
With `checkout`/`restore`/`stash` unavailable, the only non-destructive unblock is a commit. Two separate commits, and order matters:
|
||||||
|
|
||||||
- **Verify the merged behaviour through the live config, not the branch pointer.** Merged prompt/config changes take effect from the deployment's files: spawn a fresh session/subagent and have it read and quote the changed file back. That proves the change is live even while `main` is behind.
|
1. Prose-tier dirt proven byte-identical to trunk — commit alone, message states `cmp` proved no prose change.
|
||||||
- Prune your own build worktrees, but check nothing is running inside them first: `ls -l /proc/*/cwd 2>/dev/null | grep -c "wt/<name>"` (the same guard `live-sync.sh` uses).
|
2. Machine tiers (`agent/memories/**`, `agent/managed-skills/**`, `agent/config.yml`, `agent/models.yml`, `dotfiles/`) — let the heartbeat do it: `LC_ALL=C scripts/live-sync.sh` stages allowlist only, commits, merges trunk, pushes, and **exits 0 even when it aborts the merge on conflict** (the alert goes to stderr + ntfy). `exit 0` does not mean landed — check `behind=` afterwards.
|
||||||
- Delete the merged remote branches (`git push origin --delete <branch>`, allowed on the local Gitea).
|
|
||||||
|
Do NOT `git add -A`: the index may already hold dozens of staged machine-tier files, and `git add <one file>` does not clear them — they ride along into your commit.
|
||||||
|
|
||||||
|
## 4. The gate refusal that blocks the landing commit
|
||||||
|
|
||||||
|
The pre-commit secret gate scans the **staged** diff, so a single-file commit still trips on previously staged content. It fires on inert placeholders inside the skill docs that document the placeholder (e.g. `AKIA0000000000000EXAMPLE` — the AWS all-zero example id in `git-secret-gate-hook` / `harden-precommit-secret-gate`). If the checkout is behind, it is running the **pre-fix** hook while the fix sits unlanded in trunk: the landing is blocked by the very bug the PR corrects.
|
||||||
|
|
||||||
|
Audit first, then override deliberately — never `--no-verify`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git diff --cached | grep -nE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|glpat-[A-Za-z0-9]{16,}|BEGIN [A-Z ]*PRIVATE KEY|eyJ[A-Za-z0-9_-]{20,}'
|
||||||
|
git diff --cached | grep -oE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}' | sort | uniq -c # what exactly, how many
|
||||||
|
ALLOW_SECRETS=1 git commit -F msg # message records the audit + that the gate ran
|
||||||
|
```
|
||||||
|
|
||||||
|
Record in the message: the hits are inert placeholders, which files they sit in, that this checkout runs the pre-fix hook, and that the refusal dies with this landing.
|
||||||
|
|
||||||
|
## 5. Merge conflicts: resolve by proving settings identical
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git merge --no-commit --no-ff origin/main
|
||||||
|
git diff --name-only --diff-filter=U
|
||||||
|
```
|
||||||
|
|
||||||
|
For each conflicted file, get all three blobs (`HEAD`, merge base, `origin/main`) and ask what the local side uniquely holds:
|
||||||
|
|
||||||
|
- **Prose / doc pages**: diff ours vs trunk; if ours only lacks the incoming trim, take trunk.
|
||||||
|
- **Real settings files (kdl/yaml/env)**: strip comments and diff — `diff <(grep -v '^\s*//' ours) <(grep -v '^\s*//' theirs)`. Empty output ⇒ only comments differ ⇒ taking trunk changes no deployed behaviour, and you can say that in the merge message. (2026-10-21: `desk-1-home.kdl` — modes byte-identical, trunk's comment corrected the false "no 240 Hz mode exists".)
|
||||||
|
|
||||||
|
Resolve without the blocked verbs by writing the trunk blob: `git show origin/main:$f > $f && git add $f`. Then validate the live artifact (`niri validate`, the suite) before committing the merge, and push.
|
||||||
|
|
||||||
|
## 6. Verify the push actually landed
|
||||||
|
|
||||||
|
`git push` output can be swallowed when wrapped in command substitution inside a batch — verify from the remote, not from the push:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git fetch -q origin && printf 'HEAD=%s origin/main=%s behind=%s\n' \
|
||||||
|
"$(git rev-parse --short HEAD)" "$(git rev-parse --short origin/main)" "$(git rev-list --count HEAD..origin/main)"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 7. Close and prune
|
||||||
|
|
||||||
|
- Close the issues with a comment giving merge SHA + PR link + title (numbers are per-repo; a bare `#N` is not a reference).
|
||||||
|
- Prune guards, all three before `git worktree remove --force`: `ls -l /proc/*/cwd | grep -c '<wt-name>'` must be 0; `git -C <wt> status --porcelain | wc -l` must be 0; `git rev-list --count main..<branch>` must be 0. Check for `.AGENTTRASH/**` inside the worktree first — removal destroys the only copy.
|
||||||
|
- `git branch -d` (not `-D`) once the branch is reached by main; `git push origin --delete <branch>` is allowed on the local Gitea.
|
||||||
|
- Delete the temp `-K` token config file in the same session that created it.
|
||||||
|
|||||||
@@ -0,0 +1,25 @@
|
|||||||
|
---
|
||||||
|
name: okf-small-bundle-docs-audit
|
||||||
|
description: "Audit + dedupe a small repo's docs bundle for OKF conformance when the kb-lint/kb-coverage toolchain does not exist: inline conformance and link-audit scripts, owner-per-fact cuts, index trio, one log."
|
||||||
|
---
|
||||||
|
|
||||||
|
# OKF docs audit for a small repo bundle (no kb-lint toolchain)
|
||||||
|
|
||||||
|
Use for a repo whose KB is one docs directory (e.g. `dankvoice` → `dankvoice-docs/`), where `scripts/kb-lint.sh` / `kb-coverage.py` do not exist. For the big `dank/command-center` campaign see skill `docs-dedupe-verify-campaign`; same frame, no scouts — a corpus under ~1500 lines gets read fully yourself (scouts lie).
|
||||||
|
|
||||||
|
## Frame
|
||||||
|
- OKF spec lives at `~/.omp/omp-docs/references/okf-spec.md` (the command-center deploy checkout). v0.1 conformance = every non-reserved `.md` has parseable frontmatter with non-empty `type`; `index.md` = listing with NO frontmatter; `log.md` reserved, exactly one, bundle root.
|
||||||
|
- Branch off `origin/main` in a fresh worktree; docs-only + src-comment changes still need the repo's normal gate.
|
||||||
|
|
||||||
|
## Procedure
|
||||||
|
1. Inventory `wc -l` every page; read all fully. Ledger per fact: which page OWNS it (laws page owns laws, decisions own rationale, contributing owns process, README owns user-facing, GLOSSARY owns terms). Everything else becomes a pointer.
|
||||||
|
2. Hunt the rot classes that repeat: dead test citations (name in docs not in `grep -c tests/` — verify with `git log --all -S`), hand-maintained censuses (test counts, scenario lists) — replace with a pointer to the authoritative source, never refresh the copy; deleted doc lines for live env vars (compare `git show <fixed>:file | grep -c VAR` vs HEAD); sections contradicting the page's own head; open items already resolved in live state (check the live settings file, not the doc).
|
||||||
|
3. Frontmatter: `type` + one-line `description` + `tags` + `timestamp`. On edit-rejection the tool prints the file's current `#TAG` — reissue with it, no re-read needed.
|
||||||
|
4. Conformance check (inline python): walk bundle dirs, regex `^---\n(.*?)\n---` then `^type: \S` on each non-reserved `.md`.
|
||||||
|
5. Link audit (inline python): per md file, `\]\(([^)#]+)`; skip http; `os.path.normpath(join(dirname, target))` must exist. Report `N checked / ALL RESOLVE`.
|
||||||
|
6. Create `index.md` at bundle root + each subdir (listing, no frontmatter, relative links) and `log.md` at the bundle root seeded with laws/incidents only.
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
- Src docstring fixes are artifact-free only if the bundler strips comments — prove with `bun run build; cmp dist/... old` before claiming no rebuild needed.
|
||||||
|
- README must NOT enumerate design records; point at `decisions/index.md` so new records need no README edit.
|
||||||
|
- Mark superseded decision records in their frontmatter `description` ("its X design is superseded by the Y record") — never rewrite history bodies.
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
{
|
{
|
||||||
"v": 1,
|
"v": 1,
|
||||||
"lastConsolidatedAt": 1792557324990
|
"lastConsolidatedAt": 1792574106585
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
{
|
{
|
||||||
"v": 1,
|
"v": 1,
|
||||||
"lastConsolidatedAt": 1792572225299,
|
"lastConsolidatedAt": 1792593705958,
|
||||||
"lastResult": {
|
"lastResult": {
|
||||||
"at": 1792557082020,
|
"at": 1792573818108,
|
||||||
"sessions": 1,
|
"sessions": 1,
|
||||||
"deltas": 1,
|
"deltas": 2,
|
||||||
"model": "frank-zealot"
|
"model": "frank-zealot"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,7 @@
|
|||||||
|
## Okf Format
|
||||||
|
|
||||||
|
- "okf format"
|
||||||
|
|
||||||
|
## Human Readability
|
||||||
|
|
||||||
|
- "human readability"
|
||||||
@@ -1,10 +1,10 @@
|
|||||||
{
|
{
|
||||||
"v": 1,
|
"v": 1,
|
||||||
"lastConsolidatedAt": 1792572021650,
|
"lastConsolidatedAt": 1792593561710,
|
||||||
"lastResult": {
|
"lastResult": {
|
||||||
"at": 1792571693973,
|
"at": 1792574016725,
|
||||||
"sessions": 1,
|
"sessions": 1,
|
||||||
"deltas": 1,
|
"deltas": 2,
|
||||||
"model": "frank-zealot"
|
"model": "frank-zealot"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,3 @@
|
|||||||
|
## Formatting
|
||||||
|
- okf format
|
||||||
|
- formatted for human readability
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"v": 1,
|
"v": 1,
|
||||||
"lastConsolidatedAt": 1792572038693,
|
"lastConsolidatedAt": 1792593638896,
|
||||||
"lastResult": {
|
"lastResult": {
|
||||||
"at": 1792556461435,
|
"at": 1792556461435,
|
||||||
"sessions": 1,
|
"sessions": 1,
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"v": 1,
|
"v": 1,
|
||||||
"lastConsolidatedAt": 1792572210019,
|
"lastConsolidatedAt": 1792593690234,
|
||||||
"lastResult": {
|
"lastResult": {
|
||||||
"at": 1792566095222,
|
"at": 1792566095222,
|
||||||
"sessions": 1,
|
"sessions": 1,
|
||||||
|
|||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"v": 1,
|
||||||
|
"lastConsolidatedAt": 1792574109325
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"v": 1,
|
||||||
|
"lastConsolidatedAt": 1792593583799
|
||||||
|
}
|
||||||
+11
-10
@@ -96,19 +96,20 @@ providers:
|
|||||||
# halogen flash-server on the Strix Halo iGPU. Payload, shape, eras,
|
# halogen flash-server on the Strix Halo iGPU. Payload, shape, eras,
|
||||||
# laws, rollback: serverAppConfigs-docs/LLM/frank-dragoon.md — and
|
# laws, rollback: serverAppConfigs-docs/LLM/frank-dragoon.md — and
|
||||||
# scripts/frank-dragoon.env is the authority for every number there.
|
# scripts/frank-dragoon.env is the authority for every number there.
|
||||||
# contextWindow sits deliberately BELOW the engine ceiling
|
# contextWindow: 200,000 under the native 262,144 engine ceiling
|
||||||
# (HALOGEN_CTX): the clamp below leaves the gap that absorbs the
|
# (HALOGEN_CTX, 2026-10-21) — the 400-zone law: the window MUST sit
|
||||||
# harness's chars/4 under-count, so a window equal to the ceiling
|
# ~45k under the ceiling, never at it.
|
||||||
# would make every long request a guaranteed 400.
|
contextWindow: 200000
|
||||||
contextWindow: 155648
|
|
||||||
# The clamp is max_tokens = min(maxTokens, contextWindow − est −
|
# The clamp is max_tokens = min(maxTokens, contextWindow − est −
|
||||||
# 4096); the chars/4 est UNDER-counts the rendered prompt by ~3 %
|
# 4096); the chars/4 est UNDER-counts the rendered prompt by ~3 %
|
||||||
# on code/agent traffic (measured 2026-10-07: est 152,938 ->
|
# on code/agent traffic (measured 2026-10-07: est 152,938 ->
|
||||||
# server 157,388). Worst-case request = window − 4096 + 3 % est
|
# server 157,388). Worst-case request = (200,000 − 4,096) × 1.03
|
||||||
# error ≈ 156.2k, inside the engine ceiling with ~44k of margin
|
# ≈ 201.8k; the compaction trigger fires at 95 % (config.yml
|
||||||
# (robust to a ~25 % under-count). Compaction fires at 85 % of the
|
# thresholdPercent): 190,000 × 1.03 + 45,056 ≈ 240.8k — both inside
|
||||||
# window and rides on frank-zealot: input ≈ threshold −
|
# 262,144 (margin 60k / 21k). Compaction rides frank-zealot:
|
||||||
# keepRecent(20k), comfortably inside its per-slot window.
|
# input ≈ 190,000 − keepRecent(20k) ≈ 175k ≤ its 200,704 slot.
|
||||||
|
# Recompute all of it after ANY window or threshold change; tests/
|
||||||
|
# frank_dragoon_unit.bats pins this arithmetic at 95 %.
|
||||||
# HALOGEN_MAX_TOKENS_CAP in frank-dragoon.env must stay >= this.
|
# HALOGEN_MAX_TOKENS_CAP in frank-dragoon.env must stay >= this.
|
||||||
maxTokens: 45056
|
maxTokens: 45056
|
||||||
reasoning: true
|
reasoning: true
|
||||||
|
|||||||
Reference in New Issue
Block a user