Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 dank
|
1dc01bd006 | ||
 Frank
|
336201e814 |
@@ -1,61 +1,41 @@
|
||||
---
|
||||
name: audit-subagent-deliverables
|
||||
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), adversarial-review-report triage, resource-safety pass (fd/memory), and cutover with running-process restarts."
|
||||
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), merge shared-function collisions, fix scope creep, verify env var propagation."
|
||||
---
|
||||
|
||||
# Audit subagent deliverables
|
||||
Integrating code written by subagents (fan-out implementation). The code is usually correct; the failures are structural — collisions between agents, scope creep, and assumptions that never reach the runtime.
|
||||
|
||||
A subagent that wrote code is a source of plausible output, not a verified change. Integrate it the way you'd integrate an untrusted PR — with extra suspicion, because the author is an agent that shares your blind spots and its prose is more confident than its evidence.
|
||||
## Workflow
|
||||
|
||||
Order of operations: read everything → gates → tests (as evidence, not proof) → resource-safety pass → fix in-place → cutover.
|
||||
1. **Read every deliverable line by line** before integrating — not the summary.
|
||||
2. **Run the mechanical gates immediately**, in parallel: `bash -n` on every script; `bats tests/ 2>&1 | grep "not ok"` for the failure list, never `tail` (tail shows a passing tail and hides the red in the middle).
|
||||
3. **Merge shared-function collisions.** When two agents add different versions of the same function, pick the superset and re-point the other's callers at it. (Real case: one agent's `fleet_base()` carried `-b`, the other's `resolve_fleet_base()` carried `--base-url`; resolve to the one flag set the tests use.)
|
||||
4. **Delete scope creep.** Code added for a goal nobody asked for — a new env var, a new CLI flag, a new output file — goes, with its docs, unless the operator asked.
|
||||
5. **Verify env var propagation.** A var the orchestrator computes must be exported before the `bash -c` / subshell that reads it, or the child silently takes a default. Check the export site, not the doc.
|
||||
|
||||
## 1. Read every produced file, line by line
|
||||
Full file, not a diff skim. You are the reviewer who must catch the line the author never thought about.
|
||||
## Audit checks
|
||||
|
||||
## 2. Mechanical gates before logic review
|
||||
```bash
|
||||
bash -n <script> # syntax (scripts)
|
||||
bunx tsc --noEmit # types (TS)
|
||||
bun test # suite
|
||||
```
|
||||
Run the suite for information, not as the stopping point — it only proves the author's own scenarios.
|
||||
| Check | How |
|
||||
|---|---|
|
||||
| Flag names real | `bash script.sh --help` |
|
||||
| Paths exist | `test -e` each literal in the diff |
|
||||
| Env reaches the runtime | grep the export/inline-assignment site, then the read |
|
||||
| Tests assert behaviour | not `[[ -n "$X" ]]`, not `test -f <new file>` |
|
||||
| Tests not weakened | `git diff HEAD~ -- tests/` — a deleted assertion is a red flag |
|
||||
|
||||
## 3. Hunt the three classic agent defects
|
||||
- **Quoting bugs:** `set -e` + `"${@/$p/$r}"` = substitution regex containing user input (bash can't escape `/` in a `${var/re/pat}` pattern: `/` reads as the pattern/replacement separator). Fix with a separator absent from inputs (e.g. `%`) and `printf '%q'` for shell-interpolated values.
|
||||
- **`cd` assumptions:** parallel `runShell` calls share one CWD (pool mode). Any command sequence that assumes CWD (e.g. `git -C .` after a sibling's `cd`) must use explicit absolute paths.
|
||||
- **Unbounded resources:** every fd opened must be closed on ALL paths (success, throw, validation fail) — `using` for fs.sinks (closes on throw; sinks are `AsyncDisposable`, not `fs.FileHandle`, no `.close()`); bounded `Map` for caches (`MAX` + FIFO eviction); explicit caps (lines read, file sizes).
|
||||
## Red flags
|
||||
|
||||
## 4. Check the test file is real
|
||||
Tests that assert internal state (`expect(wcwd).toBe(...)`) over output contracts; tests of mock echo; tautologies. Delete them per house rules, keep behavior contracts.
|
||||
- Test asserting a new file exists rather than behaviour.
|
||||
- Doc claiming a flag the script never defines.
|
||||
- Function added with no caller.
|
||||
- Doc example that contradicts the code it documents.
|
||||
- Same helper defined twice with different flag spelling.
|
||||
- New env var read but never exported.
|
||||
|
||||
## 5. Cutover audit (repo-wide)
|
||||
`grep -r` the symbol namespace the subagent touched — `wcwd`-style globals are the classic blast radius. Grep every `cd`/`process.chdir` site and verify each paired command sequence uses explicit paths. Stale comments describing the old design must go — comments that lie are time bombs for the next reader.
|
||||
## Re-verify after the operator's own edits
|
||||
|
||||
## 6. Cutover restarts running processes
|
||||
A code fix on disk is NOT live in any process that loaded the file at spawn (bun/node load-at-spawn). After shipping fixes: restart every long-running supervisor/daemon/UI-server that carries the affected code path; prefer takeover protocols (stale-pid detection) that make restarts cheap. Symptom of forgetting: "fixed" code producing old-code symptoms in production (observed: 812 poisoned ledger rows).
|
||||
Run the gates again — their edit may undo an integration (a re-typed helper, a reverted flag).
|
||||
|
||||
## 7. Triaging an adversarial REVIEW report (not code)
|
||||
When a subagent produced findings, treat each finding as a hypothesis with a base rate ~40% (measured over 3 rounds, 2026-10):
|
||||
1. **Walk every finding end-to-end against the CURRENT tree** before accepting — cite file:line yourself. Reviewers frequently read a mid-edit snapshot; many findings are stale by construction.
|
||||
2. **Classification that kills most findings:**
|
||||
- *Stale read* — code changed during review; verify against live lines.
|
||||
- *Inverted logic* — "failed rows are skipped" when the filter makes failed rows RETRY. Re-read the filter.
|
||||
- *Unreachable* — trigger requires breaking a stated design law (single-worker, append-only). Not a defect.
|
||||
- *By-design* — the reviewer labels a spec law a bug (serial execution, non-stream client). Check the spec, not the reviewer's confidence.
|
||||
- *Constant-trust* — "XSS via manifest field" where the field is git-pinned config; "path traversal via REMOTE_TMP" where it's a hardcoded const. Ask: what untrusted input reaches this sink?
|
||||
- *Self-refuting* — the reviewer's own probe output disproves its claim (raw==dedup probe in a "dedup" finding). Quote it back.
|
||||
3. **Acceptance bar:** concrete trigger with real inputs + full path walked. Real accepts cluster in: resource caps (pids/timeout/awaited-cleanup), honest labels (default signatures), escaping discipline, self-healing probes.
|
||||
4. **One-line verdicts + evidence per finding**, shipped in the commit message; calibration section ("genuinely solid") accepted as free verification of past fixes.
|
||||
## Report
|
||||
|
||||
## Verification (the non-negotiable)
|
||||
Build/run the real artifact with its real inputs. Observe behavior at the seams:
|
||||
- concurrent calls → no CWD races;
|
||||
- resource ceilings → caps enforced;
|
||||
- cleanup → no fd/pid/dir leaks;
|
||||
- security → injection probes inert (quotes, slashes, metachars);
|
||||
- normal case → output identical to pre-audit.
|
||||
|
||||
## Evidence (2026-10-07 + 2026-10-10/11 sessions)
|
||||
- Shell-engine subagent: 426 LOC, `bun test` 39/39 green. Line-audit found: 5 leaked fds per call (`fd` sink never closed), CWD-race in `git -C .`, `set -e` + `${@/\/g\//}` slash-escaping bug, unbounded 10k-entry cache, 45s timeout on git-status.
|
||||
- Review-triage census over 3 blind adversarial rounds (same repo, ~41 findings): accepted 9, rejected ~32 — rejects dominated by stale reads (protocol rewritten mid-review), inverted filter logic, unreachable-under-law triggers, base64-alphabet "quote injection" (charset has no quotes), SIGPIPE non-leaks, "no memory cap" quoting the `--memory 4g` line above the claim. Real accepts: fork-bomb pids cap, stalled-ssh upload cap, stale-flag self-heal probe, honest census label, freshness gate on pid reuse.
|
||||
- Poison-law incident: sandbox fix shipped while supervisor ran old build → 812 rows graded by the dead protocol; takeover made recovery cheap. The restart step (#6) exists because of this.
|
||||
Per deliverable: correct / changed / removed, with file:line, plus the gate output that proves it (`bats` failure list before and after).
|
||||
|
||||
@@ -1,49 +1,44 @@
|
||||
---
|
||||
name: docker-pin-latest-census
|
||||
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag census plus digest-equals-latest verification, no docker CLI needed."
|
||||
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag enumeration, the rolling-latest trap, digest pinning, and rollback proof."
|
||||
---
|
||||
|
||||
# Is my pinned image actually the latest?
|
||||
Use when asked whether a Docker Hub / lshr.io image pin is the latest version before bumping it, or when a manifest claims an image is stale. Registry API is the answer; the `latest` tag is not.
|
||||
|
||||
Use when bumping a pinned container image tag (e.g. `lscr.io/linuxserver/*` quadlet `Image=` lines) and "latest" must be PROVEN, not assumed. Works from any box with curl — no podman/docker daemon required. `lscr.io` fronts Docker Hub; its tags are NOT on quay.io (querying quay returns nothing and reads like "no newer tags exist" — false).
|
||||
|
||||
## 1. Full tag census via the registry API
|
||||
## The API
|
||||
|
||||
```bash
|
||||
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:<owner>/<repo>:pull" \
|
||||
| python3 -c "import json,sys;print(json.load(sys.stdin)['token'])")
|
||||
curl -s "https://registry-1.docker.io/v2/<owner>/<repo>/tags/list?n=2000" \
|
||||
-H "Authorization: Bearer $tok" | python3 - <<'PY'
|
||||
import json,sys,re
|
||||
tags=json.load(sys.stdin).get('tags') or []
|
||||
# linuxserver form: <version>-<sha>-ls<N>; group by version, keep newest ls per version
|
||||
v={}
|
||||
for t in tags:
|
||||
m=re.match(r'^(\d+(?:\.\d+)+)-([0-9a-f]+)-ls(\d+)$',t)
|
||||
if m: v.setdefault(tuple(map(int,m.group(1).split('.'))),[]).append((int(m.group(3)),t))
|
||||
for k in sorted(v)[-5:]:
|
||||
print('.'.join(map(str,k)),'-> newest:',max(v[k])[1]) # max on (ls#, tag) tuple
|
||||
PY
|
||||
# hub: token then paginate (the list is 100 per page)
|
||||
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" | jq -r .token)
|
||||
for off in 0 100 200 300 400 500; do
|
||||
curl -s -H "Authorization: Bearer $tok" \
|
||||
"https://hub.docker.com/v2/repositories/${repo}/tags?page_size=100&ordering=last_updated&page=${page}" \
|
||||
| jq -r '.results[] | "\(.name)\t\(.last_updated)"'
|
||||
done | sort
|
||||
```
|
||||
|
||||
- Sort versions as **tuples of ints**, never strings (`1.4.10 < 1.4.9` lexicographically).
|
||||
- Keep the max `ls` build per version, then the newest version — both can lag independently.
|
||||
- Trap already eaten: joining the tuple key to a string mid-loop then re-looking-it-up → `KeyError`. Keep keys as tuples; format only when printing.
|
||||
- Some channels have NO `latest` tag in `/tags/list` (or the endpoint truncates) — absence there is not evidence.
|
||||
|
||||
## 2. Digest-proof the candidate against `latest`
|
||||
|
||||
```bash
|
||||
for t in latest <candidate-tag>; do
|
||||
curl -sI "https://registry-1.docker.io/v2/<owner>/<repo>/manifests/$t" \
|
||||
-H "Authorization: Bearer $tok" \
|
||||
-H "Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.docker.distribution.manifest.v2+json" \
|
||||
| grep -i '^docker-content-digest'
|
||||
done
|
||||
# lscr.io (linuxserver): tags are semver, `latest` is a rolling alias
|
||||
curl -s https://images-api.github.com/v2/linuxserver/${repo}/tags/list # wrong host for lscr
|
||||
curl -s "https://hub.docker.com/v2/repositories/linuxserver/${repo}/tags?page_size=100" | jq -r '.results[].name'
|
||||
```
|
||||
|
||||
Equal digests ⇒ the pinned tag IS what `latest` serves right now — the strongest "ensure latest" evidence, and recordable in the endpoint page's pin note (the dank-fleet law: *pinned; verified == the registry's `latest` manifest digest at pin time*). A multi-arch index digest matches only if both requests send the same Accept set — send all three types above.
|
||||
Docker Hub's own tag list (`hub.docker.com/v2/repositories/<repo>/tags`, ordered by `last_updated`) is the practical source. The registry API (`/v2/<repo>/tags/list`) needs the bearer token flow above; both return tags, neither sorts by semver.
|
||||
|
||||
## 3. Bump through git, not by hand
|
||||
## The `latest` trap
|
||||
|
||||
Edit the quadlet/unit `Image=` line + the vault page's pin row (one owner each), run the repo suite, PR, deploy `--service <svc>` only. Rollback stays "revert one line".
|
||||
- `latest` is a **rolling alias**, not a version. "Pin is older than `latest`" says nothing about whether a *versioned* tag exists.
|
||||
- `latest` can lag or be rebuilt without a version bump; comparing a digest pin against it proves nothing.
|
||||
- For linuxserver images: `stable` is the rolling branch and version tags (`2026-09-30`) are what you pin; multi-arch manifest lists mean the same tag has per-arch digests — the pin must be the manifest-list digest (`RepoDigests[0]`) you actually pull, or a re-pull on another arch silently changes the image.
|
||||
|
||||
## Verification recipe (before bumping a pin)
|
||||
|
||||
1. Read the pinned tag and digest from the manifest/quadlet.
|
||||
2. Enumerate tags from the registry API (paginate; `last_updated` is not semver order).
|
||||
3. Compare **versions**, not recency: does a version greater than the pinned one exist?
|
||||
4. If bumping: pull the new tag and record `docker/podman inspect --format '{{index .RepoDigests 0}}'` as the new digest — pin the digest, not the tag.
|
||||
5. Prove the service: it starts, version endpoint answers the new version, no crash-loop, and the rollback is the previous digest.
|
||||
|
||||
## Report
|
||||
|
||||
"pinned: X (digest …), latest available: Y, upstream changelog: …" — or "pin is current: no version tag newer than X exists". Never claim staleness from the `latest` tag alone.
|
||||
|
||||
@@ -1,62 +1,29 @@
|
||||
---
|
||||
name: fleet-service-runtime-census
|
||||
description: "Answer \"what is service running right now?\" for a dank-fleet service: live endpoint probe plus host units, image ids, live-vs-repo env, and drop-ins."
|
||||
description: "Answer \"what is service running right now?\" for a dank-fleet service: unit state, container image/digest, endpoint contract, loaded env — machine over repo, in that order."
|
||||
---
|
||||
|
||||
# Fleet service runtime census
|
||||
Answer "what is service running right now?" for a dank-fleet service without trusting the repo. Repo state is intent; the machine is truth, and they differ by however long since the last deploy.
|
||||
|
||||
Answer the question "what is `<service>` running currently?" with evidence, not from docs. Docs say what *should* run; the host says what *does*. Run all five probes, then report one table.
|
||||
## Order of operations (cheap → expensive)
|
||||
|
||||
## 1. Probe the published port (works from any LAN box, no ssh)
|
||||
1. **Unit state on the host** (rootless quadlet ⇒ `--user`; some units are root):
|
||||
`ssh -o BatchMode=yes host 'systemctl --user show S -p ActiveState -p SubState -p ExecMainStartTimestamp -p NRestarts'`
|
||||
Boot-loop tell: `NRestarts` climbing with a short `ExecMainStartTimestamp`.
|
||||
2. **The container's own view**: `podman ps --filter name=S --format '{{.Image}} {{.Status}} {{.Names}}'`, then `podman inspect` for `Image` (tag **and** `RepoDigests`) and `Env`.
|
||||
3. **The endpoint contract, from the service itself** — this is the only answer that matters to a consumer:
|
||||
`curl -s host:port/v1/models` (served model id), `curl -s host:port/props` (`n_ctx`, `n_ctx_train`, caps), `curl -s -o /dev/null -w '%{http_code}' host:port/health`.
|
||||
4. **Effective flags**: the live process is the truth. `systemctl --user cat S` shows `EnvironmentFile=`; read that file. A repo `scripts/S.env` can be ahead of, or behind, what the running unit loaded — and a unit restart is what re-reads it.
|
||||
5. **Logs last**: `journalctl --user -u S -n 80 --no-pager` for the boot ledger / refusal messages, after you know whether it is even up.
|
||||
|
||||
```bash
|
||||
curl -s -m 10 http://<host-ip>:<port>/health; echo
|
||||
curl -s -m 10 http://<host-ip>:<port>/v1/audio/voices # per-endpoint extras: /v1/models, /metrics
|
||||
```
|
||||
## Facts that bite
|
||||
|
||||
The extra route matters more than `/health` for identity claims:
|
||||
- llama.cpp and radiance **echo the requested model name** — a 200 proves nothing; read `/v1/models`.
|
||||
- halogen 404s `/completion`, `/ping`, `/props`, `/tokenize`.
|
||||
- The frank-tts façade answers `/health` **without dialing the engine**, so "façade healthy, engine down" is a real state; the voice list in that same response is the honest census.
|
||||
- **A service is up ≠ configured how you think.** llama.cpp's `/v1/models` reports `max_model_len`, not the engine's per-slot ceiling; halogen reports `n_ctx` per slot. Read `/props` before quoting any window number.
|
||||
- **`systemctl is-enabled` answers enablement, not activity.** A timerless service reports `linked`/`static`; `deploy.sh` that prints `ENABLE` for it is lying. Read `systemctl is-enabled --quiet` exit status, not the text.
|
||||
- **`NRestarts: 0` with `ActiveState=activating` is a crash-loop only if the timestamps move**; check `ExecMainStartTimestamp` twice, 5 s apart.
|
||||
- **Read-only is always allowed** from the dev box (`ssh -o BatchMode=yes`); state changes on remote boxes are not, without a per-run OK. Production services (`frank-archon`, `frank-dragoon`, `frank-zealot`) are never stopped/restarted to "see what happens".
|
||||
- **Digest-pinned images**: "when was this image updated?" is answered by `podman inspect` `RepoDigests` + the registry API, not by the unit's start time (a restart re-uses the local digest).
|
||||
|
||||
## 2. Units + containers on the host (ssh as the deploy user, no sudo)
|
||||
## Report
|
||||
|
||||
```bash
|
||||
ssh <host> 'systemctl --user status <svc>.service <engine>.service --no-pager \
|
||||
| grep -E "^(●| Active| Loaded)"; \
|
||||
podman images --format "{{.Repository}}:{{.Tag}} {{.ID}} {{.Created}}"; \
|
||||
podman ps -a --format "{{.Names}}\t{{.Status}}\t{{.Image}}"'
|
||||
```
|
||||
|
||||
- `active (running) since …` + `Up N hours (healthy)` is the answer; `activating (auto-restart)` with a ~50 ms exit is a **preflight refusal**, not an engine fault.
|
||||
- **NEVER prefix these podman calls with `sudo`.** Containers live in the deploy user's rootless storage; `sudo podman inspect frank-tts-engine` returns `no such object` while the plain form succeeds. Treating that as "container missing" is a false finding.
|
||||
|
||||
## 3. Drop-ins — the override that no health gate sees
|
||||
|
||||
```bash
|
||||
ssh <host> 'systemctl --user show <svc>.service -p DropInPaths; \
|
||||
systemctl --user show <engine>.service -p DropInPaths'
|
||||
```
|
||||
|
||||
`DropInPaths=` must be empty. A leftover bench drop-in (`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`) makes the next deploy boot a different model at a different shape while `/health` still answers 200 and the e2e gate passes.
|
||||
|
||||
## 4. Live env vs repo env — the only truth test
|
||||
|
||||
```bash
|
||||
# repo side (local checkout)
|
||||
grep -vE '^\s*#|^\s*$' scripts/<svc>.env
|
||||
# host side
|
||||
ssh <host> 'sudo cat /home/dank/serverAppConfigs/scripts/<svc>.env' | grep -vE '^\s*#|^\s*$'
|
||||
```
|
||||
|
||||
Diff them. Per repo law the env file beside the serve script **is** the authority; a doc page that disagrees with it is the bug, and host-side drift means an uncommitted edit. Also check `git log --oneline -3 -- scripts/<svc>.env` to confirm the host isn't ahead of git.
|
||||
|
||||
## 5. Report
|
||||
|
||||
Table of: unit state + since, image repo:tag@short-ID + age, backend/alias actually served (from step 1, not from the name), drop-ins, and whether live env == repo env. State explicitly when a fact came from docs rather than from a probe.
|
||||
|
||||
## Traps that produce false findings
|
||||
|
||||
- Digest-pinned images: the local image ID's *creation date* answers "is this stale" better than the tag; `latest` + `Pull=never` is a locally built image, not a registry pull.
|
||||
- `ssh` user context changes what podman sees (root vs rootless) — see step 2.
|
||||
- Two ports on one host are two providers in `models.yml`; a port mismatch is a rename or drift, not a config typo.
|
||||
One line per layer with the evidence: unit (`active (running)`, since, restarts), container (image + digest), endpoint (served id, `n_ctx`, health code), loaded env (file path + the knobs that matter), and explicitly which of those came from the machine versus the repo — that gap is the answer to most "why is it behaving like that" questions.
|
||||
|
||||
@@ -1,35 +1,37 @@
|
||||
---
|
||||
name: frank-job-hunt-maintenance
|
||||
description: "Maintain frank-job-hunt agents: live-census gate hardening loop, registry locator verification, and the workstation/Dankserver/Dankclaw deploy-sync ritual."
|
||||
description: "Maintaining frank-job-hunt: the census gate's fail-closed rules, ATS locator token derivation, and the two-host deploy-sync verification."
|
||||
---
|
||||
|
||||
# frank-job-hunt maintenance
|
||||
Use when maintaining `frank-job-hunt`: the live-census gate, the registry locator, or Dankserver/Dankclaw deploy-sync. The repo is built and deployed; agents live on Dankserver, ntfy on Dankclaw.
|
||||
|
||||
Repo `~/Projects/frank-job-hunt` (Gitea `dank/frank-job-hunt` on Dankclaw 192.168.0.143:3000). Production: Dankserver user timers `frank-job-sweep.timer` 07:30, `frank-job-digest.timer` 08:15 (`Linger=yes`). ntfy relay: Dankclaw quadlet `ntfy.service`, route `ntfy.franksdank.cloud` (no anubis), LAN record in `hosts/custom.list` (Pi-hole v6 file at `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list`; v6 API paths are hostile — edit the file, then `podman exec pihole pihole restartdns`). Secrets: `~/.config/frank-job-hunt.env` (600, workstation + Dankserver).
|
||||
## The two rules that caused every bug here
|
||||
|
||||
## Law (validated 2026-10-11)
|
||||
- The fleet judge (`frank-zealot`, dankclaw:7771) is a **ranker, never a gate** — it inflates anything reaching it (non-technical roles scored 88-91). All discrimination lives in deterministic gates (`agents/lib/gates.ts`); zealot only ranks survivors.
|
||||
- Thinking-model fault: verdict lands in `reasoning_content` with `content:""` → judge sends `chat_template_kwargs:{enable_thinking:false}` (0.7s clean JSON).
|
||||
- **Family beats topic**: role family (recruiter/sales/finance/PM/designer/people-ops/revops/…) gates `gate:scope` unconditionally — no infra-token override ("DevOps Recruiter" is recruiting). Compound technical titles keep the `INFRA_TITLE` override via `NEG_TITLE` branch.
|
||||
- **gate:region**: remote is US-only (visa reality). "Remote - Bangalore" = onsite abroad → gate; `US` regex pass wins over `NONUS` match ("U.S. & Canada" passes).
|
||||
1. **A URL is data.** The census gate and the registry locator treat portal URLs as opaque strings; nothing may template a URL from a company name. `jobs.dankclaw.example` is a fixture host (RFC 2606) — never derive a real domain from a company name, and never click a link the pipeline constructed.
|
||||
2. **A stale row is worse than a missing row.** Every gate must fail closed: an unverifiable posting is reported as unverifiable, not kept as live. Deleting is allowed; asserting is not.
|
||||
|
||||
## Gate hardening loop (live-census, TDD)
|
||||
1. `bun agents/sweep.ts --dry` (full lane ≈ 30–90 s; 852 postings ≈ 62 pass / 790 filed).
|
||||
2. Classify leaks in the top-10: family gap / region gap / scoring tail. Accept long tail (Director/Founder edge cases) — digest + human curation is the design.
|
||||
3. Token fix in `gates.ts` regexes (`NEG_FAMILY` = no override; `METRO`/`US`/`NONUS` for arrangement). NEVER dedupe-check by memory — read file first; the file has bitten line-offset edits repeatedly.
|
||||
4. Pin every leak with a named regression test in `agents/test/gates.test.ts` before/with the fix.
|
||||
5. `bun test agents/test/ && bun x tsc --noEmit` — hooks re-run these at commit.
|
||||
6. Ship: `git commit` (pre-commit = tests + tsc + docs-link + secret scan) → `git push` → `ssh dankserver 'cd ~/Projects/frank-job-hunt && git pull'` (timers pull from master; Dankserver drift = tomorrow's run is stale).
|
||||
## Census gate
|
||||
|
||||
## Registry verification (monthly, `agents/verify-registry.ts`)
|
||||
Scouts' `ats`/`ats_identifier` are guesses (14/84 were wrong). Ritual:
|
||||
1. `bun agents/verify-registry.ts` (dry, ~70 s serial) → REPAIR census.
|
||||
2. `--write` applies; repairs with live jobs preferred over empty boards (0-job boards 404 later).
|
||||
3. Unresolved = honest manual lane (vanity Workdays, self-built ATS incl. majors, neogov) — NOT a code bug; per-entry error census in sweep output is the same signal.
|
||||
4. Update `targets/README.md` census lines + `docs/log.md`; commit/push/pull (same ship step).
|
||||
`scripts/census-gate.sh` decides keep / stale / unverifiable per row.
|
||||
|
||||
## Pitfalls
|
||||
- `--ats=<x>` smoke filters the census line — "N API-sweepable" reflects the filter, not the registry; don't diagnose a phantom bug.
|
||||
- Workday vanity careers URLs (`careers.uw.edu`) are NOT CXS-API-resolvable; tenant-only identifiers are (site path required).
|
||||
- Root `README.md` vs `targets/README.md` name collision in edit anchors: use full path headers.
|
||||
- Zealot endpoint probe: `curl http://192.168.0.143:7771/v1/models`.
|
||||
- Live proof = HTTP GET on the posting URL against the ATS API or the page itself; `404`/`410` ⇒ stale, non-200 with no body ⇒ unverifiable. Never infer from age alone.
|
||||
- The gate runs against the *live* registry, so a row that passes here and then fails in production means the registry drifted — re-run before blaming the gate.
|
||||
- Test seams: fixture ATS server + `…_BASE_URL` env. Tests assert the row's fate (kept / dropped / flagged), never internal state.
|
||||
|
||||
## Registry locator
|
||||
|
||||
Resolves which ATS a company uses (Greenhouse `boards-api.greenhouse.io/v1/boards/<token>/jobs`, Lever `api.lever.co/v0/postings/<token>`, Ashby `api.ashbyhq.com/posting-api/job-board/<token>`, Workable, SmartRecruiters).
|
||||
|
||||
- Token ≠ company name: it comes from the discovered posting URL, so a locator change must keep the derivation (URL → token), not substitute a slug.
|
||||
- Unknown ATS ⇒ unverifiable, not "no jobs". "Zero jobs" is a strong claim; only a successful API answer with an empty list may say it.
|
||||
- Rate limits: 429 ⇒ back off and mark unverifiable for this pass, never stale.
|
||||
|
||||
## Deploy-sync (Dankserver agents, Dankclaw ntfy)
|
||||
|
||||
- Two hosts, two roles: inference/agents on Dankserver, notification sink on Dankclaw. A deploy that only touches one leaves the pair inconsistent — the agent can post to a sink that no longer accepts its token, silently.
|
||||
- Deploy is `scripts/deploy.sh` from the correct checkout (dev side, clean tree); it pushes the branch first, so a stale local `main` makes the push fail — `git merge --ff-only origin/main` before deploying after any externally merged PR.
|
||||
- Verify after deploy on the *served* surface: the agent answers a probe, and a test ntfy message arrives on the phone. "The unit is active" is not the proof.
|
||||
|
||||
## Tests
|
||||
|
||||
Black-box, fixtures over network: fake ATS servers on localhost, fake ntfy sink, temp registry file. Assert output rows and exit status. Never assert that an env var is set or that a helper was called.
|
||||
|
||||
@@ -13,7 +13,10 @@ http://dank:<40-char-token>@192.168.0.143%3a3000
|
||||
|
||||
```bash
|
||||
tok=$(sed -nE 's#^[a-z]+://[^:]+:([^@]+)@.*#\1#p' ~/.config/secrets/gitea | head -1)
|
||||
curl -s -K <600-perm cfg file> http://192.168.0.143:3000/api/v1/<path>
|
||||
echo "tok len ${#tok}" # must be 40 — assert before any call
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"
|
||||
curl -s -K "$cfg" http://192.168.0.143:3000/api/v1/<path>
|
||||
rm -f "$cfg" # in the SAME session: the file holds a live token
|
||||
```
|
||||
|
||||
## `git credential-store --file … fill` answers EMPTY — do not debug it
|
||||
@@ -24,20 +27,21 @@ Tried 2026-10-21 while opening `dank/command-center` PR #61: the obvious extract
|
||||
printf 'protocol=http\nhost=192.168.0.143\nport=3000\n\n' | git credential-store --file ~/.config/secrets/gitea fill
|
||||
```
|
||||
|
||||
prints **nothing**, with no error. It is not a permissions problem and not a wrong path: the stored line encodes the port inside the host field (`@192.168.0.143%3a3000`), so a `host=`+`port=` query can never match it. Add `path=` or drop `port=` and it still answers empty. Read the file with `sed` (recipe above) and move on — the chain `TOKEN=$(…) && [ -n "$TOKEN" ] && …` fails silently at the test, which reads like a curl failure if the `&&` hides it.
|
||||
prints **nothing**, with no error. Not permissions, not a wrong path: the stored line encodes the port inside the host field (`@192.168.0.143%3a3000`), so a `host=`+`port=` query can never match it. Add `path=` or drop `port=` and it still answers empty. Read the file with `sed` (recipe above) and move on — the chain `TOKEN=$(…) && [ -n "$TOKEN" ] && …` fails silently at the test, which reads like a curl failure because the `&&` hides which link broke.
|
||||
|
||||
## The silent-empty-token trap (bit 2026-10-21, cost a debug detour)
|
||||
|
||||
An `awk -F= '/^username=/{...}'` parse of this file yields `tok=""` **silently** — and reads still appear to work when the repo is **public**: `dank/serverAppConfigs` and `dank/command-center` return real issue/label data unauthenticated, so every census "passes" until the first **write** (issue comment, label, PR create/merge) fails with `{"message":"token is always required"}`-style errors or 404s (private repos 404 on an empty token — that is an auth problem, NOT a missing repo).
|
||||
|
||||
- `echo "tok len ${#tok}"` (must be 40) before believing ANY API result — reads included.
|
||||
- `curl -s -K <600-perm file>` carrying `header = "Authorization: token $tok"` — never `-H` on argv (`/proc/<pid>/cmdline` is world-readable; this leaked an HF token once).
|
||||
- Prove auth once per session: `GET /api/v1/user` → `"login":"dank"`.
|
||||
- The temp `-K` config holds a live token: `rm -f` it in the same session that made the calls, and never `cat` it.
|
||||
- `curl -s -K <600-perm file>` carrying `header = "Authorization: token $tok"` — never `-H` on argv (`/proc/<pid>/cmdline` is world-readable; this leaked an HF token once).
|
||||
- `jq -n --arg t … --arg h … --arg b "$(/bin/cat /tmp/pr.md)" '{…}'` — `$(cat file)` gets optimised away in this harness, so use `--rawfile b /tmp/pr.md` instead.
|
||||
|
||||
## Other notes
|
||||
|
||||
- `git credential fill` fails headless here (`could not read Username … terminal prompts disabled`); read the store file directly instead.
|
||||
- Never print the token: mask when inspecting the file —
|
||||
`sed -E 's#(://[^:]+:)[^@]+@#\1REDACTED@#' ~/.config/secrets/gitea`
|
||||
- Merge writes need form-encoded fields, not JSON — see `gitea-pr-api-cycle`.
|
||||
- Issue/PR numbers are **per repo**: `dank/dankvoice` #8–#11 (refactor, hardening, AbortError fix, menu defects) are unrelated to `dank/command-center` #8–#11 (SDLC, trunk-law). Read the body from the repo the work targets before referencing an issue in a commit message.
|
||||
|
||||
@@ -1,41 +1,48 @@
|
||||
---
|
||||
name: gitignore-trailing-slash-symlink-contamination
|
||||
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get committed; detect with git ls-tree mode 120000, fix with git rm --cached."
|
||||
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get tracked and pushed as 1-line blobs; same-name rules silently match worktree dirs, and .deploy-stale pollutes. Three fixes with tests."
|
||||
---
|
||||
|
||||
A `.gitignore` entry written with a **trailing slash matches directories only**. A symlink of the same name is not a directory, so it slips past the rule, and `git add -A` in a `git worktree` checkout silently **tracks** it. Cost measured 2026-10-19: a machine-specific symlink (`node_modules -> /home/blank/Projects/dankvoice/node_modules`) landed in `main` with a merged PR.
|
||||
Gitignore semantics trap with three real-world consequences (serverAppConfigs, 2026-10-19):
|
||||
|
||||
Why worktrees are the trigger: a worktree has no `node_modules` (or other installed-deps dir) of its own, so the cheap way to make build/test work there is a symlink into the primary checkout. With `node_modules/` ignored, nothing complains.
|
||||
## The rule
|
||||
|
||||
## Detect
|
||||
`.gitignore` patterns ending in `/` match directories only. Worktree-relative: `**/node_modules/` is `**/node_modules/` → a symlink named `node_modules` does NOT match, so it is "untracked" no more — it gets added, committed, and pushed as a 1-line blob containing its target path.
|
||||
|
||||
## Consequence 1: worktree symlink contamination
|
||||
|
||||
- `git add -A` then commit then push in a worktree whose root holds `node_modules -> ...` (the worktree convention symlink) commits it.
|
||||
- The repo then holds `package/frontend/node_modules` as a text file containing the absolute path on the author's box.
|
||||
- The receiving worktree gets that 1-line file instead of the symlink → "module not found" and "why is this a 30-byte file?".
|
||||
- Detection: `git ls-tree -r origin/main | awk '$1=="120000"'` (mode 120000 = symlink) or `git ls-files -s | grep 120000`.
|
||||
- Fix: ignore rules WITHOUT trailing slash (`**/node_modules`, `**/.venv`) so symlinks match too.
|
||||
|
||||
## Consequence 2: same-name worktree rules silently matching worktree dirs
|
||||
|
||||
A root rule like `wt-*` (meant to ignore local worktree checkouts) matches every path whose last segment starts with `wt-` — so legitimate content under such a directory goes untracked. `git status --ignored` is the only visible trace; detection: `git check-ignore -v <path>` prints the owning rule + line number. (Repo-specific: `dank/command-center`'s `.gitignore` carries a "per-worktree memory buckets" block ignoring `agent/memories/sharpshooter/wt-*/` and `agent/memories/summaries/wt-*/` — intentional there.)
|
||||
|
||||
## Consequence 3: `.deploy-stale`
|
||||
|
||||
`deploy.sh` renames a stale tree to `<path>.deploy-stale` before recreating it. Without an ignore rule those directories land in `git status` as untracked and get swept into `git add -A` commits; they accumulate on every server.
|
||||
|
||||
## Fixes with tests
|
||||
|
||||
```gitignore
|
||||
**/node_modules
|
||||
**/.venv
|
||||
.deploy-stale/
|
||||
```
|
||||
|
||||
Black-box test against the real rules, not a fixture: `git check-ignore -v <symlink-target>` must name the rule. `git check-ignore` takes hypothetical paths — nothing needs to exist on disk:
|
||||
|
||||
```bash
|
||||
git ls-tree HEAD -- node_modules # mode 120000 == tracked symlink
|
||||
git ls-files -s | awk '$1==120000' # every tracked symlink in the index
|
||||
git check-ignore -v natives/x.wasm dotfiles/x/.config/noctalia/Noctalia.json
|
||||
```
|
||||
|
||||
## Fix (branch, never on main directly)
|
||||
Two ignore surfaces must be kept in sync (deploy exclusions vs ignore rules): if a deployer excludes `*/Noctalia*` but the ignore list has only the lowercase class, generated files become tracked-but-never-deployed — test both halves with `git check-ignore`.
|
||||
|
||||
```bash
|
||||
git rm -q --cached node_modules # untrack; leaves the file on disk
|
||||
```
|
||||
## Recovery (if contamination already pushed)
|
||||
|
||||
plus relax the ignore rule to the slash-free form, with the reason inline so nobody re-adds the slash:
|
||||
|
||||
```
|
||||
# Trailing slash would only match directories: a node_modules symlink (the
|
||||
# shape `git worktree` checkouts end up with) slips past it and gets tracked.
|
||||
node_modules
|
||||
```
|
||||
|
||||
Then `git check-ignore -v node_modules` must print the rule, and the suite runs after (the symlink still resolves, it is just ignored).
|
||||
|
||||
## Prevent
|
||||
|
||||
In a fresh worktree, create the deps symlink **after** confirming the ignore rule has no trailing slash:
|
||||
|
||||
```bash
|
||||
git check-ignore -v node_modules || sed -i 's|^node_modules/$|node_modules|' .gitignore
|
||||
```
|
||||
|
||||
Same class for any ignored-by-convention dir that a worktree might provide as a link: `node_modules`, `vendor`, `.venv`, `target`, `dist` when built artifacts are ignored.
|
||||
1. `git rm --cached <path>` on the symlink blob, commit, push.
|
||||
2. Fix the ignore rule first (else the next `git add -A` re-adds it).
|
||||
3. Untracked ≠ deleted from history: the blob stays in every old commit. Only a rewrite removes it; that is the operator's call, not a cleanup run alone.
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: halogen-npu-lane-boot-loop-triage
|
||||
description: "Triage a frank-dragoon (halogen-flash-server) unit crash-loop caused by the NPU lane: signature classification, restart-debris trap, XRT mount-path census, and the transient env-file proof boot."
|
||||
---
|
||||
|
||||
Triage when `frank-dragoon` (or any halogen-flash-server unit with `HALOGEN_NPU_MODELS` armed) sits in `activating (auto-restart)` with `ExecStart … code=exited, status=1/FAILURE` seconds after start, while `/health` stays dark.
|
||||
|
||||
## 1. Classify the exit BEFORE touching the config
|
||||
- `NRestarts` climbing + exit in <5 s = refusal, not an engine fault. Read `journalctl -u frank-dragoon --since "-30m"` AND the engine log `/home/dank/logs/frank_dragoon.log` (engine stdout never goes to the journal).
|
||||
- Lane signature: log ends with `halogen npu: … <name>: N files checked against this image's record` then `halogen-npu: No such library '<path>'` + `the NPU engine exited before it was ready (rc=1)` → **the NPU lane exit is fatal to the whole container** (supervisor kills the Flash engine with it). Chat-down is the lane's fault even though chat code never changed.
|
||||
- KV/pre-flight lines present but no `locked in RAM` line = died before weight pinning.
|
||||
|
||||
## 2. The restart-debris trap
|
||||
`Restart=always` re-runs the failing boot every ~10 s; each attempt leaves GPU/NPU handles being released, and the next boot logs `GTT in use before this start: N GiB … held by 1 process(es)`. That NOTE is debris from your own crash-loop — do not chase it as a foreign co-tenant (real co-tenants: ComfyUI `python` on `/dev/kfd`, find with `sudo fuser -v /dev/dri/renderD128 /dev/kfd`). `sudo systemctl stop <unit>` FIRST, then any diagnosis.
|
||||
|
||||
## 3. XRT path census (the `No such library '/opt/xilinx/lib/<multiarch>/libxrt_*.so.2'` case)
|
||||
- The image ships NO XRT at all (`find / -xdev -name "libxrt*"` inside the image = empty); the NPU child `dlopen`s the host's libs through the serve script's mounts, which land at TWO destinations only: `/opt/xilinx/xrt/lib/<L>` and `$XRT_DIR/<L>`. A child searching `/opt/xilinx/lib/<multiarch>/` names a THIRD prefix nobody mounts.
|
||||
- The serve loop resolves `REAL="$(readlink -f …)"` and mounts resolved files — if a host package upgrade retargets the symlinks (check `grep libxrt /var/log/dpkg.log` + `ldconfig -p | grep xrt` mtimes vs the last good boot), the file set the child expects can change under byte-identical mounts.
|
||||
- Rootless gotcha: `podman mount` fails outside `podman unshare` and leaves `$M` EMPTY — `grep -r pat "$M/opt"` then greps your CWD = HOST tree (false hits like `/opt/fastflowlm`). Always `podman unshare bash -c 'M=$(podman mount $C); …'` or `--entrypoint ls|cat` probes instead, and distrust hits that name host-only paths.
|
||||
|
||||
## 4. Proof boot without touching production
|
||||
The serve script honors `FRANK_DRAGOON_ENV_FILE` (sanctioned seam). Throwaway script on the host:
|
||||
1. scratch env: `source /home/dank/serverAppConfigs/scripts/frank-dragoon.env` + the knob under test.
|
||||
2. `sed` the serve script to inject the candidate extra mounts into the `NPU_ARGS+=(--device …` line — mount RESOLVED real files (`.so.2.25.00`), never symlinks (vendor: dangling links inside the container).
|
||||
3. `export FRANK_DRAGOON_ENV_FILE=/tmp/… FRANK_DRAGOON_LOG=/tmp/…; FRANK_DRAGOON_CONTAINER=<scratch-name> timeout 1200 /tmp/serve-test.sh` — the container still publishes 7770, so `systemctl stop` the unit first (port binds at CREATE, exit 126 on race).
|
||||
4. Retire same session: `podman rm -f <scratch-name>`, trash `/tmp` scripts; the unit stays stopped until the real fix deploys.
|
||||
|
||||
## 5. Restore chat while the lane is unfixed
|
||||
`HALOGEN_NPU_MODELS=` (blank) is the documented one-line lane-off switch — lane absent = endpoint exactly as before the lane existed. Land through git + `deploy.sh dankclaw --service frank-dragoon`; never hand-edit the remote env.
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
name: halogen-npu-mount-prefix-proof
|
||||
description: "Prove and fix a halogen frank-dragoon NPU-lane mount failure: crash-loop where a dead lane kills the whole container, dlopen-prefix census, and the sanctioned FRANK_DRAGOON_ENV_FILE proof-boot."
|
||||
---
|
||||
|
||||
# Halogen NPU lane: dead lane kills the container — prove the mount prefix, don't guess
|
||||
|
||||
Applies when a `frank-dragoon` unit crash-loops in seconds (`exit 1` in ~200 ms,
|
||||
`NRestarts` climbing) and the engine log's last lines are:
|
||||
|
||||
```
|
||||
halogen npu: <model>: 15 files checked against this image's record
|
||||
halogen-npu: No such library '<some/prefix>/libxrt_core.so.2'
|
||||
halogen npu: the NPU engine exited before it was ready (rc=1)
|
||||
```
|
||||
|
||||
## Laws this incident wrote (2026-10-21, frank-dragoon, 88-restart loop)
|
||||
- **A dead NPU lane is fatal to the whole container**: the flash-server supervisor
|
||||
exits with the lane. Chat dies beside a healthy GPU preflight — do not read the
|
||||
GTT/weights lines as "engine fine, side feature broken".
|
||||
- The NPU child `dlopen`s distro XRT from **three** container prefixes per library:
|
||||
`/opt/xilinx/xrt/lib/`, the library's own install path, **and
|
||||
`/opt/xilinx/lib/<multiarch>/`**. The missing one is exactly what the error names.
|
||||
- Host state can flip this under a byte-identical container: the lane ran for days
|
||||
with two prefixes, then a host `libxrt2`/`libxrt-npu2` reconfigure (check
|
||||
`grep libxrt /var/log/dpkg.log`) made the child demand the third. Image digest,
|
||||
serve script, payload all unchanged ⇒ suspect HOST package state, not the deploy.
|
||||
- Fastest discriminator vs engine faults: exit in <1 s with the engine's preflight
|
||||
lines present = serve/lane gate failure, not model load. Model-load failures take
|
||||
minutes and print pin/KV lines.
|
||||
|
||||
## Proof procedure (sanctioned seam, no scratch ports)
|
||||
1. Read the error's path verbatim — it names the missing prefix.
|
||||
2. `sudo systemctl stop frank-dragoon` (unit must be down: the proof binds :7770
|
||||
at container CREATE).
|
||||
3. Scratch env file on the host using the bench-sanctioned seam:
|
||||
```
|
||||
/tmp/dragoon-nputest.env:
|
||||
source /home/dank/serverAppConfigs/scripts/frank-dragoon.env
|
||||
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
|
||||
```
|
||||
4. Scratch serve copy that appends the candidate mount to the `NPU_ARGS+=(--device`
|
||||
line (`sed`), REAL files only (`readlink -f`; vendor forbids dangling symlinks
|
||||
under /opt/xilinx), and run with:
|
||||
`FRANK_DRAGOON_ENV_FILE=/tmp/... FRANK_DRAGOON_LOG=/tmp/... timeout 1200 /tmp/serve-nputest.sh`
|
||||
5. Success = boot ledger (`… in all`) **plus** `npu_api: <model> … buckets u0…u9`
|
||||
plus `/health` `npu.status ok`. Then codify the mount in the serve script, pin
|
||||
it in `tests/frank_dragoon_unit.bats` (the fixture XRT dir's basename IS the
|
||||
multiarch name — name it `x86_64-linux-gnu`), update the endpoint page's NPU
|
||||
prerequisite to the three-prefix law, and cut over via PR +
|
||||
`deploy.sh dankclaw --service frank-dragoon`.
|
||||
6. Retire scratch: the `timeout` kills the proof container; `podman rm -f`,
|
||||
move /tmp scripts to `~/.AGENTTRASH/<date>-<topic>/`. A proof instance
|
||||
serving production while you wait is drift — deploy the merged fix promptly.
|
||||
|
||||
## Traps
|
||||
- Greping image contents rootless: `podman mount` fails (needs `podman unshare`)
|
||||
and leaves `$M` EMPTY, so `grep -r $M/opt` silently greps the HOST cwd — that
|
||||
is how host `/opt/fastflowlm` masqueraded as image content. Always check `$M`
|
||||
non-empty, or use `podman unshare bash -c 'M=$(podman mount C); …'`.
|
||||
- The entrypoint swallows `podman run IMG sh -c …` as a mode name; use
|
||||
`--entrypoint sh/cat/ls`.
|
||||
- The proof container inherits production name/port unless the override is set
|
||||
AFTER sourcing the env (the env file re-exports `FRANK_DRAGOON_CONTAINER`).
|
||||
- `Restart=always` re-runs the lane into its own GPU/NPU debris; stop the unit
|
||||
before any manual boot, don't fight the loop.
|
||||
@@ -1,60 +1,62 @@
|
||||
---
|
||||
name: harden-precommit-secret-gate
|
||||
description: "Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer."
|
||||
description: "Fix or build a pre-commit secret-scan hook that actually blocks: the -z path hole, line-wide placeholder exemptions, inert doc placeholders, glued report lines, self-test fixture paths — each with a red-first bats case."
|
||||
---
|
||||
|
||||
Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer. Use when a secret gate is wrong (misses a leak) or too right (rejects an inert example and bricks the heartbeat).
|
||||
A pre-commit secret gate that fails open is worse than no gate: it teaches the operator that scanning happened. Four defect classes, each with a black-box test that goes red against the broken version (serverAppConfigs PR #70, command-center #59 — 4 holes closed same night, all red first).
|
||||
|
||||
## The two failure directions
|
||||
## 1. Non-ASCII paths fail open
|
||||
|
||||
A gate can be **too weak** (misses a real key) or **too strong** (rejects its own documentation and takes down an automated committer). Both are outages; the second is silent.
|
||||
|
||||
## Audit recipe for an existing gate
|
||||
|
||||
1. Read the hook and list what it actually inspects. The classic scope hole: `git diff --cached -- '*.env' '*.pem'` — an extension whitelist. A credential in `agent/config.yml`, a quadlet unit, or a SKILL.md is then invisible to the gate. The rule that survives: scan **the entire staged diff**, every path, and let only binary content out.
|
||||
2. Test fail-closed behaviour: can the hook be made to print nothing and still exit 0? Common escapes — running outside a git repo, unreadable staged diff, a hook that pipes through `grep` and inherits `grep`'s exit code from the wrong stage, or `set -e` swallowing the verdict. Verdict must be: unknown ⇒ **block**.
|
||||
3. Quarantine list, never tracked: `.env`, `auth.json`, `trust.json`, `*.pem`, `*.key`, `identity*` — and the `.gitignore` policy must quarantine env files **at any depth under any name** (`**/.env*`, not `.env`), or `config/prod.env` slips past.
|
||||
4. Escape hatch must be explicit, per-commit, and loud: `ALLOW_SECRETS=1 git commit …`, and the hook says so when it honours it.
|
||||
5. Gate the gate: a test that commits a fixture repo through the hook and asserts it blocks. `tests/secret_gate.bats` here covers: shape in an unexpected extension, bcrypt hash, JWT, placeholder/indirection allowed, prose allowed, binary ignored, unreadable diff refused, deletion of a credential never blocked, `ALLOW_SECRETS` override.
|
||||
|
||||
## Self-trip: the gate rejects its own example (live 2026-10-21)
|
||||
|
||||
Symptom: a systemd timer that commits machine-written files fails **every hour**, and the journal shows the gate's own refusal:
|
||||
|
||||
```
|
||||
SECRET GATE: staged diff contains 1 secret-shaped string(s):
|
||||
agent/managed-skills/git-secret-gate-hook/SKILL.md
|
||||
Fake secrets must be inert (RFC 2606 domains, all-zero ids, `AKIA0000000000000EXAMPLE`) AND must not
|
||||
```
|
||||
|
||||
The offender is the AWS **documentation** example key — entirely inert — and `AKIA[0-9A-Z]{16}` cannot tell it from a leak. So the gate's own advice about writing an inert fake is uncommittable by the gate. Auto-learned skills describe credential shapes for a living, so this recurs by construction, and an automated committer cannot use `ALLOW_SECRETS=1`: that would blind the gate for the *other* 64 staged files.
|
||||
|
||||
Diagnose, in this order:
|
||||
`git diff --cached --name-only` quotes non-ASCII paths (`"docs/\303\251.md"`), so `git diff` on that name returns **empty**, and a loop that `continue`s on an empty diff skips the file silently. Fail closed with NUL delimiting:
|
||||
|
||||
```bash
|
||||
journalctl --user -u live-sync.service -b --no-pager | tail -25 # which file, which line
|
||||
git -C ~/.omp status --short | head # how much is queued behind it
|
||||
# then scan the WHOLE staged queue, not just the reported file:
|
||||
git diff --cached -U0 | awk '/^diff --git/{f=$3;sub("^a/","",f)} /^\+/ && !/^\+\+\+/ {
|
||||
l=substr($0,2)
|
||||
if (l ~ /sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|glpat-[A-Za-z0-9_-]{16,}|AKIA[0-9A-Z]{16}|xox[baprs]-[A-Za-z0-9-]{10,}|eyJ[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{10,}\.|-----BEGIN [A-Z ]*PRIVATE KEY-----/)
|
||||
print f": "substr(l,1,90) }'
|
||||
git diff --cached -z --name-only | while IFS= read -r -d '' f; do ...
|
||||
```
|
||||
|
||||
The second count is the decision input: one hit ⇒ fix the line; many ⇒ the gate needs an exemption channel before anything lands.
|
||||
Test: a tracked file whose name contains a non-ASCII character, holding a fake key on an added line, must be blocked.
|
||||
|
||||
Fixes, in order of preference:
|
||||
## 2. Placeholder filters must be value-scoped
|
||||
|
||||
1. **Describe the shape instead of printing a matching literal** — `AKIA` followed by zeroes and `EXAMPLE`, spelled out in prose, cannot match. One line, the gate stays strict, the next timer run drains the queue.
|
||||
2. **Per-line allow marker** honoured by the hook (e.g. a trailing `secret-gate: allow` on the flagged line) — durable, keeps documentation honest, still fail-closed everywhere else. Needs a test both ways: a marked line passes, an unmarked one still blocks. Note the risk honestly: an attacker-adjacent string can self-authorise, so the marker belongs in review-able prose tiers, not in a diff that also touches secrets config.
|
||||
3. Never widen the regex to skip markdown, never run the automation with the override env var set.
|
||||
`if echo "$line" | grep -q "$placeholders"` exempts the whole line, so a comment word vouches for the credential beside it:
|
||||
|
||||
Inert-fake rule for anything that must *show* a credential shape: RFC 2606 domains, all-zero ids, the word `EXAMPLE` — and if the shape itself trips the gate, describe it in prose rather than print it.
|
||||
```python
|
||||
token = "sk-…" # example config for the docs
|
||||
```
|
||||
|
||||
## Prose must not be collaterally blocked
|
||||
Filter only the captured value (the `sk-…` group), never the line. Two directions to test: `key = "example"` stays allowed; `token = "sk-<real>" # example` is blocked.
|
||||
|
||||
Markdown explaining credential handling is the bulk of what a knowledge repo commits. `xox` inside a sentence about Slack token prefixes, or a `sk-` prefix discussed abstractly, must pass; the shapes are anchored long-suffix patterns for exactly that reason. When a gate fires on prose, that is a bug in the gate — but if the prose contains a *literal* matching token, the fix is the prose (option 1 above), not the pattern.
|
||||
## 3. Inert placeholders in documentation
|
||||
|
||||
## Verifying a gate change
|
||||
A skill/README that *documents* a secret gate quotes the vendor's placeholder ids (`AKIA0000000000000EXAMPLE`, all-zero JWTs). Exempt them by value shape (all-zero / `EXAMPLE` suffix / RFC 2606 domain), or the machine tier stops committing — and the refusal usually lands on a heartbeat commit, where it looks like infrastructure failure, not policy. Also note: an unlanded fix means the checkout runs the **old** hook, so a landing can be blocked by the very bug the PR corrects; audit the staged diff and use the documented override (`ALLOW_SECRETS=1`, never `--no-verify`), recording the audit in the commit message.
|
||||
|
||||
Write the fixture repo in a temp dir, commit through the hook, assert block/allow for each case. Do not test the regex by grepping the hook's source for the pattern — a copy of the pattern in a comment is not a gate that runs.
|
||||
## 4. Report-line assembly
|
||||
|
||||
`report+=(" $file: $line")` inside a loop that keeps only the last hit per file glues the next filename onto the previous line when a second hit exists — the report attributes the finding to the wrong file. Test a two-file staged set where both match, and assert each report line names exactly one existing path.
|
||||
|
||||
## 5. Self-test fixtures in the hook itself
|
||||
|
||||
The hook's own fixtures contain fake keys by necessity, so the gate would refuse any commit touching the hook or its tests (or a docs page quoting a key as documentation). Pin the bypass to specific paths and to added lines only, and print what was exempted:
|
||||
|
||||
```bash
|
||||
case "$f" in .githooks/pre-commit|tests/secret-gate.bats)
|
||||
echo "secret-gate: self-test fixtures exempt: $f" >&2; continue;; esac
|
||||
```
|
||||
|
||||
## The scan rule
|
||||
|
||||
Scan **added content only** — `git diff --cached -U0 | grep '^+[^+]'` — or every commit touching a file with an old key in context gets blocked, and the operator's response to a gate that blocks everything is to override it always.
|
||||
|
||||
## Test harness (fixture repo, never the real checkout)
|
||||
|
||||
`git init` a temp repo, `git config core.hooksPath .githooks`, copy the hook in, `git add` a payload, then run it with the env overrides the hook honours (point its cache/notify paths at `$TMP` so tests never touch the user's config). Pin, in order:
|
||||
|
||||
| Case | Expect |
|
||||
|---|---|
|
||||
| fake key on an added line | exit 1, message names the file |
|
||||
| `key = "example"` | pass |
|
||||
| real key + `# example` comment | exit 1 |
|
||||
| non-ASCII filename with a key | exit 1 |
|
||||
| two files with hits | report names both |
|
||||
| self-test fixtures | pass, with the exempt line printed |
|
||||
|
||||
`bats -r` after each edit. `--no-verify` is forbidden on Frank's boxes: the hook IS the control.
|
||||
|
||||
@@ -1,48 +1,49 @@
|
||||
---
|
||||
name: html-report-ground-truth-audit
|
||||
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix, and prove structure + render"
|
||||
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix wrong values, delete filler that restates git, verify the served artifact byte-identical, and flag unverifiable content to the operator."
|
||||
---
|
||||
|
||||
Use after generating a large HTML report (e.g. via `repo-change-html-report`) whose claims were partly reconstructed from session memory. Fabricated numbers, invented code blocks, and silently-swallowed sections are the failure modes; the audit makes each traceable to the repo, a live command, or a recorded run.
|
||||
Ground-truth audit of an HTML report (generated by an agent, for a human). Three defects hide in a plausible report: **wrong numbers, unverifiable claims, and filler that restates git for a reader who has git.**
|
||||
|
||||
## Law
|
||||
|
||||
Every numeric claim must be re-measured against the repo, not the narrative that produced it.
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Enumerate every checkable claim
|
||||
Grep the report for numerics, file paths, symbol names, and code blocks:
|
||||
- line counts (`wc -l` the real file)
|
||||
- test counts (`grep -c '@test' tests/<f>.bats` — NOT suite-run totals: conditional skips make raw counts differ; run-observed totals are legitimate run history, keep them labeled as such)
|
||||
- diff stats (`git diff --shortstat main..HEAD` — the footer's files/insertions/deletions)
|
||||
- commit hashes (`git log --oneline`)
|
||||
- "verbatim" code blocks (see 3)
|
||||
- diagram node lists and captions (see 4)
|
||||
1. **Dump claims with file:line anchors.**
|
||||
`sed 's/<[^>]*>//g' report.html | grep -vE '^\s*$' | grep -nE 'commit|file|line|fix|refactor' > /tmp/claims.txt`
|
||||
Then enumerate the ground truth: `git log --oneline origin/main..HEAD` for commits; `git diff --stat origin/main..HEAD` for the per-file census — **sort it**: `--sort=-deletions` and `--sort=-creations` give the top movers, so the "largest single change" and "largest net code change" are computed facts.
|
||||
|
||||
### 2. Ground numeric claims with one command each
|
||||
Fix every mismatch in place. Record the was/now pair — the audit table IS the evidence.
|
||||
2. **Check code blocks compile.** Write every snippet to a temp file and `bash -n` each. Common breakages from HTML generation: `"`/`&` entities leaking into code, and `<br>` line breaks that split a line mid-pipe.
|
||||
|
||||
### 3. Verbatim code blocks: prove them
|
||||
For a block labeled verbatim, the exact string must exist:
|
||||
- live code: `grep -n '<distinctive line>' <file>`
|
||||
- historical: `git show <sha> -- <file> | grep -E '^[+-].*(<token1>|<token2>)'`
|
||||
If the block was fabricated (plausible-looking pseudo-code), replace it with the real excerpt. If condensed, the block header must say "condensed" and every shown line must be real.
|
||||
3. **Check the diagrams carry information.** A mermaid `flowchart LR` whose nodes merely restate the bullet list above it is decoration — delete it.
|
||||
|
||||
### 4. Diagram claims: run the engine
|
||||
Behavioral diagrams (plan traces, closure graphs) must be re-derived by running the actual CLI:
|
||||
`python3 scripts/service-map.py plan --host <h> <path>` — then edit node labels, captions, and the prose count ("four services") to match output. Cross-check derived-set diagrams against the source relation (e.g. `grep -rl 'Network=gluetun.container' */**/*.container` for who rides a tunnel).
|
||||
4. **Check counts.** Commit and file counts must match `git rev-list --count` and `--name-only | sort -u | wc -l`. A report whose sections were written by different subagents drifts: each counts its own slice ("4 commits / 14 files" vs the true 11 / 25).
|
||||
|
||||
### 5. Check for swallowed sections
|
||||
Large prior-session PUT ranges silently overwrite neighbors. Grep for headings/captions the plan promised; re-insert any that vanished, anchored on a fresh `read` of the region.
|
||||
5. **Flag what cannot be verified.** Test pass claims, external tool output, runtime behaviour — if the repo can't confirm it, say so to the operator instead of asserting it.
|
||||
|
||||
### 6. Structural balance
|
||||
Python tag-balance pass over the file:
|
||||
- per-tag open/close counts (`div`, `pre`, `figure`, `svg`, `table`, `span`, `figcaption`)
|
||||
- running-div-depth scan printing per-section balance to localize stray closers
|
||||
Common defect: `.ln`/`.nt` pair emitted without its `.cg` wrapper → stray `</div>`; fix by opening `<div class="cg">` before the pair.
|
||||
## Filler test (delete on failure)
|
||||
|
||||
### 7. Render proof
|
||||
`file://` is blocked on the shared browser — serve the dir: `bash` service `python3 -m http.server <port> --bind 127.0.0.1 --directory <dir>`, ready on that port. Then:
|
||||
- `browser.open({name, url: 'http://127.0.0.1:<port>/<file>'})`
|
||||
- DOM assertions via `tab.evaluate(() => …)` — NOT `tab.run` (that runs in Bun; no `document`). Sections are `<h2 id>` + siblings: scope queries to `document.body`/`document.querySelectorAll`, not `#ch5 …`.
|
||||
- screenshot the repaired regions (scroll via `tab.evaluate(el.scrollIntoView())`), then `tab.close()` and kill the server proc (`write proc://<name>/kill`).
|
||||
- A `file → line-count → description` table restating `git diff --stat`? Delete — the reader has git.
|
||||
- A "what this means" / "conclusion" / "key takeaways" section? Delete — it is the same facts in softer prose. (Same rule as vault pages: point of use or gone.)
|
||||
- Keep only what a reader cannot rebuild from `git log` + `git show`.
|
||||
|
||||
## Deliver
|
||||
A was/now table of every fix with its ground command, plus the list of claims verified intact. State honestly which numbers are run-history and not recomputable.
|
||||
## Serve and verify the artifact
|
||||
|
||||
**Verify the file that is served, not the source.** If a `public/` copy is byte-identical, edit both or the served artifact stays wrong. Then re-fetch:
|
||||
|
||||
```bash
|
||||
base=http://host:port/reports/x.html
|
||||
for i in $(seq 1 30); do curl -fsS -o /tmp/got.html "$base" && break; sleep 1; done # service may still be activating
|
||||
for i in 1 2; do curl -fsS "$base" | md5sum; done # stable across refetches
|
||||
diff <(curl -fsS "$base") /path/to/source.html && echo "served == source"
|
||||
```
|
||||
|
||||
## Voice drift
|
||||
|
||||
If the operator asked for a tone ("no filler", "terse"), the audit must enforce it too — marketing phrasing ("ships the", "This means:") is a style defect, fix it in the same pass, not a later one.
|
||||
|
||||
## Report
|
||||
|
||||
Fixes grouped by defect class (wrong number / unverifiable / filler / style), then what stays unverifiable and why, with the measured ground truth quoted.
|
||||
|
||||
@@ -1,50 +1,82 @@
|
||||
---
|
||||
name: land-merged-pr-with-blocked-tree-ops
|
||||
description: "Use when a merged PR must land in a checkout whose working tree is dirty and workspace policy blocks checkout/stash — prove the delta content-free, hand the operator the two-liner, verify via live config meanwhile."
|
||||
description: "Use when a merged PR must land in a checkout whose working tree is dirty and workspace policy blocks git checkout/restore/stash: commit-the-dirt pattern, the secret-gate false positive that blocks the landing commit, conflict resolution proven settings-vs-comments, and heartbeat (live-sync) handoff."
|
||||
---
|
||||
|
||||
Landing a merged PR when the deployment checkout is dirty **and** workspace policy blocks `git checkout`/`git stash`. Use when `git merge --ff-only origin/main` aborts with "Please commit your changes or stash them before you merge" in a SHARED workspace where other agents have work in flight. Earned 2026-10-20 on `dank/command-center` (the checkout IS the deployment).
|
||||
Landing a merged trunk into a checkout that IS the deployment (dank/command-center `~/.omp`), when tracked files are dirty and policy forbids `checkout`/`restore`/`stash` on shared state. Re-earned 2026-10-21 landing PRs #61 + #63 into `~/.omp` (27 commits behind, two conflicts, one gate refusal).
|
||||
|
||||
# Land a merged PR with blocked tree ops
|
||||
## 1. Classify the dirt before touching it
|
||||
|
||||
## Why it blocks
|
||||
|
||||
`--ff-only` refuses when **any** incoming path is dirty in the worktree — it checks dirtiness against the *old* HEAD, not whether your content matches the target. Proven: overwriting the file so it is **byte-identical to the target commit** (`cmp` clean against `git show origin/main:<path>`) still aborts with the same message. So "make the file match main" is not a way in, and neither is `git merge` from another worktree (the branch is checked out in the dirty one).
|
||||
|
||||
The blocking set is what you must know, exactly:
|
||||
`git status --porcelain` then compare against what the merge brings in — only overlap blocks the merge:
|
||||
|
||||
```bash
|
||||
comm -12 <(git diff --name-only HEAD..origin/main | LC_ALL=C sort -u) \
|
||||
<(git status --porcelain | awk '{print $NF}' | LC_ALL=C sort -u)
|
||||
{ git diff --name-only; git diff --cached --name-only; } | sort -u > /tmp/dirty.txt
|
||||
git diff --name-only HEAD origin/main | sort -u > /tmp/incoming.txt
|
||||
comm -12 /tmp/dirty.txt /tmp/incoming.txt # these paths must be clean
|
||||
```
|
||||
|
||||
`LC_ALL=C sort` matters — git's own output order trips `comm: file N is not in sorted order` and you get garbage overlap lists. In the case this was earned on, 12 incoming paths and 57 dirty ones intersected in exactly **one** file.
|
||||
Note the first column of porcelain: `M ` = **staged**, ` M` = unstaged. A staged file shows nothing in `git diff` — look at `--cached` too, or you will "verify" a file is unchanged while it is staged. `comm` needs both inputs sorted; unsorted input prints "file is not in sorted order" and a partial answer.
|
||||
|
||||
## Procedure
|
||||
## 2. Prove a dirty file content-free before resolving
|
||||
|
||||
1. **Fetch, don't merge yet.** List incoming commits (`git log --oneline HEAD..origin/main`) so you can show what is waiting.
|
||||
2. **Compute the blocking set** with the `comm` above (both inputs `LC_ALL=C sort -u`).
|
||||
3. **Prove the local delta is content-free**, in the direction that matters — lines the working tree has that `main` does not:
|
||||
```bash
|
||||
diff <(git show origin/main:<path>) <path> | grep -c '^>' # local-unique lines
|
||||
```
|
||||
Zero means the working tree is a strict subset of what was merged: discarding loses nothing.
|
||||
4. **Preserve before you touch:** copy the current file to `~/.AGENTTRASH/` (snapshot), and copy anything you had moved into a gitignored `.AGENTTRASH/` inside a build worktree to its permanent home in the deployment — otherwise pruning the worktree destroys the only copy of a quarantined page.
|
||||
5. **Do NOT work around the policy.** `git checkout -- <path>`, `git stash`, and `git stash push -- <path>` are all blocked in a shared workspace and the block is correct: it protects other agents' in-flight work. Do not reach for `git reset --soft` + `git restore --staged` to slide the pointer past it either — that leaves the index holding staged *reverts* of every incoming path, one careless `git commit -a` away from undoing the merge.
|
||||
6. **Hand the operator the two-liner** and keep working on what does not depend on it:
|
||||
```bash
|
||||
git -C ~/.omp checkout -- <path> && git -C ~/.omp merge --ff-only origin/main
|
||||
```
|
||||
Say why it is safe (step 3's number, plus where the snapshots live) and what it unblocks.
|
||||
`cmp` it against the trunk blob, never against memory:
|
||||
|
||||
## Why waiting is not free
|
||||
```bash
|
||||
git show origin/main:$f > /tmp/incoming
|
||||
cmp -s "$f" /tmp/incoming && echo "IDENTICAL to trunk" # commit/discard is content-free
|
||||
```
|
||||
|
||||
`scripts/live-sync.sh` merges with a plain `git merge --no-edit origin/main` (its line ~88), so the hourly heartbeat hits the identical wall and raises its ntfy alert `merge conflict with origin/main — local commit kept; needs you at the keyboard`. Report that consequence: the dirt is not merely untidy, it alerts.
|
||||
If identical, the dirt is a sync artifact, not user work. Say so in the commit message with the `cmp` result — that is what makes a direct commit on `main` defensible for a prose-tier file.
|
||||
|
||||
Also note for later: `git branch -d <merged-branch>` **refuses until the commits reach `main`** — `-d` checks against the current HEAD, not `origin/main` — so local branch deletion goes on the post-landing list, not the pruning list.
|
||||
## 3. Commit rather than discard (policy-blocked verbs)
|
||||
|
||||
## What you CAN do while it waits
|
||||
With `checkout`/`restore`/`stash` unavailable, the only non-destructive unblock is a commit. Two separate commits, and order matters:
|
||||
|
||||
- **Verify the merged behaviour through the live config, not the branch pointer.** Merged prompt/config changes take effect from the deployment's files: spawn a fresh session/subagent and have it read and quote the changed file back. That proves the change is live even while `main` is behind.
|
||||
- Prune your own build worktrees, but check nothing is running inside them first: `ls -l /proc/*/cwd 2>/dev/null | grep -c "wt/<name>"` (the same guard `live-sync.sh` uses).
|
||||
- Delete the merged remote branches (`git push origin --delete <branch>`, allowed on the local Gitea).
|
||||
1. Prose-tier dirt proven byte-identical to trunk — commit alone, message states `cmp` proved no prose change.
|
||||
2. Machine tiers (`agent/memories/**`, `agent/managed-skills/**`, `agent/config.yml`, `agent/models.yml`, `dotfiles/`) — let the heartbeat do it: `LC_ALL=C scripts/live-sync.sh` stages allowlist only, commits, merges trunk, pushes, and **exits 0 even when it aborts the merge on conflict** (the alert goes to stderr + ntfy). `exit 0` does not mean landed — check `behind=` afterwards.
|
||||
|
||||
Do NOT `git add -A`: the index may already hold dozens of staged machine-tier files, and `git add <one file>` does not clear them — they ride along into your commit.
|
||||
|
||||
## 4. The gate refusal that blocks the landing commit
|
||||
|
||||
The pre-commit secret gate scans the **staged** diff, so a single-file commit still trips on previously staged content. It fires on inert placeholders inside the skill docs that document the placeholder (e.g. `AKIA0000000000000EXAMPLE` — the AWS all-zero example id in `git-secret-gate-hook` / `harden-precommit-secret-gate`). If the checkout is behind, it is running the **pre-fix** hook while the fix sits unlanded in trunk: the landing is blocked by the very bug the PR corrects.
|
||||
|
||||
Audit first, then override deliberately — never `--no-verify`:
|
||||
|
||||
```bash
|
||||
git diff --cached | grep -nE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|glpat-[A-Za-z0-9]{16,}|BEGIN [A-Z ]*PRIVATE KEY|eyJ[A-Za-z0-9_-]{20,}'
|
||||
git diff --cached | grep -oE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}' | sort | uniq -c # what exactly, how many
|
||||
ALLOW_SECRETS=1 git commit -F msg # message records the audit + that the gate ran
|
||||
```
|
||||
|
||||
Record in the message: the hits are inert placeholders, which files they sit in, that this checkout runs the pre-fix hook, and that the refusal dies with this landing.
|
||||
|
||||
## 5. Merge conflicts: resolve by proving settings identical
|
||||
|
||||
```bash
|
||||
git merge --no-commit --no-ff origin/main
|
||||
git diff --name-only --diff-filter=U
|
||||
```
|
||||
|
||||
For each conflicted file, get all three blobs (`HEAD`, merge base, `origin/main`) and ask what the local side uniquely holds:
|
||||
|
||||
- **Prose / doc pages**: diff ours vs trunk; if ours only lacks the incoming trim, take trunk.
|
||||
- **Real settings files (kdl/yaml/env)**: strip comments and diff — `diff <(grep -v '^\s*//' ours) <(grep -v '^\s*//' theirs)`. Empty output ⇒ only comments differ ⇒ taking trunk changes no deployed behaviour, and you can say that in the merge message. (2026-10-21: `desk-1-home.kdl` — modes byte-identical, trunk's comment corrected the false "no 240 Hz mode exists".)
|
||||
|
||||
Resolve without the blocked verbs by writing the trunk blob: `git show origin/main:$f > $f && git add $f`. Then validate the live artifact (`niri validate`, the suite) before committing the merge, and push.
|
||||
|
||||
## 6. Verify the push actually landed
|
||||
|
||||
`git push` output can be swallowed when wrapped in command substitution inside a batch — verify from the remote, not from the push:
|
||||
|
||||
```bash
|
||||
git fetch -q origin && printf 'HEAD=%s origin/main=%s behind=%s\n' \
|
||||
"$(git rev-parse --short HEAD)" "$(git rev-parse --short origin/main)" "$(git rev-list --count HEAD..origin/main)"
|
||||
```
|
||||
|
||||
## 7. Close and prune
|
||||
|
||||
- Close the issues with a comment giving merge SHA + PR link + title (numbers are per-repo; a bare `#N` is not a reference).
|
||||
- Prune guards, all three before `git worktree remove --force`: `ls -l /proc/*/cwd | grep -c '<wt-name>'` must be 0; `git -C <wt> status --porcelain | wc -l` must be 0; `git rev-list --count main..<branch>` must be 0. Check for `.AGENTTRASH/**` inside the worktree first — removal destroys the only copy.
|
||||
- `git branch -d` (not `-D`) once the branch is reached by main; `git push origin --delete <branch>` is allowed on the local Gitea.
|
||||
- Delete the temp `-K` token config file in the same session that created it.
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
name: okf-small-bundle-docs-audit
|
||||
description: "Audit + dedupe a small repo's docs bundle for OKF conformance when the kb-lint/kb-coverage toolchain does not exist: inline conformance and link-audit scripts, owner-per-fact cuts, index trio, one log."
|
||||
---
|
||||
|
||||
# OKF docs audit for a small repo bundle (no kb-lint toolchain)
|
||||
|
||||
Use for a repo whose KB is one docs directory (e.g. `dankvoice` → `dankvoice-docs/`), where `scripts/kb-lint.sh` / `kb-coverage.py` do not exist. For the big `dank/command-center` campaign see skill `docs-dedupe-verify-campaign`; same frame, no scouts — a corpus under ~1500 lines gets read fully yourself (scouts lie).
|
||||
|
||||
## Frame
|
||||
- OKF spec lives at `~/.omp/omp-docs/references/okf-spec.md` (the command-center deploy checkout). v0.1 conformance = every non-reserved `.md` has parseable frontmatter with non-empty `type`; `index.md` = listing with NO frontmatter; `log.md` reserved, exactly one, bundle root.
|
||||
- Branch off `origin/main` in a fresh worktree; docs-only + src-comment changes still need the repo's normal gate.
|
||||
|
||||
## Procedure
|
||||
1. Inventory `wc -l` every page; read all fully. Ledger per fact: which page OWNS it (laws page owns laws, decisions own rationale, contributing owns process, README owns user-facing, GLOSSARY owns terms). Everything else becomes a pointer.
|
||||
2. Hunt the rot classes that repeat: dead test citations (name in docs not in `grep -c tests/` — verify with `git log --all -S`), hand-maintained censuses (test counts, scenario lists) — replace with a pointer to the authoritative source, never refresh the copy; deleted doc lines for live env vars (compare `git show <fixed>:file | grep -c VAR` vs HEAD); sections contradicting the page's own head; open items already resolved in live state (check the live settings file, not the doc).
|
||||
3. Frontmatter: `type` + one-line `description` + `tags` + `timestamp`. On edit-rejection the tool prints the file's current `#TAG` — reissue with it, no re-read needed.
|
||||
4. Conformance check (inline python): walk bundle dirs, regex `^---\n(.*?)\n---` then `^type: \S` on each non-reserved `.md`.
|
||||
5. Link audit (inline python): per md file, `\]\(([^)#]+)`; skip http; `os.path.normpath(join(dirname, target))` must exist. Report `N checked / ALL RESOLVE`.
|
||||
6. Create `index.md` at bundle root + each subdir (listing, no frontmatter, relative links) and `log.md` at the bundle root seeded with laws/incidents only.
|
||||
|
||||
## Gotchas
|
||||
- Src docstring fixes are artifact-free only if the bundler strips comments — prove with `bun run build; cmp dist/... old` before claiming no rebuild needed.
|
||||
- README must NOT enumerate design records; point at `decisions/index.md` so new records need no README edit.
|
||||
- Mark superseded decision records in their frontmatter `description` ("its X design is superseded by the Y record") — never rewrite history bodies.
|
||||
@@ -1,4 +1,4 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792557324990
|
||||
"lastConsolidatedAt": 1792574106585
|
||||
}
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792572225299,
|
||||
"lastConsolidatedAt": 1792575105798,
|
||||
"lastResult": {
|
||||
"at": 1792557082020,
|
||||
"at": 1792573818108,
|
||||
"sessions": 1,
|
||||
"deltas": 1,
|
||||
"deltas": 2,
|
||||
"model": "frank-zealot"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
## Okf Format
|
||||
|
||||
- "okf format"
|
||||
|
||||
## Human Readability
|
||||
|
||||
- "human readability"
|
||||
@@ -1,10 +1,10 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792572021650,
|
||||
"lastConsolidatedAt": 1792574961654,
|
||||
"lastResult": {
|
||||
"at": 1792571693973,
|
||||
"at": 1792574016725,
|
||||
"sessions": 1,
|
||||
"deltas": 1,
|
||||
"deltas": 2,
|
||||
"model": "frank-zealot"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
## Formatting
|
||||
- okf format
|
||||
- formatted for human readability
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792572038693,
|
||||
"lastConsolidatedAt": 1792575038705,
|
||||
"lastResult": {
|
||||
"at": 1792556461435,
|
||||
"sessions": 1,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792572210019,
|
||||
"lastConsolidatedAt": 1792574970034,
|
||||
"lastResult": {
|
||||
"at": 1792566095222,
|
||||
"sessions": 1,
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792574109325
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792575103668
|
||||
}
|
||||
+11
-10
@@ -96,19 +96,20 @@ providers:
|
||||
# halogen flash-server on the Strix Halo iGPU. Payload, shape, eras,
|
||||
# laws, rollback: serverAppConfigs-docs/LLM/frank-dragoon.md — and
|
||||
# scripts/frank-dragoon.env is the authority for every number there.
|
||||
# contextWindow sits deliberately BELOW the engine ceiling
|
||||
# (HALOGEN_CTX): the clamp below leaves the gap that absorbs the
|
||||
# harness's chars/4 under-count, so a window equal to the ceiling
|
||||
# would make every long request a guaranteed 400.
|
||||
contextWindow: 155648
|
||||
# contextWindow: 200,000 under the native 262,144 engine ceiling
|
||||
# (HALOGEN_CTX, 2026-10-21) — the 400-zone law: the window MUST sit
|
||||
# ~45k under the ceiling, never at it.
|
||||
contextWindow: 200000
|
||||
# The clamp is max_tokens = min(maxTokens, contextWindow − est −
|
||||
# 4096); the chars/4 est UNDER-counts the rendered prompt by ~3 %
|
||||
# on code/agent traffic (measured 2026-10-07: est 152,938 ->
|
||||
# server 157,388). Worst-case request = window − 4096 + 3 % est
|
||||
# error ≈ 156.2k, inside the engine ceiling with ~44k of margin
|
||||
# (robust to a ~25 % under-count). Compaction fires at 85 % of the
|
||||
# window and rides on frank-zealot: input ≈ threshold −
|
||||
# keepRecent(20k), comfortably inside its per-slot window.
|
||||
# server 157,388). Worst-case request = (200,000 − 4,096) × 1.03
|
||||
# ≈ 201.8k; the compaction trigger fires at 95 % (config.yml
|
||||
# thresholdPercent): 190,000 × 1.03 + 45,056 ≈ 240.8k — both inside
|
||||
# 262,144 (margin 60k / 21k). Compaction rides frank-zealot:
|
||||
# input ≈ 190,000 − keepRecent(20k) ≈ 175k ≤ its 200,704 slot.
|
||||
# Recompute all of it after ANY window or threshold change; tests/
|
||||
# frank_dragoon_unit.bats pins this arithmetic at 95 %.
|
||||
# HALOGEN_MAX_TOKENS_CAP in frank-dragoon.env must stay >= this.
|
||||
maxTokens: 45056
|
||||
reasoning: true
|
||||
|
||||
Reference in New Issue
Block a user