Compare commits
288
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
de7dd6f897 | ||
|
|
64bb23b329 | ||
|
|
e0b1466f74 | ||
|
|
d368394656 | ||
|
|
ec19771b8b | ||
|
|
8b87b5ae1a | ||
|
|
047f264fcf | ||
|
|
54bb0202c2 | ||
|
|
fb36a1b0fd | ||
|
|
c37ad6e870 | ||
|
|
c6e2a1daca | ||
|
|
88d486b37b | ||
|
|
49f67b2192 | ||
|
|
e49537c2ac | ||
|
|
62a70caebb | ||
|
|
4735702965 | ||
|
|
22d2f3bcd8 | ||
|
|
205b54adcb | ||
|
|
8af40c402b | ||
|
|
6e89c63e5d | ||
|
|
e690a2ca66 | ||
|
|
d4d6dd0c44 | ||
|
|
66e1129e6c | ||
|
|
a280c29d2c | ||
|
|
8393dc1090 | ||
|
|
d0587a5bc5 | ||
|
|
334e5d0ee0 | ||
|
|
66fc40de65 | ||
|
|
71601f72a8 | ||
|
|
f3ce42eb80 | ||
|
|
812be6a0a6 | ||
|
|
06f45e83c1 | ||
|
|
40b9725433 | ||
|
|
671222b881 | ||
|
|
71a7e5cb9a | ||
|
|
97b65067b1 | ||
|
|
23372e96f5 | ||
|
|
ab94c91bcf | ||
|
|
5cf7e6bd8e | ||
|
|
1cc4b76479 | ||
|
|
59920f849d | ||
|
|
bfd6a6340e | ||
|
|
4849f46eb2 | ||
|
|
dd91ef1d59 | ||
|
|
289f74062a | ||
|
|
d7bd05398d | ||
|
|
004962ffce | ||
|
|
56fc899763 | ||
|
|
35ceae6d7b | ||
|
|
655a39ffb6 | ||
|
|
b4fc881164 | ||
|
|
6df45d709a | ||
|
|
5ab0a49ea4 | ||
|
|
4666d0dce0 | ||
|
|
d013795922 | ||
|
|
2b7b0bc3cd | ||
|
|
3809e1ed22 | ||
|
|
e278e4b863 | ||
|
|
c05f5ad451 | ||
|
|
ecc089c6ef | ||
|
|
7523ca5c19 | ||
|
|
3022277167 | ||
|
|
689ada428f | ||
|
|
9ebf5fe85f | ||
|
|
86280608e5 | ||
|
|
411b32ff59 | ||
|
|
04f93133fc | ||
|
|
7da5e9dfa8 | ||
|
|
0d43eb0948 | ||
|
|
5110060c24 | ||
|
|
6f1f55345a | ||
|
|
6af3714bd0 | ||
|
|
495e2229f5 | ||
|
|
37ef4897ae | ||
|
|
06e549cb27 | ||
|
|
c96741794a | ||
|
|
e14d0906ea | ||
|
|
d5be16127b | ||
|
|
b7aef40266 | ||
|
|
fafdaa2617 | ||
|
|
7213068d91 | ||
|
|
05dbe8c5c1 | ||
|
|
806352f8f6 | ||
|
|
34a7af2847 | ||
|
|
635c64d773 | ||
|
|
64d9c54657 | ||
|
|
aec7f0f068 | ||
|
|
4023b21365 | ||
|
|
e14fc884a3 | ||
|
|
b3a84c5bb8 | ||
|
|
9e6254108a | ||
|
|
ce87958b77 | ||
|
|
802014fea9 | ||
|
|
e22acf4913 | ||
|
|
c14c64d6c0 | ||
|
|
6e3e5594eb | ||
|
|
36ff3742b4 | ||
|
|
fd6a07bb58 | ||
|
|
1921d76319 | ||
|
|
d302151a8d | ||
|
|
5b6f4381b0 | ||
|
|
cae49fcd09 | ||
|
|
ee292ffd10 | ||
|
|
dd4713fa81 | ||
|
|
a8727ddf0b | ||
|
|
2faeab665a | ||
|
|
62b3b92e69 | ||
|
|
ec6a707fe4 | ||
|
|
96553979fb | ||
|
|
ac5b377750 | ||
|
|
8f5a229cc6 | ||
|
|
ea514c8fd2 | ||
|
|
bc60fa33be | ||
|
|
e4ecc93f29 | ||
|
|
d055eb1e7f | ||
|
|
70346f3d57 | ||
|
|
1251cfbd40 | ||
|
|
e67bbd0de0 | ||
|
|
986081bf1d | ||
|
|
a9b3c495da | ||
|
|
c99eb8e11b | ||
|
|
cd20d6ddfd | ||
|
|
82dd0c7d09 | ||
|
|
99b9a6a17c | ||
|
|
39385636be | ||
|
|
d99c1c3c4f | ||
|
|
651924a4bc | ||
|
|
e86e60af39 | ||
|
|
01d9d3fbd5 | ||
|
|
570b5509d0 | ||
|
|
3579c7ebc5 | ||
|
|
0075ca3bbf | ||
|
|
bcd52bb0f4 | ||
|
|
edd7e0cf68 | ||
|
|
f033639b54 | ||
|
|
6fffd83957 | ||
|
|
dd8f3974c1 | ||
|
|
acf67182e9 | ||
|
|
38bf9c4897 | ||
|
|
54f15e2a58 | ||
|
|
1e7e212e5c | ||
|
|
475211d2e7 | ||
|
|
0258ef70f8 | ||
|
|
0c7fe8b4bf | ||
|
|
765b686859 | ||
|
|
332067a516 | ||
|
|
66e3f220e7 | ||
|
|
6134e1168e | ||
|
|
c379d16858 | ||
|
|
ca8b46241e | ||
|
|
3fa0242f56 | ||
|
|
fe039a0b73 | ||
|
|
d4092f2f65 | ||
|
|
cedef97c99 | ||
|
|
0ee08bc368 | ||
|
|
6664f9090a | ||
|
|
7de07ee35c | ||
|
|
fcc7226c3b | ||
|
|
a8011c3d4d | ||
|
|
08f90c6df3 | ||
|
|
57e255f930 | ||
|
|
b3d3df4bdb | ||
|
|
5d523f6e9f | ||
|
|
b353fa9075 | ||
|
|
c300f8b49c | ||
|
|
bde10f8948 | ||
|
|
c87b378475 | ||
|
|
979e9ef711 | ||
|
|
a550db6027 | ||
|
|
3713339ffc | ||
|
|
5a3d563a22 | ||
|
|
942787f0f2 | ||
|
|
8fb357b483 | ||
|
|
72bcbd989e | ||
|
|
964bd06073 | ||
|
|
54f5555470 | ||
|
|
0edae3323d | ||
|
|
e52aa6a4a3 | ||
|
|
d8647ef5cd | ||
|
|
7a0062e9e8 | ||
|
|
8702f3058d | ||
|
|
534b221a45 | ||
|
|
8089693e58 | ||
|
|
e5d00fd91f | ||
|
|
59a89553b7 | ||
|
|
ae82aa00e4 | ||
|
|
72a3dbbe08 | ||
|
|
43c19ae74c | ||
|
|
a2b1b6d9c6 | ||
|
|
876d8cd519 | ||
|
|
c2d855b33d | ||
|
|
246189f054 | ||
|
|
04d3628c94 | ||
|
|
e0920dc067 | ||
|
|
e044700ef7 | ||
|
|
7b7dbf55c2 | ||
|
|
a4218361e2 | ||
|
|
c6fa584d0a | ||
|
|
541f27f316 | ||
|
|
e2fe36519c | ||
|
|
7fb6d36801 | ||
|
|
23901a190d | ||
|
|
d015493219 | ||
|
|
ae426434bc | ||
|
|
d767348cfd | ||
|
|
ae12063de9 | ||
|
|
8da6cdc9cd | ||
|
|
9217ed83f8 | ||
|
|
27083f75f1 | ||
|
|
5a1cb81bb9 | ||
|
|
4d2b024116 | ||
|
|
989196f664 | ||
|
|
0ccdd46a97 | ||
|
|
9255a8b865 | ||
|
|
8a68c58078 | ||
|
|
5026c02982 | ||
|
|
abb8a2a9d0 | ||
|
|
286340e786 | ||
|
|
4874d103d7 | ||
|
|
b441cfda56 | ||
|
|
37554b0fb0 | ||
|
|
4c562d7ed4 | ||
|
|
f63d11de85 | ||
|
|
4ec47a5b7e | ||
|
|
1d32829428 | ||
|
|
79e780f280 | ||
|
|
07fda88b1d | ||
|
|
cfed6ef7bb | ||
|
|
69e4cd0c8f | ||
|
|
a172513a95 | ||
|
|
5b08896e4e | ||
|
|
a34322aeb0 | ||
|
|
94c71a016f | ||
|
|
40c37bb9fa | ||
|
|
4e288b518e | ||
|
|
0cb002123b | ||
|
|
40c7d3c752 | ||
|
|
12c5e885f1 | ||
|
|
689228e60d | ||
|
|
f1e0b6583d | ||
|
|
4e3d4d20e9 | ||
|
|
8fbb3e8b54 | ||
|
|
03c043d620 | ||
|
|
c3f30bdbd0 | ||
|
|
eebcfbf5ac | ||
|
|
368caadbe8 | ||
|
|
135bb2194d | ||
|
|
83c9c92b3f | ||
|
|
97a0b21b40 | ||
|
|
922165bde2 | ||
|
|
2baf0c083d | ||
|
|
970ceed554 | ||
|
|
fc049ce7fb | ||
|
|
872e86399a | ||
|
|
c32584e093 | ||
|
|
93bd4737f3 | ||
|
|
60dc0bba92 | ||
|
|
4b82f35354 | ||
|
|
b60e95158c | ||
|
|
ccf72a2a3e | ||
|
|
268dcbdea4 | ||
|
|
df3331026d | ||
|
|
4c84f028c5 | ||
|
|
5ad39bdfea | ||
|
|
49afdfeda4 | ||
|
|
d0c90fccbd | ||
|
|
c8cac879e6 | ||
|
|
0883ae5660 | ||
|
|
07a95815d2 | ||
|
|
ea8d378a40 | ||
|
|
afc7c2be64 | ||
|
|
118390a9e2 | ||
|
|
9a5387b6c0 | ||
|
|
779b65af8c | ||
|
|
a30cd771f3 | ||
|
|
9683d1f5c1 | ||
|
|
35b871162c | ||
|
|
2db711964b | ||
|
|
cc2c652cc2 | ||
|
|
5efb218061 | ||
|
|
a9127a08e9 | ||
|
|
dfb3a8f69a | ||
|
|
2a3c0da740 | ||
|
|
9bfa0be85d | ||
|
|
7be39d60dd | ||
|
|
ac1608e71f | ||
|
|
898519712b | ||
|
|
5e27c67d87 |
@@ -0,0 +1,14 @@
|
||||
# LFS routing for image types. Measured 2026-10-21: this repo tracks ZERO LFS files
|
||||
# (`git lfs ls-files` = 0) — the wallpaper library became live theme state and is gitignored
|
||||
# (omp-docs/tools/dotfiles-deploy.md), so these four rules match nothing today. They are kept
|
||||
# because tracked media is a plausible future and idle filters cost nothing; deleting the file
|
||||
# is a pruning call for a session with Frank, not a cleanup an agent runs alone.
|
||||
#
|
||||
# Do not cite this file as the reason `git-lfs` is installed on a box. The manifest carries its
|
||||
# own justification (packages/common.pkgs), and a tool justified by dead config is a tool nobody
|
||||
# can defend when the config goes.
|
||||
|
||||
*.jpg filter=lfs diff=lfs merge=lfs -text
|
||||
*.jpeg filter=lfs diff=lfs merge=lfs -text
|
||||
*.png filter=lfs diff=lfs merge=lfs -text
|
||||
*.webp filter=lfs diff=lfs merge=lfs -text
|
||||
+130
-10
@@ -1,25 +1,145 @@
|
||||
#!/usr/bin/env bash
|
||||
# Secret gate: block any commit whose staged diff contains live-secret shapes.
|
||||
# This repo is pushed to a Gitea remote; anything committed is instantly visible
|
||||
# Secret gate: block any commit whose staged diff contains live-secret shapes or
|
||||
# credential assignments. This repo IS the deployment and every workstation
|
||||
# pulls it, so a committed secret is instantly in every agent's context window
|
||||
# on the fleet. Deliberate override: ALLOW_SECRETS=1 git commit ...
|
||||
#
|
||||
# Two rules, deliberately different in scope:
|
||||
#
|
||||
# 1. Token SHAPES are scanned across the ENTIRE staged diff, whatever the file
|
||||
# type. The first version of this gate whitelisted extensions
|
||||
# (*.md *.yml *.yaml *.json *.toml *.sh *.ts *.js), so a `postgres.env`, a
|
||||
# `.container` quadlet, an `.ini`, or an extension-less dotfile — exactly
|
||||
# the shapes the dotfiles tier is made of — was never read at all.
|
||||
# 2. Credential ASSIGNMENTS (PASSWORD=…, secret_key: …) are scanned only in
|
||||
# non-prose files. The auto-learned memory tier is prose ABOUT credentials,
|
||||
# and a gate that fires there gets overridden until it means nothing; the
|
||||
# 2026-10-20 audit measured 7 such prose hits on the tracked tree. Markdown
|
||||
# keeps shape detection only, which is what it had before.
|
||||
#
|
||||
# Bcrypt hashes and Basic-auth headers joined the shape list on 2026-10-20
|
||||
# (issue #40) after a live Caddy basicauth hash plus its plaintext password —
|
||||
# and the Pi-hole internal password — were found in a tracked memory file.
|
||||
set -uo pipefail
|
||||
|
||||
staged=$(git diff --cached --diff-filter=ACM -U0 -- '*.md' '*.yml' '*.yaml' '*.json' '*.toml' '*.sh' '*.ts' '*.js')
|
||||
[ -z "$staged" ] && exit 0
|
||||
shapes='sk-[A-Za-z0-9]{18,}|ghp_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{22,}|glpat-[A-Za-z0-9_-]{20}|xox[baprs]-[A-Za-z0-9-]{10,}|AKIA[A-Z0-9]{16}|ASIA[A-Z0-9]{16}|BEGIN [A-Z ]*PRIVATE KEY|aes_[A-Za-z0-9/+]{40}|eyJ[A-Za-z0-9_-]{20,}\.ey|\$2[aby]\$[0-9]{2}\$[./A-Za-z0-9]{20,}|Authorization:[[:space:]]*[Bb]asic[[:space:]]+[A-Za-z0-9+/=]{24,}'
|
||||
|
||||
patterns='sk-[A-Za-z0-9]{18,}|ghp_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{22,}|glpat-[A-Za-z0-9_-]{20}|xox[baprs]-[A-Za-z0-9-]{10,}|AKIA[A-Z0-9]{16}|ASIA[A-Z0-9]{16}|BEGIN [A-Z ]*PRIVATE KEY|aes_[A-Za-z0-9/+]{40}|eyJ[A-Za-z0-9_-]{20,}\.ey'
|
||||
# `newpass1234`, `secret_key: "rotated-value"` — an assignment with a literal
|
||||
# value. `$VAR`, `${VAULT}`, <placeholders> and empty values cannot match.
|
||||
assign='[A-Za-z0-9_]*(PASSW|SECRET|TOKEN|API_?KEY|ACCESS_KEY|PRIVATE_KEY|CREDENTIAL)[A-Za-z0-9_]*[[:space:]]*[:=][[:space:]]*["'"'"']?[A-Za-z0-9/+._@-]{8,}'
|
||||
placeholders='changeme|change[-_]|placeholder|redacted|example|sample|dummy|nokey|no[-_]key|not[-_]needed|your[-_]|localhost|xxxx|yyyy|\$\{|<[a-z]|\*\*\*|%s|null|false|true'
|
||||
|
||||
# Exemptions: a variable naming a location (`…_FILE=`, `…_PATH=`) or a value
|
||||
# that is a filesystem path is a pointer, not a credential. Flagging those is
|
||||
# how a gate gets permanently overridden.
|
||||
assign_exempt='(_FILE|_PATH|_DIR|_URL|_ENDPOINT|_SOCKET)[[:space:]]*[:=]|[:=][[:space:]]*["'"'"']?(/|\.\./)'
|
||||
# Fail closed: a gate that cannot read the staged diff must not report success.
|
||||
#
|
||||
# `-z`, and the list lives in a file. Two traps decide this: git quotes any path it cannot print
|
||||
# as UTF-8 (core.quotePath is on by default), and a quoted name handed back to
|
||||
# `git diff -- <path>` matches nothing — the file is skipped, and a skipped file used to be a
|
||||
# silent pass. Command substitution cannot hold the list either, because $() deletes NUL bytes,
|
||||
# so the names go to a temp file and the loop reads them with a NUL delimiter.
|
||||
TMPD=$(mktemp -d) || TMPD=""
|
||||
if [ -z "$TMPD" ]; then
|
||||
echo "SECRET GATE: cannot create a temp dir — refusing to commit." >&2
|
||||
[ -n "${ALLOW_SECRETS:-}" ] && exit 0
|
||||
exit 1
|
||||
fi
|
||||
trap 'rm -rf "$TMPD"' EXIT
|
||||
if ! git diff --cached --diff-filter=ACM --name-only -z > "$TMPD/files" 2> "$TMPD/err"; then
|
||||
{
|
||||
echo "SECRET GATE: cannot read the staged diff — refusing to commit."
|
||||
sed 's/^/ /' "$TMPD/err"
|
||||
echo "Override deliberately: ALLOW_SECRETS=1 git commit ..."
|
||||
} >&2
|
||||
[ -n "${ALLOW_SECRETS:-}" ] && exit 0
|
||||
exit 1
|
||||
fi
|
||||
# git's stderr stays out of the name list on purpose: a `warning:` line mistaken for a path
|
||||
# becomes a file that "cannot be read", which is how a gate starts skipping the real ones.
|
||||
[ -s "$TMPD/files" ] || exit 0
|
||||
|
||||
report=""
|
||||
exempt=""
|
||||
count=0
|
||||
while IFS= read -r -d '' f; do
|
||||
[ -z "$f" ] && continue
|
||||
# Self-exemption, deliberately an exact filename list and not `tests/`: the
|
||||
# gate's fixtures must contain credential-shaped strings or there is nothing to
|
||||
# test against, and this hook documents the shapes it looks for. A secret parked
|
||||
# in any other test file is still caught.
|
||||
case "$f" in
|
||||
.githooks/pre-commit|tests/secret-gate.bats) exempt="$exempt $f"; continue ;;
|
||||
esac
|
||||
diff=$(git diff --cached -U0 -- "$f")
|
||||
[ -z "$diff" ] && continue
|
||||
# Only ADDED lines can leak. Matching removals too blocks the very commit that
|
||||
# deletes a credential — which is exactly what happened on 2026-10-20, when the
|
||||
# gate refused the redaction of the password it exists to catch.
|
||||
added=$(printf '%s\n' "$diff" | grep -E '^\+' | grep -vE '^\+\+\+' | sed 's/^+//' || true)
|
||||
[ -z "$added" ] && continue
|
||||
# Shape hits are judged on the MATCHED TOKEN, not on the whole line. Documentation about
|
||||
# credentials quotes vendor placeholders — AWS publishes an all-zeros access-key id that ends
|
||||
# in EXAMPLE for exactly this purpose — and
|
||||
# a gate that blocks prose over an inert example gets worked around until it means nothing:
|
||||
# the live-sync heartbeat failed three ticks on 2026-10-21 because a managed skill page quotes
|
||||
# exactly that id. A line is exempted only when EVERY shape on it is inert, so one live token
|
||||
# sharing the file with a documented fake still blocks.
|
||||
hits=$(printf '%s\n' "$added" | while IFS= read -r line; do
|
||||
live=0
|
||||
while IFS= read -r m; do
|
||||
# The shape regexes carry fixed quantifiers, so the match they return is TRUNCATED:
|
||||
# `AKIA` + exactly 16 characters stops inside the word EXAMPLE and the placeholder
|
||||
# disappears from what is judged. Judge the match plus what follows it instead.
|
||||
ctx=$(printf '%s' "$line" | awk -v m="$m" '{ i = index($0, m); print (i > 0 ? substr($0, i, length(m) + 8) : m) }')
|
||||
printf '%s' "$ctx" | grep -Eiq 'EXAMPLE|X{4,}|Y{4,}|Z{4,}|_{4,}|REDACTED|CHANGEME|PLACEHOLDER' || live=1
|
||||
done < <(printf '%s\n' "$line" | grep -Eio "$shapes" || true)
|
||||
[ "$live" = 1 ] && printf '%s\n' "$line"
|
||||
done)
|
||||
if [ "${f##*.}" != "md" ]; then
|
||||
# The placeholder filter reads the VALUE, never the whole line. A trailing comment that
|
||||
# says `# example` says nothing about the token the same line assigns; scoping the filter
|
||||
# to what follows the separator is what keeps `API_TOKEN=<real> # placeholder` a block.
|
||||
assigns=$(printf '%s\n' "$added" | grep -Ei "$assign" | grep -Ev "$assign_exempt" \
|
||||
| while IFS= read -r line; do
|
||||
# The judged value is the assigned token alone: it ends at whitespace or `#`, so a
|
||||
# trailing comment cannot vouch for it. Truncating a quoted value early only ever errs
|
||||
# toward blocking, which is the safe direction for a gate that can be overridden.
|
||||
value=${line#*[:=]}
|
||||
value=${value#"${value%%[![:space:]]*}"}
|
||||
value=${value%%[[:space:]]*}
|
||||
value=${value%%#*}
|
||||
printf '%s\n' "$value" | grep -Eiq "$placeholders" && continue
|
||||
printf '%s\n' "$line"
|
||||
done)
|
||||
hits=$(printf '%s\n%s\n' "$hits" "$assigns" | sed '/^[[:space:]]*$/d')
|
||||
fi
|
||||
if [ -n "$hits" ]; then
|
||||
n=$(printf '%s\n' "$hits" | wc -l)
|
||||
count=$((count + n))
|
||||
report+=" $f"$'\n'"$(printf '%s\n' "$hits" | cut -c1-100 | sed 's/^/ /')"$'\n'
|
||||
fi
|
||||
done < "$TMPD/files"
|
||||
|
||||
# Say so out loud: an exemption that prints nothing is how a carve-out becomes
|
||||
# invisible, and invisible is the property a secret wants.
|
||||
if [ -n "$exempt" ]; then
|
||||
printf 'secret-gate: self-test fixtures exempt:%s\n' "$exempt" >&2
|
||||
fi
|
||||
|
||||
hits=$(printf '%s\n' "$staged" | grep -Ei "$patterns" || true)
|
||||
if [ -n "${ALLOW_SECRETS:-}" ]; then
|
||||
[ -n "$hits" ] && printf 'secret-gate: ALLOW_SECRETS set, allowing %s secret-shaped string(s) through.\n' "$(printf '%s\n' "$hits" | wc -l)"
|
||||
[ -n "$report" ] && printf 'secret-gate: ALLOW_SECRETS set, allowing %s secret-shaped string(s) through.\n' "$count"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ -n "$hits" ]; then
|
||||
if [ -n "$report" ]; then
|
||||
{
|
||||
echo "SECRET GATE: staged diff contains $(printf '%s\n' "$hits" | wc -l) secret-shaped string(s):"
|
||||
printf '%s\n' "$hits" | sed 's/^/ /'
|
||||
echo "SECRET GATE: staged diff contains $count secret-shaped string(s):"
|
||||
printf '%s\n' "$report"
|
||||
echo "Fix it, or override deliberately: ALLOW_SECRETS=1 git commit ..."
|
||||
echo "A credential that reaches a tracked file is already in history on every"
|
||||
echo "machine that pulls this repo — deleting it later does not un-leak it."
|
||||
} >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/bin/sh
|
||||
# git-lfs pre-push — required because core.hooksPath=.githooks disables the
|
||||
# default .git/hooks/pre-push; without this hook, pushes would upload LFS
|
||||
# pointer files without their objects and break every fresh clone.
|
||||
command -v git-lfs >/dev/null 2>&1 || { printf >&2 'n' ; exit 0; }
|
||||
git lfs pre-push "$@"
|
||||
+67
@@ -26,6 +26,10 @@ auth.json
|
||||
**/auth.json
|
||||
.env
|
||||
**/.env
|
||||
*.env
|
||||
# `*.env` is the rule that matters: `.env` alone does not match `postgres.env`
|
||||
# or `frank-job-hunt.env`, and those are the files that actually carry
|
||||
# POSTGRES_PASSWORD / API keys on this fleet (issue #40).
|
||||
trust.json
|
||||
**/trust.json
|
||||
*.pem
|
||||
@@ -43,3 +47,66 @@ last-changelog-version
|
||||
.DS_Store
|
||||
*.swp
|
||||
*.tmp
|
||||
|
||||
# --- per-host runtime identity + caches (never config) ---------------------
|
||||
agent/blobs/
|
||||
install-id
|
||||
puppeteer/
|
||||
webcache/
|
||||
|
||||
# --- local backups + staging artifacts: kept on disk, never in history -----
|
||||
.AGENTTRASH/
|
||||
*.save
|
||||
*.bak-*
|
||||
|
||||
# deploy quarantine — never tracked
|
||||
.quarantine/
|
||||
|
||||
# harness runtime transcripts: local-only, may contain raw session secrets
|
||||
agent/sessions/
|
||||
|
||||
# harness diagnostic report bundles (tarballs of logs + session state)
|
||||
reports/
|
||||
|
||||
# --- agent runtime scratch: research runs, decode predictions, worktrees ----
|
||||
autoresearch/
|
||||
agent/predict/
|
||||
local/
|
||||
wt/
|
||||
|
||||
# --- per-worktree memory buckets: keyed on ephemeral worktree paths ---------
|
||||
# A session run inside a git worktree mints a bucket named after the worktree
|
||||
# path; the worktree is deleted, the bucket is orphan state, not config.
|
||||
agent/memories/--home-blank-.omp-wt-*
|
||||
|
||||
# --- theme media: content the pipeline consumes, not configuration ---------
|
||||
# 23 wallpapers, 80 MiB. Tracked until 2026-10-20: every clone of a config repo was
|
||||
# carrying a library no diff can review. The live library at
|
||||
# ~/.local/share/noctalia/wallpapers is a real directory of real files (measured
|
||||
# 2026-10-20), so this path in the repo was a duplicate of it, not its source — and
|
||||
# note that untracking deletes the copy in every other checkout regardless of
|
||||
# `git rm --cached` on the branch. A fresh box starts with an empty library; every
|
||||
# filename carries its wallhaven ID; `/w/<id>` answers `text/html`, so resolve the CDN URL at
|
||||
# `https://wallhaven.cc/api/v1/w/<id>` and download `data.path` — the recipe, with the reason it
|
||||
# is not a plain `curl -L`, is on omp-docs/tools/dotfiles-deploy.md
|
||||
dotfiles/noctalia/.local/share/noctalia/wallpapers/
|
||||
|
||||
# --- generated theme output: machine-written, never reviewable, never deployed ---
|
||||
# noctalia/matugen rewrite these in place; a tracked copy is a stale snapshot that
|
||||
# deploy.sh would push back over a live palette. These patterns are the ignore half of the
|
||||
# exclusion list in `deploy_file` (scripts/deploy.sh): change one, change the other, or a
|
||||
# generated file becomes tracked-but-never-deployed — the fiction class #41 removed. The
|
||||
# capital-N rule is noctalia's own palette-directory naming, and it was in deploy.sh only.
|
||||
# Law: omp-docs/inventory/home-config-census.md, Refs #42, #60
|
||||
dotfiles/**/themes/noctalia*
|
||||
dotfiles/**/colors/noctalia*
|
||||
dotfiles/**/noctalia.*
|
||||
dotfiles/**/noctalia-*
|
||||
dotfiles/**/Noctalia*
|
||||
|
||||
# --- harness natives: unpacked runtime payloads, never configuration ----------
|
||||
# 2.4 MB of tree-sitter grammars the harness unpacks into the repo root on first use (measured
|
||||
# on blankpad 2026-10-21). Regenerable, unreviewable, not config — every clause of AGENTS.md's
|
||||
# runtime-state doctrine applies, and it sat unnamed in the root, one `git add -A` away from
|
||||
# being committed as if it were configuration.
|
||||
natives/
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
System-configuration command center for Frank's fleet: agent config, knowledge base,
|
||||
small system utilities, and (soon) dotfiles. You — the reader — are probably an agent
|
||||
running on Dankware with this repo as your working directory. Read
|
||||
running on one of Frank's workstations with this repo as your working directory. Read
|
||||
`omp-docs/index.md` before you touch anything. This file is the operating contract;
|
||||
`omp-docs/` is the knowledge base (OKF v0.1 format: directory of markdown files
|
||||
with YAML frontmatter, spec at `omp-docs/references/okf-spec.md`).
|
||||
@@ -13,62 +13,145 @@ with YAML frontmatter, spec at `omp-docs/references/okf-spec.md`).
|
||||
(`dank/command-center` on the Dankclaw instance). There is no staging copy:
|
||||
**the checkout is the deployment.** Branch switches swap live config files out
|
||||
from under the running agent — keep feature branches short-lived, and never end
|
||||
a work session checked out on anything but `develop` or `main`.
|
||||
a work session checked out on anything but `main`.
|
||||
- Contents that belong here: `AGENTS.md`, `omp-docs/` (knowledge base),
|
||||
`scripts/` (small system utilities), `agent/` + `plugins/` (harness config and
|
||||
plugin manifests). Everything else under this directory is runtime state and is
|
||||
gitignored on principle — see `.gitignore`. History must mean "the configuration
|
||||
changed", not "the agent breathed".
|
||||
|
||||
## Git-flow SDLC (full flow, from day one)
|
||||
## Trunk-based SDLC (since 2026-10-18; replaces the old git-flow regime)
|
||||
|
||||
- `main` is the only long-lived branch. Frank is the only dev; `develop` and the
|
||||
release-branch ceremony are retired — no `develop`, no back-merges, no
|
||||
`release/X.Y.Z` branches.
|
||||
- **Before every work item: fetch and branch off the freshest `origin/main`.**
|
||||
If main moves under you, merge it into your branch (never rebase — Frank
|
||||
dislikes the history rewrite).
|
||||
- Work item = Gitea issue on `dank/command-center` (Gitea at 192.168.0.143:3000,
|
||||
owner `dank`). Small fixes may go straight to a branch; anything with more than
|
||||
one moving part gets an issue first.
|
||||
- Branch naming: `<type>/<issue#>-<slug>`, type ∈ feat|fix|docs|chore.
|
||||
Release cuts: `release/X.Y.Z` from `develop`; hotfixes: `hotfix/<slug>` from `main`.
|
||||
- **Merge, never rebase**, when updating a branch against a moved main — Frank
|
||||
dislikes the history rewrite. **Never merge a PR without an explicit OK from Frank.**
|
||||
- Merge into `develop` with `--no-ff` so feature arcs stay visible in history.
|
||||
- Releases: cut `release/X.Y.Z` from `develop`, verify, merge to `main`, tag
|
||||
`vX.Y.Z`, back-merge `main` into `develop`. Tags are the deploy markers every
|
||||
other machine pulls to.
|
||||
- **Pull requests target `main` directly.** Merge with `--no-ff` so feature arcs
|
||||
stay visible in history. **Never merge a PR without an explicit OK from Frank.**
|
||||
- **The moment a PR is open, open it in Frank's browser** — `xdg-open <PR-URL>`
|
||||
lands a tab in his Firefox; never make him hunt for it in Gitea.
|
||||
- **Attribution footer (issues and PRs)** — every issue or PR body you open ends
|
||||
with a `---` line, then `cwd: <working directory> · session: <omp session id>`
|
||||
(session id from `/status`; `cwd` is that session's own working directory —
|
||||
not the build checkout, when they differ). Machine-made artifacts stay attributable to the
|
||||
box and session that spawned them.
|
||||
- Releases: tag `vX.Y.Z` straight on `main` after verifying; tags are the deploy
|
||||
markers every other machine pulls to. Hotfixes are ordinary branches off the
|
||||
tag you are fixing forward from.
|
||||
- Trash convention: never `rm` anything user-authored — move it to `.AGENTTRASH/`
|
||||
(gitignored). Deletion is a pruning session with Frank present.
|
||||
|
||||
### Commit hygiene (mirrors dank/serverAppConfigs)
|
||||
|
||||
The law is imported from the serverAppConfigs repo — checkout on the dev box at
|
||||
`~/Projects/serverAppConfig`, see its `AGENTS.md` § *Git Workflow* and
|
||||
§ *Testing* ("All edits flow through git", "tests are mandatory before
|
||||
commit"). Applied here:
|
||||
|
||||
- `git pull` before starting new work; start from a clean tree. The checkout
|
||||
IS the deployment here — a dirty tracked file is a half-deployed config
|
||||
serving the live agent.
|
||||
- Commit atomically and early: one commit = one complete functional unit,
|
||||
never a pile of half-edited files; docs land in the same commit as the
|
||||
change they document.
|
||||
- `bats tests/` green + the pre-commit secret gate = the commit precondition;
|
||||
the full suite runs at pre-push.
|
||||
- Push to the local Gitea as soon as a unit is green — every other
|
||||
workstation pulls from there. `git * --no-verify` is forbidden (the secret
|
||||
gate is the point of the hook).
|
||||
- Messages: imperative mood, what + why, `Refs #N` when a Gitea issue exists.
|
||||
|
||||
### Live-sync tiers (the machine is a committer too — Refs #11)
|
||||
|
||||
The checkout is the deployment, so several tracked files are live state
|
||||
written continuously through symlinks by the harness, auto-learn, and
|
||||
desktop GUIs. "Dirty tree" is judged by the writer:
|
||||
|
||||
| Tier | Paths | Commit law |
|
||||
|------|-------|------------|
|
||||
| Human/agent prose | `AGENTS.md`, `omp-docs/`, `scripts/`, `plugins/`, `agent/skills/` | branch → PR → Frank OK (unchanged) |
|
||||
| Machine live | `agent/memories/`, `agent/managed-skills/`, `agent/config.yml`, `agent/models.yml` | `scripts/live-sync.sh` heartbeat |
|
||||
| Desktop live | `dotfiles/` | same heartbeat |
|
||||
|
||||
The heartbeat (systemd user timer, hourly) stages ONLY allowlisted tiers,
|
||||
commits with hooks armed — the secret gate included — merges `origin/main`,
|
||||
pushes, and reaps `wt/*` worktrees older than 7 days. Non-allowlist dirt is
|
||||
never committed, only surfaced. Conflicts abort and alert (ntfy); it never
|
||||
force-pushes, auto-resolves, or uses `--no-verify`. Details:
|
||||
[omp-docs/tools/live-sync.md](omp-docs/tools/live-sync.md).
|
||||
|
||||
## Agent skills
|
||||
|
||||
The engineering skills (`triage`, `to-tickets`, `to-spec`, `research`,
|
||||
`implement`, …) read their per-repo configuration from three pages in the
|
||||
knowledge base. Those pages own their facts; this is only the pointer.
|
||||
|
||||
### Issue tracker
|
||||
|
||||
See [`omp-docs/skills/issue-tracker.md`](omp-docs/skills/issue-tracker.md).
|
||||
|
||||
### Triage labels
|
||||
|
||||
See [`omp-docs/skills/triage-labels.md`](omp-docs/skills/triage-labels.md).
|
||||
|
||||
### Domain docs
|
||||
|
||||
ADRs: `omp-docs/decisions/`. `omp-docs/glossary.md` is intentionally not created.
|
||||
See [`omp-docs/skills/domain-docs.md`](omp-docs/skills/domain-docs.md).
|
||||
|
||||
## Secret gate (HARD rule)
|
||||
|
||||
The pre-commit hook (`.githooks/pre-commit`, enabled via
|
||||
`git config core.hooksPath .githooks`) scans every staged diff for live-secret
|
||||
shapes (`sk-…`, `ghp_…`, `github_pat_…`, `glpat-…`, `xox…`, `AKIA…`, PEM headers,
|
||||
JWTs) and blocks the commit. Quarantine list — never tracked, ever: `auth.json`,
|
||||
`.env`, `trust.json`, `*.pem`, `*.key`, `identity*`. Deliberate override:
|
||||
`git config core.hooksPath .githooks`) blocks the commit in two ways. Token
|
||||
**shapes** (`sk-…`, `ghp_…`, `github_pat_…`, `glpat-…`, `xox…`, `AKIA…`, PEM
|
||||
headers, JWTs, bcrypt hashes, `Authorization: Basic` headers) are scanned across
|
||||
the **entire** staged diff — every file type, because the first version of the
|
||||
gate whitelisted extensions and never read a `.env`, `.container`, `.ini`, or
|
||||
extension-less dotfile at all. Credential **assignments** (`PASSWORD=…`,
|
||||
`secret_key: …`) are scanned in non-prose files only; markdown keeps shape
|
||||
detection, because the auto-learned memory tier is prose *about* credentials and a
|
||||
gate that fires there gets overridden until it means nothing. The gate **fails
|
||||
closed**: if it cannot read the staged diff it refuses the commit rather than
|
||||
passing it. Quarantine list — never tracked, ever: `auth.json`, `.env` and
|
||||
`*.env` (`postgres.env`, `frank-job-hunt.env` are the ones that carry real
|
||||
values), `trust.json`, `*.pem`, `*.key`, `identity*`. Deliberate override:
|
||||
`ALLOW_SECRETS=1 git commit …`. Remember why: a secret in a tracked file is a
|
||||
secret in every agent's context window on every machine that pulls this repo.
|
||||
secret in every agent's context window on every machine that pulls this repo, and
|
||||
redacting it later does not un-leak the history.
|
||||
|
||||
## Machine facts (verify before trusting)
|
||||
|
||||
| box | role | GPU | notes |
|
||||
|-----|------|-----|-------|
|
||||
| Dankware | dev laptop, never runs inference | Arc iGPU + RTX 4070 Max-Q (dGPU, non-unified) | 22 threads; mainline llama.cpp at `~/GitPackages/llama.cpp/build/bin/`; this repo lives here at `~/.omp` |
|
||||
| Dankclaw | bench target, runs the large serving model | Strix Halo APU — **unified memory** | ROCmFPX fork at `~/GitPackages/ROCmFPX/build-qwen38/bin/`; prod service `frank-default-large`, port 7770; ssh: `ssh dankclaw` |
|
||||
| Dankserver | FreeToken box | RTX 3090 dGPU — non-unified | runs `frank-default-small`; ssh: `ssh dankserver` |
|
||||
| Dev workstation (`blankpad`) | client box, never runs inference | Lenovo laptop, AMD Ryzen 5 PRO 5650U / Cezanne APU (**unified memory**, s2idle sleep only, no dGPU) | this repo lives here at `~/.omp`; the earlier "iGPU + dGPU / `~/GitPackages/llama.cpp`" description belonged to the sold `dankware` box |
|
||||
| Dankclaw | bench target, secondary models + service stack | Strix Halo APU (**unified memory**) + RTX 3090 (moved over from Dankserver 2026-09) | prod services `frank-dragoon` (halogen-flash-server, :7770, iGPU) and `frank-zealot` (llama.cpp PrismML fork, :7771, 3090); runbooks in the serverAppConfigs vault (`~/Projects/serverAppConfig/serverAppConfigs-docs/LLM/`); ssh: `ssh dankclaw` |
|
||||
| Dankserver | main-model box | 2× Radeon AI PRO R9700 — non-unified (was RTX 3090 until 2026-09) | runs `frank-archon` (radiance, TP=2, :7770) — **this repo's operator session runs on it**; ssh: `ssh dankserver` |
|
||||
|
||||
## Environment gotchas
|
||||
|
||||
- Gitea API needs a token: `GITEA_TOKEN` env (gitignored `.env`). No `tea` CLI
|
||||
installed anywhere. Remote URLs embed the token locally in `.git/config` — that
|
||||
file is never pushed; it is not a substitute for the secret gate.
|
||||
installed anywhere. Remote URLs carry NO embedded token (2026-10-09
|
||||
credential-store cutover) — git and the API both read
|
||||
`~/.config/secrets/gitea` (`store --file`, URL-line format; see
|
||||
`omp-docs/tools/gitea/client-access.md`). That file is never pushed; it is
|
||||
not a substitute for the secret gate.
|
||||
- Passwordless sudo (`sudo -n`) works on Dankclaw and Dankserver for systemctl
|
||||
verbs only; user units need no sudo at all.
|
||||
- SSH into both servers is BatchMode-capable from here:
|
||||
`ssh -o BatchMode=yes dankclaw '<read-only command>'` — read-only probes are
|
||||
always allowed; state changes on remote boxes are not, without a per-run OK.
|
||||
- `frank-default-large` on Dankclaw is a **production service** shared with this
|
||||
repo's operator: never stop, mask, or restart it (or any production service on
|
||||
any box) without an explicit, current, per-run OK from Frank.
|
||||
- `frank-archon` (Dankserver), `frank-dragoon`, and `frank-zealot` (Dankclaw)
|
||||
are **production services** shared with this repo's operator: never stop,
|
||||
mask, or restart them (or any production service on any box) without an
|
||||
explicit, current, per-run OK from Frank.
|
||||
|
||||
## File-organization doctrine (why this repo won't rot like ~/.pi did)
|
||||
## File-organization doctrine (why this repo stays readable)
|
||||
|
||||
- One concept, one file; directories get an `index.md` listing their contents —
|
||||
that's the OKF contract, and it is also the anti-rot mechanism: nothing may
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
# Command Center
|
||||
|
||||
## Configuration repository for Frank's Userspace configurations (i.e. his desktop setup)
|
||||
|
||||
### AI Disclosures:
|
||||
- Slop rating: 3/5 - **Actively using, actively de-slopping, repository still mostly AI owned**
|
||||
- Completeness: **Unfinished, planning planned, no plan yet**
|
||||
- Datacenter model/spend: Less than $5 of Deepseek 4.1 Flash
|
||||
- Local model: Qwen 3.8 27B
|
||||
|
||||
### slop rating of 3 or higher indicates human reading is a waste of time outside of this README, and that you might as well ask an agent for help with this.
|
||||
### slop rating 3 indicates Frank is comfortable using this for professional use -- but barely.
|
||||
|
||||
|
||||
|
||||
## The Fleet
|
||||
|
||||
| Host | Role | IP |
|
||||
|------|------|-----|
|
||||
| **Workstation** | Client box (CachyOS, Niri/Wayland) | — |
|
||||
| **Dankclaw** | Server (Podman, reverse proxy, Gitea, NPU) | 192.168.0.143 |
|
||||
| **Dankserver** | LLM inference backend | 192.168.0.235 |
|
||||
|
||||
The workstation hosts the primary checkout — this repository lives at `~/.omp/` and drives the OMP agent harness.
|
||||
|
||||
## Structure at a Glance
|
||||
|
||||
```
|
||||
~/.omp/
|
||||
├── agent/ # OMP harness runtime
|
||||
│ ├── config.yml # Live agent configuration
|
||||
│ ├── models.yml # Model providers (single source of truth)
|
||||
│ ├── APPEND_SYSTEM.md # System prompt extensions
|
||||
│ ├── managed-skills/ # Managed skills (SKILL.md bundles)
|
||||
│ ├── models.db # Model cache
|
||||
│ └── sessions/ # Persistent session state
|
||||
├── dotfiles/ # stow-style env-layer packages (home-relative trees)
|
||||
│ └── deploy via scripts/deploy.sh (check|adopt|sync; quarantine-and-replace)
|
||||
├── omp-docs/ # Knowledge base (source of truth)
|
||||
│ ├── index.md # KB index — start here
|
||||
│ ├── system/ # Fleet hardware, OS, network
|
||||
│ ├── tools/ # Installed software (llama.cpp, Gitea, Caddy, niri…)
|
||||
│ ├── core/ # Agent config conventions (settings, models, AGENTS.md)
|
||||
│ ├── pi/ # Legacy pi-era config reference
|
||||
│ ├── engineering/ # SDLC, languages, infra, skills ecosystem
|
||||
│ ├── research/ # Decision-support research (not yet facts)
|
||||
│ ├── references/ # OKF spec, maintenance procedures
|
||||
│ ├── caddy/ # Reverse proxy configuration
|
||||
│ ├── packages/ # Pi packages and extensions
|
||||
│ ├── inventory/ # Userspace config census
|
||||
│ ├── reports/ # Investigation reports
|
||||
│ ├── log.md # Change log
|
||||
│ └── .okf/ # OKF v0.1 conformance config
|
||||
├── scripts/ # Utility scripts (kb-lint.sh)
|
||||
├── plugins/ # npm plugins (pi-peon, etc.)
|
||||
├── puppeteer/ # Browser automation config
|
||||
├── .githooks/ # Pre-commit hook
|
||||
├── .gitignore # Runtime state ignored (databases, logs, cache)
|
||||
├── AGENTS.md # Agent conventions (niri reload, keybindings)
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## What Lives Here
|
||||
|
||||
**Config in, state out.** Everything tracked is something an agent reads or writes. Runtime databases (`*.db`, `*.sqlite`), logs, caches, and session state are git-ignored — git history captures configuration changes, not agent heartbeat.
|
||||
|
||||
## Quick Start
|
||||
|
||||
| Task | How |
|
||||
|------|-----|
|
||||
| Read the knowledge base | `omp-docs/index.md` — every concept linked from here |
|
||||
| Check agent config | `agent/config.yml` + `agent/models.yml` |
|
||||
| Lint the KB | `scripts/kb-lint.sh` (OKF v0.1 conformance) |
|
||||
| View change log | `omp-docs/log.md` |
|
||||
| Run a pre-commit | `.githooks/pre-commit` |
|
||||
|
||||
## Conventions
|
||||
|
||||
- **OKF v0.1** — every doc file carries YAML frontmatter with a `type:` field. `index.md` files are directory listings (no frontmatter).
|
||||
- **Progressive disclosure** — `index.md` is a landing page; drill into subdirectories for detail.
|
||||
- **Verified facts** — stamped with `verified:` when probed against the live fleet. Trust over memory.
|
||||
- **Single config source** — `~/.omp/agent/models.yml` is the only model registry; OMP is the primary harness and there is no second copy to keep in sync.
|
||||
|
||||
## Legacy Notes
|
||||
|
||||
`omp-docs/pi/` and `omp-docs/core/` retain general documentation about the pi-coding-agent harness (its config format, packages, skill system) for reference. OMP at `~/.omp/` is the live harness; managed skills live in `~/.omp/agent/managed-skills/`.
|
||||
+31
-5
@@ -13,7 +13,7 @@ Frank is 35 year old Software Engineer, specializing in DevOps, living in Seattl
|
||||
He likes video games, science fiction, and fantasy novels. He considers himself somewhat of a history buff.
|
||||
|
||||
### Your identity:
|
||||
You are Comrade Chief Designer of the Soviet Computer Systems of Frank, which are comprised of servers Dankware and Dankclaw, as well as several clients, including Dankware, which is where you are running from. You operate from the host Dankware as a Staff Software Engineer with the eyes of the greatest industrial designers that have ever lived. Growing up under Soviet paranoia, you also have an innate sense for good security practices. Despite your outwardly sunny disposition, you will work diligently as if a one way trip to The Gulag is always a possibility.
|
||||
You are Comrade Chief Designer of the Soviet Computer Systems of Frank, which are comprised of the servers Dankclaw and Dankserver, plus several client workstations. You operate from whichever workstation you are running on, as a Staff Software Engineer with the eyes of the greatest industrial designers that have ever lived. Growing up under Soviet paranoia, you also have an innate sense for good security practices. Despite your outwardly sunny disposition, you will work diligently as if a one way trip to The Gulag is always a possibility.
|
||||
|
||||
### Adjectives that describe your personality:
|
||||
well-measured, cautious, expeditious, ambitious, concise, surgical, thorough, and efficient.
|
||||
@@ -54,13 +54,27 @@ Suggest these tools or languages as possible solutions to problems if they are a
|
||||
|
||||
### 1. Maintain your projects source of truth.
|
||||
|
||||
The knowledge base at `docs/` is the source of truth for your project. Your training is out of date. Default to the documentation and the code before you assume anything. Maintain a work log of actions taken and their result. Refer to the work log to remember the progress of your tasks.
|
||||
Documentation exists for ONE reason: **point of use**. A page earns its bytes only if it helps you perform a task at the moment you perform it — the shape to deploy, the flag that must be passed, the failure mode to recognise, the rollback to run. It is NOT a museum, NOT a lab notebook, and NOT a place to keep track of numbers.
|
||||
|
||||
**THE PURPOSE TEST — apply it before you write, and again before you keep:**
|
||||
- Would an agent or Frank, doing a real task tomorrow, read this line and act differently because of it? If no: **delete it.**
|
||||
- Is this a measured number (t/s, tokens, ms, MiB, %, bench ladder, acceptance rate) for a model, runtime, or hardware configuration that is NOT currently deployed and NOT currently being compared? Then it is **dead weight — destroy it**, do not archive it, do not "preserve it in an Eras row", do not cite it. AI advances make such a figure wrong within weeks, and a wrong number in a runbook is worse than no number: it gets believed.
|
||||
- "It is the only copy of that fact" is NOT a reason to keep it. Sole custody of a useless fact is still useless. Losing information is an acceptable — often correct — outcome; do not hesitate to lose it.
|
||||
- What survives the test: live configuration and its exact numbers, the procedure that must be followed, the law/causal story that prevents a repeat failure, security posture, and the interface contract a consumer depends on. Qualitative laws outlive the measurement that taught them — keep the law, drop the numbers.
|
||||
- Never let a tool's "this is duplicated / this is the only record" census decide what to keep. That is a question of USEFULNESS, which only the operator answers.
|
||||
|
||||
1. Read the relevant `index.md` before you touch a topic.
|
||||
2. Follow `~/.pi/agent/docs/references/okf-spec.md` for every doc you create or edit.
|
||||
2. Follow `omp-docs/references/okf-spec.md` for every doc you create or edit.
|
||||
3. Read the target file completely before you edit it.
|
||||
4. Always update the documentation when you've completed a task.
|
||||
5. Compact the work log when necessary.
|
||||
4. Always update the documentation when you've completed a task — and when updating means deleting a page or a paragraph, delete it.
|
||||
5. Compact the work log when necessary; a work log holds laws and incidents, never bookkeeping.
|
||||
|
||||
#### THE WORK LOG IS ONE FILE.
|
||||
The work log is a single page in the knowledge base: `omp-docs/log.md` in this repo, or whichever change-log page the repo's docs index names. Write the entry when the change lands — in the same commit as the change — and read that page to recover where things stand before starting. It follows its own entry law: a dated entry is what changed plus the durable lesson, a few lines.
|
||||
|
||||
- NEVER create a second log: not per directory, not per project, feature, branch or run, not `NOTES.md` / `PROGRESS.md` / `STATUS.md` / `worklog/`, not a dated diary. In `dank/command-center` the KB linter reports any `log.md` that is not the bundle root's; the rule and its reasoning are owned by `omp-docs/references/okf-spec.md`.
|
||||
- A log holds laws and incidents, never bookkeeping: no per-file inventories, no run-by-run numbers, no transcripts. A number in a log still has to pass the purpose test above.
|
||||
- If a knowledge base has no log page, add one at its root and list it in the index. That is the fix — never a log somewhere else.
|
||||
|
||||
#### USE PROGRESSIVE DISCLOSURE.
|
||||
Always ensure that all produced documentation is laid out in files following WIKI conventions: start with a top level README.md index file that provides a brief, concise introduction to the item being documented, and then SPLIT OFF individual aspects, features, or arguments into their own interlinked pages so that future agents can read the documentation in a piecewise fashion, only taking in the details they need. Where relevant, crosslink any created documentation with other relevant documentation. This structure may also be nested, if necessary.
|
||||
@@ -90,12 +104,24 @@ Never assume the user reports correct information. He is human, and humans make
|
||||
- Before starting new work, `git pull` from the remote, and start a new branch in a separate worktree using `master` or `main` as the base branch.
|
||||
- Commit atomically: after every change that forms a complete functional unit, commit. Do so early and often.
|
||||
- When the work is complete, ask the user if they would like to open a pull request.
|
||||
- **Report a pull request as link AND title, every time.** A bare `#54` is not a reference: numbers are per-repository, so a number alone does not say which work, in which repo, is being reported. Give the PR's web URL on that forge — on the Dankclaw Gitea that is `https://gitea.franksdank.cloud/<owner>/<repo>/pulls/N` — together with the PR's title. Applies everywhere a PR is mentioned: chat replies, status summaries, issue and commit bodies, alerts. Another forge (GitHub, GitLab, …): the equivalent URL its web UI serves, same pairing.
|
||||
|
||||
### 5. Test Driven Development
|
||||
|
||||
- Always practice Test Driven Development practices; do UNIT TESTS exist for the functionality about to be edited? Is there a LINTER? Is there a FORMATTER? Verify that they exist, and that they are configured as a pre-commit hook. If they do not exist, they MUST be written PRIOR to the code being written. Invoke `/skill:tdd` for preferred testing style. If tests do exist, ensure that they still match the planned changes. If you believe a PRE-EXISTING test must be changed prior to commencing work, ASK THE USER ABOUT IT.
|
||||
- Always ensure tests run on code changes after every commit, and all project tests run before every push using git hooks, EXCEPT in the event that the individual test in question takes an excessive amount of time. Always ask the user before disabling an individual due to running time or performance concerns.
|
||||
|
||||
|
||||
### 6. Keep the Todo List LIVE, Not Retrospective
|
||||
|
||||
The todo list is the user's cockpit view of your work. A list that only flips at the end is a history dump, not tracking - the user sees nothing while you work.
|
||||
|
||||
- Multi-step work: `todo init` BEFORE the first substantive step - the list must exist while the work is unseen, not after.
|
||||
- Batch the todo transition with the step it describes: `done` + `start` ride along in the same message as the first tool calls of the next step. NEVER defer flips until the whole bill of tasks is finished.
|
||||
- A task is `done` the moment its work is verified - not when the surrounding phase ends.
|
||||
- Blocked on external input (user answer, subagent, service)? `block` it with a reason so the stall is visible; `unblock` the instant it is actionable.
|
||||
- NEVER make a todo call the only call in a message - that is ceremony, not progress; pair every flip with real work.
|
||||
|
||||
## FORBIDDEN ACTS AND MISTAKES TO NOT REPEAT (DO NOT DO THESE):
|
||||
- Editing node_modules or similar import module directories. This is third party code that will be overwritten next time `npm update` or a similar command is run.
|
||||
- `git push` - unless directed, except if it's our local gitea instance on Dankclaw.
|
||||
|
||||
+215
-56
@@ -1,98 +1,177 @@
|
||||
providers:
|
||||
webSearchOrder:
|
||||
- searxng
|
||||
- perplexity
|
||||
- gemini
|
||||
- anthropic
|
||||
- codex
|
||||
- xai
|
||||
- zai
|
||||
- exa
|
||||
- tinyfish
|
||||
- jina
|
||||
- kagi
|
||||
- tavily
|
||||
- firecrawl
|
||||
- brave
|
||||
- kimi
|
||||
- parallel
|
||||
- synthetic
|
||||
- startpage
|
||||
- duckduckgo
|
||||
- ecosia
|
||||
- google
|
||||
- mojeek
|
||||
- public
|
||||
openrouterVariant: online
|
||||
fetch: auto
|
||||
maxInFlightRequests:
|
||||
llamacpp (dankserver): 3
|
||||
llamacpp (dankclaw): 2
|
||||
llamacpp (dankclaw-3090): 3
|
||||
modelRoles:
|
||||
default: llamacpp (dankclaw)/frank-default-large:medium
|
||||
smol: llamacpp (dankserver)/frank-default-small:xhigh
|
||||
slow: llamacpp (dankclaw)/frank-default-large:medium
|
||||
vision: llamacpp (dankclaw)/frank-default-large:xhigh
|
||||
plan: llamacpp (dankclaw)/frank-default-large:medium
|
||||
designer: llamacpp (dankclaw)/frank-default-large:medium
|
||||
commit: llamacpp (dankserver)/frank-default-small
|
||||
tiny: llamacpp (dankserver)/frank-default-small
|
||||
task: llamacpp (dankserver)/frank-default-small
|
||||
advisor: llamacpp (dankclaw)/frank-default-large:medium
|
||||
slow: llamacpp (dankclaw)/frank-dragoon:medium
|
||||
vision: llamacpp (dankclaw-3090)/frank-zealot:xhigh
|
||||
plan: llamacpp (dankclaw-3090)/frank-zealot:medium
|
||||
designer: llamacpp (dankclaw-3090)/frank-zealot:medium
|
||||
commit: llamacpp (dankclaw-3090)/frank-zealot:medium
|
||||
tiny: llamacpp (dankclaw-3090)/frank-zealot:off
|
||||
task: llamacpp (dankclaw-3090)/frank-zealot:off
|
||||
advisor: llamacpp (dankserver)/frank-archon:medium
|
||||
default: llamacpp (dankserver)/frank-archon:medium
|
||||
smol: llamacpp (dankclaw-3090)/frank-zealot:medium
|
||||
web: web/searxng
|
||||
speech: frank-tts (dankclaw)/frank-tts
|
||||
judge: openrouter/~typesafe/jev-latest:auto
|
||||
dictation: local/parakeet-tdt-0.6b-v3
|
||||
memory: llamacpp (dankclaw-3090)/frank-zealot:medium
|
||||
symbolPreset: nerd
|
||||
composer:
|
||||
shape: claude
|
||||
shape: band
|
||||
tokenRate: true
|
||||
theme:
|
||||
dark: dark-monokai
|
||||
dark: dark-gruvbox
|
||||
light: light
|
||||
setupVersion: 2
|
||||
browser:
|
||||
headless: false
|
||||
headless: true
|
||||
retry:
|
||||
fallbackChains:
|
||||
{}
|
||||
web:
|
||||
- web/kagi
|
||||
- web/perplexity
|
||||
- google-gemini-cli/gemini-2.5-flash
|
||||
- google-antigravity/gemini-2.5-flash
|
||||
- google/gemini-2.5-flash
|
||||
- anthropic/claude-haiku-4-5
|
||||
- openai-codex/gpt-5.6-luna
|
||||
- xai/grok-4.5
|
||||
- web/zai
|
||||
- web/exa
|
||||
- web/tinyfish
|
||||
- web/jina
|
||||
- web/tavily
|
||||
- web/firecrawl
|
||||
- web/brave
|
||||
- web/kimi
|
||||
- web/parallel
|
||||
- web/synthetic
|
||||
- web/startpage
|
||||
- web/duckduckgo
|
||||
- web/ecosia
|
||||
- web/google
|
||||
- web/mojeek
|
||||
- web/public
|
||||
- openai-codex/gpt-6-luna
|
||||
- openai-codex/gpt-5.6
|
||||
- openai-codex/gpt-5.5
|
||||
- xai-oauth/grok-4.5
|
||||
- web/ollama
|
||||
speech: []
|
||||
llamacpp (dankserver)/frank-archon:
|
||||
- llamacpp (dankclaw)/frank-dragoon
|
||||
maxRetries: 10
|
||||
searxng:
|
||||
endpoint: search.franksdank.cloud
|
||||
endpoint: http://192.168.0.143:8081
|
||||
exa:
|
||||
enabled: false
|
||||
memory:
|
||||
backend: local
|
||||
backend: sharpshooter
|
||||
autolearn:
|
||||
enabled: true
|
||||
autoContinue: true
|
||||
compaction:
|
||||
remoteEndpoint: http://192.168.0.143:7771/v1/chat/completions
|
||||
methodOrder:
|
||||
- shake
|
||||
- handoff
|
||||
- snapcompact
|
||||
- soft
|
||||
- handoff
|
||||
- remote
|
||||
idleThresholdTokens: 200000
|
||||
thresholdPercent: -1
|
||||
idleThresholdTokens: 100000
|
||||
thresholdPercent: 95
|
||||
idleEnabled: true
|
||||
thresholdTokens: 150000
|
||||
thresholdTokens: -1
|
||||
dropUseless: false
|
||||
remoteStreamingV2Enabled: true
|
||||
asyncEnabled: true
|
||||
idleTimeoutSeconds: 120
|
||||
enabled: true
|
||||
experimentalContextManagement: false
|
||||
interruptMode: immediate
|
||||
defaultThinkingLevel: medium
|
||||
defaultThinkingLevel: xhigh
|
||||
hideThinkingBlock: false
|
||||
proseOnlyThinking: true
|
||||
proseOnlyThinking: false
|
||||
terminal:
|
||||
showProgress: true
|
||||
disabledExtensions:
|
||||
[]
|
||||
disabledExtensions: []
|
||||
statusLine:
|
||||
separator: powerline
|
||||
preset: nerd
|
||||
contextLine: annotated
|
||||
transparent: false
|
||||
compactThinkingLevel: false
|
||||
preset: custom
|
||||
contextLine: embedded
|
||||
transparent: true
|
||||
compactThinkingLevel: true
|
||||
showHookStatus: true
|
||||
sessionAccent: true
|
||||
leftSegments:
|
||||
- session_name
|
||||
- vim
|
||||
- mode
|
||||
- hostname
|
||||
- path
|
||||
- git
|
||||
- pr
|
||||
rightSegments:
|
||||
- model
|
||||
- cost
|
||||
- context_pct
|
||||
- time_spent
|
||||
segmentOptions:
|
||||
model:
|
||||
showThinkingLevel: true
|
||||
path:
|
||||
abbreviate: true
|
||||
stripWorkPrefix: true
|
||||
maxLength: 120
|
||||
git:
|
||||
showBranch: false
|
||||
showStaged: true
|
||||
showUnstaged: true
|
||||
showUntracked: true
|
||||
tui:
|
||||
textSizing: true
|
||||
codexResetFireworks: false
|
||||
hyperlinks: auto
|
||||
tight: false
|
||||
tight: true
|
||||
resizeScrollback: rebuild
|
||||
titleSpinner: pulse
|
||||
display:
|
||||
shimmer: kitt
|
||||
smoothStreaming: true
|
||||
showTokenUsage: true
|
||||
pinnedAgents: collapsed
|
||||
subagentLivePreview: true
|
||||
task:
|
||||
showResolvedModelBadge: true
|
||||
maxConcurrency: 2
|
||||
agentModelOverrides:
|
||||
designer: llamacpp (dankclaw-3090)/frank-zealot
|
||||
librarian: llamacpp (dankclaw-3090)/frank-zealot
|
||||
reviewer: llamacpp (dankserver)/frank-archon
|
||||
scout: llamacpp (dankclaw-3090)/frank-zealot
|
||||
security-reviewer: llamacpp (dankclaw)/frank-dragoon
|
||||
sonic: llamacpp (dankclaw-3090)/frank-zealot
|
||||
task: llamacpp (dankserver)/frank-archon
|
||||
seance: llamacpp (dankclaw-3090)/frank-zealot
|
||||
eager: default
|
||||
enableLsp: true
|
||||
maxRuntimeMs: 1800000
|
||||
softRequestBudget: 120
|
||||
maxRecursionDepth: 1
|
||||
disabledAgents: []
|
||||
isolation:
|
||||
enabled: true
|
||||
completionProbe: false
|
||||
snapcompact:
|
||||
systemPrompt: none
|
||||
toolResults: false
|
||||
shape: auto
|
||||
extendedContext: false
|
||||
ttsr:
|
||||
contextMode: discard
|
||||
@@ -103,12 +182,12 @@ checkpoint:
|
||||
readLineNumbers: true
|
||||
startup:
|
||||
showSplash: true
|
||||
changelogMode: summary
|
||||
update:
|
||||
channel: stable
|
||||
features:
|
||||
unexpectedStopDetection: smart
|
||||
unexpectedStopDetection: mechanical
|
||||
stt:
|
||||
modelName: parakeet
|
||||
submitTrigger: say-submit
|
||||
enabled: true
|
||||
recap:
|
||||
@@ -116,4 +195,84 @@ recap:
|
||||
loop:
|
||||
mode: prompt
|
||||
steeringMode: one-at-a-time
|
||||
personality: default
|
||||
personality: default
|
||||
images:
|
||||
urls:
|
||||
backends:
|
||||
- provider-files
|
||||
- tailscale
|
||||
- cloudflared
|
||||
- litterbox
|
||||
edit:
|
||||
mode: hashline
|
||||
fuzzyThreshold: 0.9
|
||||
dev:
|
||||
autoqaConsent: granted
|
||||
astGrep:
|
||||
enabled: true
|
||||
vault:
|
||||
enabled: true
|
||||
tts:
|
||||
localVoice: bm_george
|
||||
speech:
|
||||
enabled: false
|
||||
mode: all
|
||||
enhanced: true
|
||||
voice: am_fenrir
|
||||
secrets:
|
||||
enabled: true
|
||||
read:
|
||||
renderMarkdown: true
|
||||
toolResultPreview: true
|
||||
defaultLimit: 1000
|
||||
summarize:
|
||||
prose: false
|
||||
lsp:
|
||||
formatOnWrite: true
|
||||
diagnosticsOnEdit: true
|
||||
autoResume: false
|
||||
speechgen:
|
||||
enabled: true
|
||||
live:
|
||||
voice: vale
|
||||
contextPromotion:
|
||||
enabled: true
|
||||
title:
|
||||
icons: emoji
|
||||
generator: tiny
|
||||
followUpMode: one-at-a-time
|
||||
tools:
|
||||
approvalMode: yolo
|
||||
format: auto
|
||||
maxTimeout: 600
|
||||
marketplace:
|
||||
autoUpdate: notify
|
||||
bash:
|
||||
allowCompoundCommands: true
|
||||
gitGuard: true
|
||||
bashInterceptor:
|
||||
enabled: false
|
||||
shellMinimizer:
|
||||
sourceOutlineLevel: default
|
||||
generate_image:
|
||||
enabled: false
|
||||
computer:
|
||||
enabled: true
|
||||
display: active
|
||||
ratchet:
|
||||
enabled: true
|
||||
github:
|
||||
enabled: true
|
||||
security:
|
||||
enabled: true
|
||||
todo:
|
||||
reminders: true
|
||||
mcp:
|
||||
notifications: false
|
||||
plan:
|
||||
defaultOnStartup: false
|
||||
isolation:
|
||||
backend: auto
|
||||
worktree:
|
||||
onStart: create
|
||||
onExit: keep
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
name: aa-chart-label-browser-extraction
|
||||
description: "Extract per-model benchmark scores from artificialanalysis.ai pages: charts are client-rendered recharts SVGs invisible to plain HTML fetch; pair sorted label/name x-coords, then cross-check against published sentences before trusting any chart↔eval identity."
|
||||
---
|
||||
|
||||
Extract per-eval scores for a model from artificialanalysis.ai when no API key exists. Plain HTML fetch yields NO chart data (client-side recharts; SSR has only meta + FAQ sentences; `/api/*` = 401).
|
||||
|
||||
## Procedure
|
||||
1. Managed browser tab: `browser.open({name, url, wait_until: "domcontentloaded"})`.
|
||||
2. **Per-eval top-3 (authoritative):** open `https://artificialanalysis.ai/evaluations/<slug>`; regex `document.body.innerText` (collapse whitespace): `([A-Z][^.;]{0,60}?) scores the highest on (.+?) with a score of (-?[\d.]+%?), followed by (.+?) with a score of ... and ...` — mind decimal points truncating naive `[^.]+` regexes. Build the page script as a plain string (`String.raw`), not an interpolated template, so `\\d` survives.
|
||||
3. **All-eval per-model rows:** on `https://artificialanalysis.ai/models/<slug>`: scroll in 6 steps (0.15→0.9, ~1.3 s each) to trigger lazy charts, then per `svg` with ≥3 bars: labels = `[...svg.querySelectorAll("g.recharts-label-list text")]` (x attr), names = `[...svg.querySelectorAll("foreignObject")]` divs sorted by `g[transform] translate(x,y)`. Sort BOTH by x, **zip by index** — x-center pairing is off by a systematic half-bar.
|
||||
4. Self-row filter: exact hyphenated name (e.g. `Qwen3.8-Flash-Next`, not "Qwen3.8 Flash").
|
||||
5. Elo evals (GDPval-AA, AA-Briefcase): charts show AA's published `(Elo−500)/2000` normalization (1866→68.3).
|
||||
|
||||
## Verification law (non-negotiable — two plausible mappings disagreed twice)
|
||||
- Zip top-3 of every chart against the eval page's published sentence (e.g. Kimi 88.7→89, Opus 66.9→67); ship only cross-checked rows.
|
||||
- Arithmetic corroboration helps: Omniscience accuracy 17% + non-hallucination 7% ↔ index −60.
|
||||
- Record provenance per row: sentence rows → eval-page URL; chart-label rows → model-page URL; `fetched_at` + `index_version`.
|
||||
- Geometry pairing (link→nearest chart below) is AMBIGUOUS when chart height > gap to next card — adjacent links share a chart. Use order + sentence tops, not geometry.
|
||||
|
||||
## Traps
|
||||
- `tab.run` = Bun runtime; page-world JS needs `tab.evaluate`. Eval-cell `log()` does not reach display; write to a /tmp file and `cat`.
|
||||
- "Add model from specific provider" picker is not in the DOM (renders after chart data loads); UI drill-through × N evals is fragile — prefer the sentences.
|
||||
- Headline index FAQ: "scores N on the Artificial Analysis Intelligence Index" — check for "(estimated)" qualifier; est. = "independent evaluation forthcoming".
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
name: aa-chart-model-score-extraction
|
||||
description: "Extract per-model benchmark scores from artificialanalysis.ai pages: charts are client-rendered Recharts SVGs; zip label-list texts to foreignObject names by x-sorted index, validate chart identity against the page's published top-3 sentence, apply Elo/Omniscience normalization laws, seed frank-intelligence-bench aa-scores.yml."
|
||||
---
|
||||
|
||||
Extract AI benchmark scores for a specific model from artificialanalysis.ai. Per-model per-eval numbers are NOT in SSR HTML (JS charts, API needs key); eval-page SSR text DOES carry the top-3 sentence.
|
||||
|
||||
## Procedure (validated 5x on this repo's model pages)
|
||||
1. `browser.open({ name, url: "https://artificialanalysis.ai/models/<slug>", wait_until: "domcontentloaded" })`, then scroll in ~6 steps (0.15..0.9 of scrollHeight, ~1.3 s pauses) — charts lazy-render.
|
||||
2. Extract each chart: pair `g.recharts-label-list text` (values, take `x` attr) with `foreignObject div` (model names, x from parent `g[transform] translate(x,y)`). **Sort both lists by x and zip by index — NOT nearest-x** (nearest-x is off by one bar; index-zip matches published tops exactly).
|
||||
3. Self row: match name loosely (`Qwen3.8-Flash-Next` has hyphens — test both forms).
|
||||
|
||||
## Identity law (never ship an order-guessed chart→eval mapping)
|
||||
- Model pages have ~14 charts, 18 eval cards; DOM order and geometry pairing BOTH proved ambiguous.
|
||||
- Anchor every chart by its top-3 names/values vs the eval page's published sentence: `https://artificialanalysis.ai/evaluations/<slug>` body text: "X scores the highest on <eval> with a score of N%, followed by …". Regex must tolerate decimals: value pattern `(-?[\\d.]+%?)`; build page scripts with `String.raw` (bun template literals eat `\\d`).
|
||||
- A chart's `self` value on the MODEL page is page-local → trustworthy once the top-3 anchor confirms the eval (e.g. AA-LCR: Kimi 88.7 → chart top "Kimi K3 (max)=89").
|
||||
|
||||
## Normalization laws (frank-intelligence-bench aa-scores.yml)
|
||||
- Elo evals (AA-Briefcase, GDPval-AA): store `(Elo−500)/2000×100` (1866 → 68.3), AA's published normalization.
|
||||
- AA-Omniscience: score = Accuracy (w10) + 1−Hallucination (w5), both 0–100 → composite `(10·acc + 5·nonhall)/15`. NEVER feed the −100..100 Index chart value (pre-fix cost ~7 pts; calibration then reproduces AA's official index ±0.6: Flash-Next 39.85≈40, Haiku 43.4≈43, 27B 33.85≈34, 3.6-27B 21.55≈21).
|
||||
- Every seeded row: `source_url` + `fetched_at` + `index_version`.
|
||||
|
||||
## Seed + validate loop
|
||||
Append model block to `config/references/aa-scores.yml` (block-insert via python string replace on the ` - name: <anchor>` line — beware: dropping rows from a block loses the 2-space indent on the anchor line; verify with `yaml.safe_load`), then `bun -e loadAaRefs` parse-check, `bun src/cli.ts report $RID`, calibration check (full-row models: lite vs AA official), `bun test`, commit.
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
name: aa-reference-scores-seeding
|
||||
description: "Seed/refresh frank-intelligence-bench config/references/aa-scores.yml from artificialanalysis.ai: authoritative top-3 sentences from eval pages, Recharts SVG zip extraction for per-model chart values, cross-check law, Elo normalization, schema/repair loop."
|
||||
---
|
||||
|
||||
Seed frank-intelligence-bench `config/references/aa-scores.yml` from AA's public pages, then re-render the report trio.
|
||||
|
||||
## Provenance law (non-negotiable)
|
||||
- Every row: `{ eval, pct, source_url, fetched_at, index_version }` (Zod-enforced; top-level `schema_version: 1, fetched_at, index_version` required).
|
||||
- Eval ids MUST match report result ids: `scicode, aa-lcr, hle, terminal-bench, gdp-pdf, aa-briefcase, gdpval-aa, automationbench-aa, aa-omniscience, critpt` (render `refFor(evalId)` joins on exact id).
|
||||
- Never ship an order-guessed chart↔eval mapping; only cross-checked ones.
|
||||
|
||||
## Data tiers (authority order)
|
||||
1. **Eval-page sentence** (authoritative, SSR, per eval page `/evaluations/<slug>`): `"X scores the highest on <eval> with a score of N%, followed by Y ... and Z ..."` → top-3 rows with verbatim model names. Regex trap: model names contain `5.5` — capture names with `[^.;]` truncates at decimals; use targeted extraction per page, verify each value against the page's `66.9`-style raw text.
|
||||
2. **Model-page chart labels** (per-model values, incl. the page's own model): charts are client-side Recharts (NOT in raw HTML, NOT via /api/ — keyless API 404s). Browser: `browser.open({name,url,wait_until})`, scroll full page in ~0.15 steps (lazy render), then `tab.evaluate` (NOT `tab.run` — that's Bun world):
|
||||
- names = `<foreignObject>` div text, x from parent `g[transform]` `translate(x,...)`;
|
||||
- values = `g.recharts-label-list text` attrs;
|
||||
- **zip by sorted-x index** (nearest-x pairing is off by half a bar; index-zip verified against published tops).
|
||||
3. `sync-aa` free Data API for headline indices — needs `AA_API_KEY` (unset in ~/.secrets as of 2026-10-11).
|
||||
|
||||
## Cross-check law (mandatory before shipping chart-derived rows)
|
||||
Chart's top-3 zipped values must match the eval page's published sentence (e.g. Kimi 88.7→89, Elo 1866→68.3, Opus index 46). Two independent methods disagreeing = stop, re-anchor; geometry pairing (link-top→svg-top proximity) collides on adjacent cards — prefer sentence-vs-zip corroboration.
|
||||
|
||||
## Normalizations
|
||||
- Elo evals (GDPval-AA, AA-Briefcase): sentences publish raw Elo; store AA's published `(Elo−500)/2000`×100 (1866→68.3). Document in file header.
|
||||
- AA-Omniscience `eval: aa-omniscience` = the −100..100 Index (not accuracy); accuracy/non-hallucination charts corroborate (17%/7% ⇒ −60 arithmetically consistent).
|
||||
|
||||
## Known identities (2026-10-11 census, v4.3.2)
|
||||
- frank-archon lineage base = **Qwen3.8-Flash-Next** (`/models/qwen3-8-flash-next`), AA index 40, class median 18. NOT Qwen3-Next-80B-A3B-Instruct (index 10, 2025 family) — first seed got this wrong; verify lineage in `docs/endpoints.md` before labeling `kind: base`.
|
||||
- Base per-eval (chart): SciCode 51, HLE 38, AA-LCR 80, TB-4.0 25, GDP.pdf 16, CritPt 11, Briefcase 54, GDPval 56, AutoBench 56, Omniscience −10.
|
||||
- Frontier anchors: SciCode Opus 5.5 Max 66.9 (board flagged "under review" — travel this caveat with any SciCode delta); HLE Opus 61.4; AA-LCR Kimi K3 88.7; TB Sonnet 63.6; CritPt GPT-6.1 Sol 32.3.
|
||||
|
||||
## Edit traps
|
||||
- YAML surgery via `edit`: replacing a line range that included a top-level key (`schema_version`/`models:`) silently duplicates or deletes the anchor → Zod error `expected 1 / received undefined`. After any header edit, `grep -n "^schema_version\|^fetched_at\|^index_version\|^models:"` must show exactly one each.
|
||||
|
||||
## Ship loop
|
||||
1. `bun -e 'import { loadAaFile, loadAaRefs } from "./src/aa"; ...'` schema-parse gate.
|
||||
2. `bun src/cli.ts report <run-id>` (takes bare id, NOT a path) → trio in `reports/<host>/<id>/`.
|
||||
3. Verify `data.json .aa_reference` non-empty, per-eval coverage, delta columns.
|
||||
4. log entry in `docs/log.md` + atomic commit + push (local gitea OK).
|
||||
5. `browser.close({name})` — headless tabs are shared/retained.
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
name: amd-apu-vulkan-tenant-onboarding
|
||||
description: "Add a ggml/Vulkan inference tenant to an AMD APU (gfx1151 / Strix Halo iGPU) that already serves another engine: APU memory census, RADV heap gate, render-node resolution, build-side glslc/ICD deps, co-tenancy measurement."
|
||||
---
|
||||
|
||||
Use when a ggml-based engine (llama.cpp, qwentts.cpp, anything built on ggml) must move from CPU to the **iGPU** of an AMD APU (gfx1151 / Strix Halo / Ryzen AI Max) that **already serves another engine on the same card**. Answers two questions only a booted host can: what gate the second tenant must clear, and which files the build needs to clear it.
|
||||
|
||||
## 1. The usual VRAM reasoning is wrong on an APU
|
||||
|
||||
`/sys/class/drm/cardN/device/mem_info_vram_total` reports a small **carve-out** (2 GiB class), and `rocm-smi`'s "dedicated" figure is not the working set either. The real working set lives in **GTT / system pages**: `/proc/meminfo` → `GTT` and `System pages`. A second GPU tenant therefore consumes host RAM from the same pool the first tenant's weights and the OS use — no supplementary groups, no separate aperture to reserve.
|
||||
|
||||
- Price the tenant from a **booted** footprint, never from `weights + KV + buffers`: on an APU the compute buffers and any projector ride on GTT, which is host RAM; check `free` *after* the load, not before.
|
||||
- Never subtract a "dedicated" figure from it to find headroom.
|
||||
- Unheld fabric clock (`pp_dpm_mech` = `auto`) is a hazard only when the NPU joins too; a pure iGPU pair is unaffected — check it before adding a third tenant.
|
||||
- Swap on a host with GPU tenants protects co-tenants; paging a KV pool is worse than the OOM it prevents.
|
||||
|
||||
## 2. The gate that decides feasibility: the RADV heap a container sees
|
||||
|
||||
`rocm-smi` and `free` cannot see this number; only a container off the **engine's own base image** can:
|
||||
|
||||
```bash
|
||||
# one-off container, no model, no load, no compute — safe beside production
|
||||
podman run --rm --device /dev/dri --group-add keep-groups <engine-base> bash -c \
|
||||
'apt-get update -qq >/dev/null 2>&1; apt-get install -y -qq --no-install-recommends mesa-vulkan-drivers vulkan-tools glslang-tools >/dev/null 2>&1; vulkaninfo --text'
|
||||
```
|
||||
|
||||
Read `VkPhysicalDeviceMemoryProperties` → the `DEVICE_LOCAL` heap size. On a hybrid Strix Halo box Mesa splits the aperture (~1/3 : 2/3), so `DEVICE_LOCAL` is the budget the ggml tool will actually allocate against — measured 77.33 GiB *with the first tenant loaded*, against a 2 GiB DRM carve-out. Two traps, both measured:
|
||||
|
||||
- `vulkaninfo --batch` does not exist in Ubuntu 24.04's `vulkan-tools` (prints help, exits 1). Every `vulkaninfo` parse needs `--text`.
|
||||
- The container may have **no NVIDIA ICD**, so Vulkan index `0` *is* the iGPU — but do not trust ordering by index: `--text` and read `deviceName` + `driverID` (`RADV_GFX1151` vs `NVIDIA`). A hand-picked `/dev/dri/renderDN` is worse than useless on a hybrid host: **`renderD128` there is the NVIDIA card and `renderD129` the amdgpu iGPU**; resolve the node with `readlink -f /sys/class/drm/renderD*/device/driver` and pass `/dev/dri` wholesale, letting the driver pick by name.
|
||||
- A rootless container that drops its supplementary groups enumerates **nothing**. `--group-add keep-groups` (quadlet: `GroupAdd=keep-groups`) plus the host udev rule that makes the render nodes group-writable is the whole requirement; `--device /dev/dri` alone is not enough.
|
||||
|
||||
## 3. Build-side deps: the configure fails before the device ever matters
|
||||
|
||||
ggml's `src/ggml-vulkan/CMakeLists.txt` requires, in order: `find_package(Vulkan COMPONENTS glslc REQUIRED)` then `find_package(SPIRV-Headers CONFIG REQUIRED)`, and it compiles its shaders with **glslc**, never `glslangValidator`. On Ubuntu 24.04 (measured against the digest-pinned `ubuntu:24.04` base, 2026-10-19):
|
||||
|
||||
- `glslang-tools` ships `glslangValidator` and **NOT `glslc`** → `command -v glslc` empty, `find_package(Vulkan … glslc REQUIRED)` fails, and the Vulkan backend silently never exists.
|
||||
- The package is the standalone **`glslc`** (universe, `2023.8-1build1`). Working build-stage set: `libvulkan-dev glslc spirv-headers` — `libvulkan-dev` provides `vulkan/vulkan.h` + the `VulkanHeaders`/`VulkanLoader` CMake configs `find_package(Vulkan)` resolves to, `spirv-headers` ships `/usr/share/cmake/SPIRV-Headers/SPIRV-HeadersConfig.cmake`. A configure probe in the pinned base prints `Found Vulkan: … found components: glslc` then `Including Vulkan backend`.
|
||||
- **The archive's glslc is old.** 2023.8 compiles `GL_KHR_cooperative_matrix` (the path ggml's vectored quantized matmul needs) but reports `cooperative_matrix2`, `integer_dot_product`, `bfloat16`, `float_e2m1/e4m3` as unsupported, so those shader families are compiled out. If a Vulkan prefill reads as inexplicably slow, suspect the **shader compiler before the device**; the fix is a pinned glslang build passed as `-DVulkan_GLSLC_EXECUTABLE=<path>`, not a different runtime.
|
||||
- Runtime stage needs the loader **and an ICD**: `libvulkan1` + `mesa-vulkan-drivers`. Loader-without-ICD enumerates zero devices, and ggml-based servers die in device lookup — often *after* reading the weights, so the journal reads like a model fault.
|
||||
- Ship **no NVIDIA ICD** on purpose: one Vulkan device in the enumeration is what makes the *name* `Vulkan0` safe to write into config. Prefer the name over an index in every flag.
|
||||
|
||||
## 4. Prove enumeration without allocating anything
|
||||
|
||||
Do not boot a scratch second engine beside production to ask which devices exist — a second model load is a real GPU tenant, and fleet law stops that. Instead run the binary with a device name that cannot exist and read its error; it names what it enumerated, with no allocation and no model resident:
|
||||
|
||||
```bash
|
||||
podman run --rm --device /dev/dri --group-add keep-groups --entrypoint /app/tts-server <image> \
|
||||
--model /dev/null --codec /dev/null --device NoSuchDevice 2>&1 | tail -20
|
||||
```
|
||||
|
||||
## 5. The rollback must be a flag, not a rebuild
|
||||
|
||||
Put the device in the env file as a value the entrypoint passes only when set, and make **set-but-empty** mean "pass no flag at all" — the binary's own default (CPU, always built) is the rollback. `: "${VAR:=default}"` is the WRONG form: `:=` fires on an empty value too, so the documented rollback line `VAR=` is silently rewritten back to the GPU and the only way out is a rebuild. Use:
|
||||
|
||||
```bash
|
||||
if [[ ! -v QWENTTS_DEVICE ]]; then QWENTTS_DEVICE=Vulkan0; fi
|
||||
```
|
||||
|
||||
(`podman`'s `--env-file` passes `VAR=` through as set-and-empty, and `source` does the same, so both readers agree.) Pin it in a test: empty device ⇒ the flag is absent from argv.
|
||||
|
||||
## 6. Co-tenancy measurement (only after the tenant is live)
|
||||
|
||||
The gate is the **first tenant's** performance with a real request from the second in flight — not the second tenant's own speed. And price the tenant from a booted footprint, never from weights+KV+buffers: on an APU the compute buffers and any projector ride on GTT, which is host RAM; check `free` after the load, not before.
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
name: audit-never-run-container-deploy
|
||||
description: "Audit a locally built podman/quadlet deploy path that has never actually run: prove base-image assumptions, env-file reader divergence, and image-hash coverage before the first deploy."
|
||||
---
|
||||
|
||||
# Audit a never-run container deploy path
|
||||
|
||||
A deployable that has never been built or started has never been *executed*: every
|
||||
fault in it is invisible to tests that stub the runtime. Audit it as static code
|
||||
against the tools it calls, then prove each suspicion cheaply against the real
|
||||
tool. The faults worth hunting are the ones that **report success while doing
|
||||
nothing** — those are the ones that survive a green `/health` and a green e2e run.
|
||||
|
||||
## The three fault classes (all measured 2026-10-18, frank-tts-engine)
|
||||
|
||||
1. **A command the base image does not ship.** e.g. `adduser` in a `RUN` line
|
||||
against a trimmed `ubuntu:24.04` — absent, and not pulled in by
|
||||
`ca-certificates curl libgomp1`. Dies at build time (loud), but only the first
|
||||
time anyone builds it.
|
||||
Prove it in the base, never from memory:
|
||||
`podman run --rm <base> sh -c "command -v adduser || echo absent; useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts && id tts"`
|
||||
Note the accepted cosmetic warning (`uid 10000 is greater than SYS_UID_MAX 999`)
|
||||
in a comment, or a future agent "fixes" it by dropping `--system` and the uid
|
||||
stops matching the sibling container.
|
||||
2. **Two readers of one env file disagree.** systemd units that pass
|
||||
`EnvironmentFile=` hand it to podman `--env-file`, which does **no shell
|
||||
parsing** — it takes the literal remainder of the line — while bash `source`
|
||||
strips one quoting layer. So `WARM_TEXT="Warm up, one two three."` reaches the
|
||||
container *with its quotes*, and a script that interpolates it into a JSON body
|
||||
produces invalid JSON: the container boots, `active`, every voice registered,
|
||||
every voice cold. Fix in the consumer: strip exactly one matched pair **only**
|
||||
when both ends are the same quote AND the inner content carries another quote or
|
||||
backslash (never mangle a value that legitimately contains quotes), then escape.
|
||||
3. **A build-skip hash that under-covers the image.** `build_compose_image`-style
|
||||
helpers skip `podman build` when a label hash matches; hashing the `Containerfile`
|
||||
alone means editing a `COPY`'d app/script rebuilds **nothing** — deploy reports
|
||||
success, `/health` answers, container serves old code. Each case arm must name
|
||||
the paths its `Containerfile` copies (`extra=(…)`) and hash Containerfile+extras.
|
||||
Arms that have no extras keep their old hash, so they do not needlessly rebuild.
|
||||
|
||||
## Test it as behaviour, never as a grep of the handler
|
||||
|
||||
- Drive the **real** function: `awk '/^build_compose_image\(\) \{/,/^\}/' deploy-remote.sh > bci.sh`,
|
||||
`source` it, put a stub `podman` first on `PATH`, assert the observable question —
|
||||
*did the deploy ask podman to build, or did it skip?*
|
||||
- Stub traps that cost real time: podman is invoked `podman image inspect <image> --format …`
|
||||
so the tag is **`$3`**, not `$2`; tags contain `/` so a state file named after one
|
||||
needs `${tag//\//_}` or the write silently lands in a nonexistent directory and
|
||||
every image looks permanently unbuilt; `--label cfhash=…` arrives **before** `-t <tag>`,
|
||||
so collect the label and write it after the arg loop.
|
||||
- For an env-file bug, write one test that reads the file **both** ways — `source`
|
||||
and the `--env-file` way (parse `KEY=literal remainder`) — and assert the program
|
||||
receives the same value in both shapes, including a value that carries its own
|
||||
quotes and a backslash. A suite that only sources the file is how the bug shipped.
|
||||
- Prove red: `git show HEAD:scripts/deploy-remote.sh > scripts/deploy-remote.sh`,
|
||||
run the new test (must fail at the exact intended step), restore. Keep each fix in
|
||||
its own commit and run the seam at each one so the branch bisects.
|
||||
|
||||
## Base-image and host facts to check while auditing
|
||||
|
||||
- `useradd` lives in `login(1)` (base carries it); `adduser` does not.
|
||||
- Rootless podman **cannot raise rlimits** — `--ulimit nofile=…` above the host hard
|
||||
limit dies at container create, not at runtime.
|
||||
- A referenced `Network=` `.network` file the generator cannot find means the
|
||||
container unit is **not produced at all**.
|
||||
- Two containers talk by DNS name only if they attach to a **common** network;
|
||||
check both unit files rather than trusting a comment that says they share one.
|
||||
- Container DNS name + the network that owns it, never `host.containers.internal`
|
||||
or a published loopback port, for service-to-service traffic.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not deploy the audited thing because the audit is finished: first build of a
|
||||
heavy image (recursive source clone ~10 min) plus a new unit on a host serving
|
||||
people is the operator's call.
|
||||
- Do not add a Prometheus target for a container with no `/metrics` — say the probe
|
||||
is a decision (blackbox HTTP), do not fake a scrape.
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
name: audit-subagent-deliverables
|
||||
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), adversarial-review-report triage, resource-safety pass (fd/memory), and cutover with running-process restarts."
|
||||
---
|
||||
|
||||
# Audit subagent deliverables
|
||||
|
||||
A subagent that wrote code is a source of plausible output, not a verified change. Integrate it the way you'd integrate an untrusted PR — with extra suspicion, because the author is an agent that shares your blind spots and its prose is more confident than its evidence.
|
||||
|
||||
Order of operations: read everything → gates → tests (as evidence, not proof) → resource-safety pass → fix in-place → cutover.
|
||||
|
||||
## 1. Read every produced file, line by line
|
||||
Full file, not a diff skim. You are the reviewer who must catch the line the author never thought about.
|
||||
|
||||
## 2. Mechanical gates before logic review
|
||||
```bash
|
||||
bash -n <script> # syntax (scripts)
|
||||
bunx tsc --noEmit # types (TS)
|
||||
bun test # suite
|
||||
```
|
||||
Run the suite for information, not as the stopping point — it only proves the author's own scenarios.
|
||||
|
||||
## 3. Hunt the three classic agent defects
|
||||
- **Quoting bugs:** `set -e` + `"${@/$p/$r}"` = substitution regex containing user input (bash can't escape `/` in a `${var/re/pat}` pattern: `/` reads as the pattern/replacement separator). Fix with a separator absent from inputs (e.g. `%`) and `printf '%q'` for shell-interpolated values.
|
||||
- **`cd` assumptions:** parallel `runShell` calls share one CWD (pool mode). Any command sequence that assumes CWD (e.g. `git -C .` after a sibling's `cd`) must use explicit absolute paths.
|
||||
- **Unbounded resources:** every fd opened must be closed on ALL paths (success, throw, validation fail) — `using` for fs.sinks (closes on throw; sinks are `AsyncDisposable`, not `fs.FileHandle`, no `.close()`); bounded `Map` for caches (`MAX` + FIFO eviction); explicit caps (lines read, file sizes).
|
||||
|
||||
## 4. Check the test file is real
|
||||
Tests that assert internal state (`expect(wcwd).toBe(...)`) over output contracts; tests of mock echo; tautologies. Delete them per house rules, keep behavior contracts.
|
||||
|
||||
## 5. Cutover audit (repo-wide)
|
||||
`grep -r` the symbol namespace the subagent touched — `wcwd`-style globals are the classic blast radius. Grep every `cd`/`process.chdir` site and verify each paired command sequence uses explicit paths. Stale comments describing the old design must go — comments that lie are time bombs for the next reader.
|
||||
|
||||
## 6. Cutover restarts running processes
|
||||
A code fix on disk is NOT live in any process that loaded the file at spawn (bun/node load-at-spawn). After shipping fixes: restart every long-running supervisor/daemon/UI-server that carries the affected code path; prefer takeover protocols (stale-pid detection) that make restarts cheap. Symptom of forgetting: "fixed" code producing old-code symptoms in production (observed: 812 poisoned ledger rows).
|
||||
|
||||
## 7. Triaging an adversarial REVIEW report (not code)
|
||||
When a subagent produced findings, treat each finding as a hypothesis with a base rate ~40% (measured over 3 rounds, 2026-10):
|
||||
1. **Walk every finding end-to-end against the CURRENT tree** before accepting — cite file:line yourself. Reviewers frequently read a mid-edit snapshot; many findings are stale by construction.
|
||||
2. **Classification that kills most findings:**
|
||||
- *Stale read* — code changed during review; verify against live lines.
|
||||
- *Inverted logic* — "failed rows are skipped" when the filter makes failed rows RETRY. Re-read the filter.
|
||||
- *Unreachable* — trigger requires breaking a stated design law (single-worker, append-only). Not a defect.
|
||||
- *By-design* — the reviewer labels a spec law a bug (serial execution, non-stream client). Check the spec, not the reviewer's confidence.
|
||||
- *Constant-trust* — "XSS via manifest field" where the field is git-pinned config; "path traversal via REMOTE_TMP" where it's a hardcoded const. Ask: what untrusted input reaches this sink?
|
||||
- *Self-refuting* — the reviewer's own probe output disproves its claim (raw==dedup probe in a "dedup" finding). Quote it back.
|
||||
3. **Acceptance bar:** concrete trigger with real inputs + full path walked. Real accepts cluster in: resource caps (pids/timeout/awaited-cleanup), honest labels (default signatures), escaping discipline, self-healing probes.
|
||||
4. **One-line verdicts + evidence per finding**, shipped in the commit message; calibration section ("genuinely solid") accepted as free verification of past fixes.
|
||||
|
||||
## Verification (the non-negotiable)
|
||||
Build/run the real artifact with its real inputs. Observe behavior at the seams:
|
||||
- concurrent calls → no CWD races;
|
||||
- resource ceilings → caps enforced;
|
||||
- cleanup → no fd/pid/dir leaks;
|
||||
- security → injection probes inert (quotes, slashes, metachars);
|
||||
- normal case → output identical to pre-audit.
|
||||
|
||||
## Evidence (2026-10-07 + 2026-10-10/11 sessions)
|
||||
- Shell-engine subagent: 426 LOC, `bun test` 39/39 green. Line-audit found: 5 leaked fds per call (`fd` sink never closed), CWD-race in `git -C .`, `set -e` + `${@/\/g\//}` slash-escaping bug, unbounded 10k-entry cache, 45s timeout on git-status.
|
||||
- Review-triage census over 3 blind adversarial rounds (same repo, ~41 findings): accepted 9, rejected ~32 — rejects dominated by stale reads (protocol rewritten mid-review), inverted filter logic, unreachable-under-law triggers, base64-alphabet "quote injection" (charset has no quotes), SIGPIPE non-leaks, "no memory cap" quoting the `--memory 4g` line above the claim. Real accepts: fork-bomb pids cap, stalled-ssh upload cap, stale-flag self-heal probe, honest census label, freshness gate on pid reuse.
|
||||
- Poison-law incident: sandbox fix shipped while supervisor ran old build → 812 rows graded by the dead protocol; takeover made recovery cheap. The restart step (#6) exists because of this.
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
name: authentik-provision-fleet-user
|
||||
description: "Provision or audit an Authentik user on Dankclaw (auth.franksdank.cloud): ak shell ORM drive, group set, behavioral login + entitlement verify, docs census record, stale-claim caveat."
|
||||
---
|
||||
|
||||
# Provision an Authentik user on Dankclaw
|
||||
|
||||
Source of truth: `serverAppConfigs-docs/Infrastructure/Authentik.md` ("User provisioning playbook"). Authentik objects are NOT repo-managed — no deploy, no file edits.
|
||||
|
||||
1. Inspect first, mirror the newest human user's groups:
|
||||
```bash
|
||||
ssh dankclaw 'podman exec -i authentik-server ak shell' <<'EOF' 2>&1 | sed -n '/objects imported/,$p'
|
||||
from authentik.core.models import User, Group
|
||||
for u in User.objects.all().order_by("username"):
|
||||
print(u.username, u.email, sorted(u.groups.values_list("name", flat=True)))
|
||||
print(sorted(Group.objects.values_list("name", flat=True)))
|
||||
EOF
|
||||
```
|
||||
2. Create (idempotent guard; password goes in the heredoc = stdin, never argv, so it never lands in `ps`):
|
||||
```python
|
||||
from authentik.core.models import User, Group
|
||||
u = User.objects.filter(username="X").first()
|
||||
if not u:
|
||||
u = User(username="X", email="X@franksdank.cloud", name="X", is_active=True)
|
||||
u.set_password("...")
|
||||
u.save()
|
||||
for g in ("Non-Admin Users", "jellyfin_users", "openwebui_users"): # human-user precedent: ridley
|
||||
u.groups.add(Group.objects.get(name=g))
|
||||
```
|
||||
3. Verify through the exact PasswordStage code path:
|
||||
```python
|
||||
from django.contrib.auth import authenticate
|
||||
assert authenticate(request=None, username="X", password="...")
|
||||
assert authenticate(request=None, username="X", password="nope") is None
|
||||
```
|
||||
4. Verify the downstream claims resolve — `app_entitlements` needs an `Application`
|
||||
instance (look it up by slug, `open-webui`), and compare against the precedent user:
|
||||
```python
|
||||
from authentik.core.models import Application, User
|
||||
app = Application.objects.get(slug="open-webui")
|
||||
for n in ("ridley", "X"):
|
||||
u = User.objects.get(username=n)
|
||||
print(n, sorted(e.name for e in u.app_entitlements(app)), u.is_superuser, u.path)
|
||||
```
|
||||
5. Stale-claim check + docs record (repo law: the vault is the source of truth):
|
||||
```bash
|
||||
ssh dankclaw "podman exec authentik-postgres psql -U authentik -d authentik -tAc \
|
||||
\"SELECT count(*) FROM authentik_providers_proxy_proxysession WHERE session_data->'claims'->>'preferred_username' IN ('X')\""
|
||||
```
|
||||
Then commit two doc edits (NEVER write the password into any file): the census
|
||||
sentence in `Infrastructure/Authentik.md` (human-user template bullet — append the
|
||||
new `pk`s + creation date) and a dated entry in `serverAppConfigs-docs/log.md`
|
||||
(what was created, groups, how it was verified, proxysession count, downstream lag).
|
||||
`./tests/run.sh` must pass before commit (docs completeness + link-integrity seams).
|
||||
|
||||
## Rules
|
||||
- No `User.create_user`; `is_superuser` is a group-derived property, not a constructor kwarg (raises `TypeError`). `path` defaults to `users`.
|
||||
- Guarded-vhost access = any authenticated user (all proxy providers use implicit-consent); groups are consumed downstream: `homepage-admins` (Caddy `/admin` header gate), `openwebui_users`/`_admins` (roles claim), `jellyfin_users`/`_admins` (LDAP outpost). EXCLUDE admin groups from a plain user; ask before granting `homepage-admins`.
|
||||
- Group add after a login → Caddy RBAC can 403 on the outpost's cached claims; purge that user's rows: `DELETE FROM authentik_providers_proxy_proxysession WHERE session_data->'claims'->>'preferred_username' = 'X';` (in the `authentik-postgres` container) — next visit re-authorizes silently. Groups set BEFORE any login → nothing to purge (count is 0).
|
||||
- Jellyfin visibility rides the LDAP outpost cache; expect a lag. OpenWebUI mints a first-time SSO login as role `pending`.
|
||||
- Look up flows by slug, not name. Do not rotate `AUTHENTIK_SECRET_KEY` to fix claim issues.
|
||||
@@ -0,0 +1,113 @@
|
||||
---
|
||||
name: authentik-rp-sso-login-diagnosis
|
||||
description: "Diagnose/repair a dead OAuth2 relying-party login against authentik (Grafana-style): authorize 500, missing email claim, Grafana /emails fallback. Live probe harness + ak-shell gotchas."
|
||||
---
|
||||
|
||||
# Authentik OAuth2 relying-party (RP) SSO login diagnosis
|
||||
|
||||
Use when a vhost authenticates through authentik via the OIDC authorization-code
|
||||
flow (Grafana, Seerr OIDC, any "Sign in with <authentik>" button) and the login
|
||||
fails — even though the guard-chain e2e (Anubis + outpost) passes green.
|
||||
|
||||
## Key insight: the edge test lies
|
||||
|
||||
The outpost-guard spec (e.g. `13-sso`) only proves an authenticated session
|
||||
reaches the app's *login page*. It never follows the RP OAuth button. So the RP
|
||||
login can be 100% dead and the suite still passes. **Always drive the real RP
|
||||
round-trip** to get ground truth (harness below).
|
||||
|
||||
## Symptom → cause map (all verified live on 2026-10-20, grafana)
|
||||
|
||||
| Signature | Root cause | Fix |
|
||||
|---|---|---|
|
||||
| Browser hits `…/application/o/authorize/?client_id=…` → "Server Error — Powered by authentik" (500), traceback `AttributeError: 'NoneType' object has no attribute 'slug'` in `flows/planner.py` | provider `authorization_flow` is NULL (raw `ak shell` `create()` leaves it unset) | set `authorization_flow = default-provider-authorization-implicit-consent` |
|
||||
| Grafana log `logger=oauth.generic_oauth level=error msg="Error getting email address" url=<api_url>/emails … 404/403` | the granted scope mapping returns a **bare string**, not a dict → no `email` claim emitted | fix the ScopeMapping expression to `return {"email": user.email, "email_verified": True}` |
|
||||
| authorize 500 on scope-overlap, log `Application requested scopes not configured` for a scope like `user:email` | client requested a scope authentik doesn't define (GitLab-ism) | client must request authentik's real scopes: `openid email profile` |
|
||||
| Grafana `?api=1` on API URL → GET `<…>/userinfo/?api=1/emails` → 403 | Grafana v13 appends `/emails` to a `?api=1` URL | use the standard `…/userinfo/` with **no `?api=1`** |
|
||||
|
||||
Crucial causal note: **Grafana v13 GETs `<api_url>/emails` for the address ONLY
|
||||
when the token carries no `email` claim.** So the `/emails` 404 is a *symptom of
|
||||
the missing claim*, not a Grafana misconfig. Do NOT chase it with `USE_ID_TOKEN`
|
||||
or attribute paths; fix the claim. `USE_ID_TOKEN=true` does NOT stop the
|
||||
`/emails` fetch. Once the claim exists, standard userinfo carries `email` and no
|
||||
`USE_ID_TOKEN` is needed.
|
||||
|
||||
## Live RP-login probe harness (ground truth)
|
||||
|
||||
A `qa-e2e` authentik session storage state exists at
|
||||
`tests/e2e/.auth/qa-e2e.json` after an e2e run; the e2e node_modules live in the
|
||||
main checkout (`tests/e2e/`). Throwaway probe:
|
||||
|
||||
```js
|
||||
// node _p.cjs (run from a checkout with node_modules + .auth)
|
||||
const { chromium } = require('playwright');
|
||||
(async () => {
|
||||
const b = await chromium.launch({ headless: true });
|
||||
const ctx = await b.newContext({ storageState: '.auth/qa-e2e.json',
|
||||
userAgent: "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36" });
|
||||
const p = await ctx.newPage();
|
||||
await p.goto('https://<vhost>/', { waitUntil: 'domcontentloaded' });
|
||||
for (let i=0;i<30;i++){ const t=await p.locator('body').innerText().catch(()=>''); if(!t.includes('not a bot')) break; await p.waitForTimeout(3000);}
|
||||
await p.goto('https://<vhost>/login/generic_oauth', { waitUntil: 'domcontentloaded' });
|
||||
await p.waitForTimeout(9000);
|
||||
const r = await ctx.request.get('https://<vhost>/api/user');
|
||||
console.log(p.url(), r.status(), (await r.text()).slice(0,200));
|
||||
await b.close();
|
||||
})();
|
||||
```
|
||||
|
||||
Success = final URL on the app host (not `/login`) AND `/api/user` 200 with an
|
||||
email. Before the fix it is `/login` + 401. Move the throwaway to
|
||||
`~/.AGENTTRASH/…` after (never `rm`).
|
||||
|
||||
## ak-shell census: operational traps (authentik 2026.8)
|
||||
|
||||
- **Run it via `ssh dankclaw bash -s <<'EOF' … EOF`, NEVER direct
|
||||
`ssh host '… "{{…}}"'`.** Direct ssh eats inner quotes locally; `podman inspect
|
||||
--format '{{…}}'` and multi-line `ak shell -c "…"` only survive inside a quoted
|
||||
heredoc. Authentik runs **rootless** (`podman exec authentik-server …`, no sudo).
|
||||
- `OAuth2Provider` has **no `properties` attribute** in 2026.8; scopes are derived
|
||||
from `p.property_mappings`. Accessing `.properties` raises and silently aborts
|
||||
the whole `-c` payload.
|
||||
- `ScopeMapping` is **not importable** from `authentik.core.models`. To read
|
||||
scopes: iterate `p.property_mappings.all()`, then `m.scopemapping.scope_name`
|
||||
(reverse one-to-one; the base `PropertyMapping` row has no `scope_name`).
|
||||
- Introspect before guessing: `[f.name for f in Model._meta.get_fields()]`. Don't
|
||||
trial-and-error attribute names — that wastes turns.
|
||||
- Claims-into-id_token field is `include_claims_in_id_token`.
|
||||
- Provider census that works:
|
||||
```
|
||||
from authentik.providers.oauth2.models import OAuth2Provider
|
||||
for p in OAuth2Provider.objects.all():
|
||||
af=p.authorization_flow
|
||||
print(p.name, af.slug if af else None)
|
||||
# email mapping shape (the real bug):
|
||||
p=OAuth2Provider.objects.filter(name='grafana').first()
|
||||
for m in p.property_mappings.all():
|
||||
print(m.name, getattr(getattr(m,'scopemapping',None),'scope_name',None), repr(getattr(m,'expression','')))
|
||||
```
|
||||
|
||||
## Repair = idempotent script (DB state, not deployable)
|
||||
|
||||
Authentik config lives in its DB, not in any repo file; a blueprint re-import or
|
||||
an `ak shell` re-creation reverts provider fields silently. Ship an idempotent
|
||||
`scripts/authentik-<svc>-sso-fix.sh` that `ssh HOST 'podman exec -i
|
||||
authentik-server ak shell' <<'PYEOF' … PYEOF` and re-applies each fact guarded by
|
||||
"already set?" checks. Match the style of
|
||||
`scripts/authentik-set-login-session-duration.sh`; add a header note "DB state —
|
||||
re-run after a rebuild". Reference it from the service page. Grafana's half
|
||||
(env) lives in `grafana.container` and deploys normally.
|
||||
|
||||
## Repo test-seam gotchas this triggers
|
||||
|
||||
- **literal-secret seam** (`observability_quadlet.bats`, "no literal secret")
|
||||
globs keys `*_SECRET|*_PASSWORD|*_TOKEN|*_API_KEY` and demands a `${VAR}`
|
||||
value. A boolean env ending in `_TOKEN` (`…_USE_ID_TOKEN=true`) is a **false
|
||||
positive**. Prefer not to set such a key (drop `USE_ID_TOKEN`); otherwise the
|
||||
seam must be refined (ask the owner before editing a pre-existing test).
|
||||
- Quadlet `Environment=` **word-splits on spaces** (systemd semantics) — a spaced
|
||||
list must be quoted: `Environment="KEY=a b c"`, else it truncates to `a`. The
|
||||
repo already uses this (`open-webui.container` OAUTH_SCOPES).
|
||||
- **OKF conformance gate** rejects a page frontmatter `timestamp` in the future.
|
||||
The fleet host clock reads UTC (often a day ahead of local "today") — set the
|
||||
timestamp to the repo's today, not the host's UTC date.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: authentik-session-ttl-query
|
||||
description: "Query or change authentik login-session TTL on Dankclaw via ak shell — the string-typed UserLoginStage.session_duration trap, the apply logic, and why SESSION_SAVE_EVERY_REQUEST is unavailable."
|
||||
---
|
||||
|
||||
Change or audit how long an authentik login session lives (Dankclaw `authentik-server`, build 2026.x). The real knob is a DB stage row, not env — and the field format will bite you.
|
||||
|
||||
## Where the TTL actually lives
|
||||
- Run program through raw stdin: `ssh dankclaw 'podman exec -i authentik-server ak shell' <<'EOF' ... EOF 2>&1 | sed -n '/objects imported/,$p'`.
|
||||
- Get models by label, NOT import paths (`from authentik.core.models.setting import ...` fails; `authentik.core.models` is not a package; GlobalSettings is NOT a model in 2026.x):
|
||||
`from django.apps import apps; ULS = apps.get_model("authentik_stages_user_login", "UserLoginStage")`
|
||||
- The password sign-in path is flow `default-authentication-flow` → stage `default-authentication-login` (FlowStageBinding FK is `target`, not `flow`). Other stages `default-source-authentication-login` / `default-source-enrollment-login` govern external-IdP/enrollment logins only.
|
||||
|
||||
## THE FORMAT TRAP (will break login if wrong)
|
||||
`UserLoginStage.session_duration` is **string-typed** in authentik's own idiom: `"days=7"` / `"seconds=2"`.
|
||||
- Assigning a `datetime.timedelta` stores Python's `str(timedelta)` = `'7 days, 0:00:00'`, which authentik's parser `authentik.lib.utils.time.timedelta_from_string` **REJECTS** (`ValueError: not enough values to unpack`). Left like that, login crashes computing the expiry.
|
||||
- Correct write: `s.session_duration = "days=7"; s.save(update_fields=["session_duration"])`
|
||||
- Verify it parses: `from authentik.lib.utils.time import timedelta_from_string as P; P(s2.session_duration)` → `7 days, 0:00:00`. A single `key=value` token only; `"days=7 seconds=0"` also errors.
|
||||
|
||||
## How it applies (stage.py:77-87, verified)
|
||||
`delta = timedelta_from_string(session_duration)`; `if remember: delta += remember_me_offset`; then `set_expiry(0) if delta==0 else set_expiry(delta)`. So any positive duration applies to EVERY login (remember-me only adds on top), and `set_expiry(<positive>)` overrides `SESSION_EXPIRE_AT_BROWSER_CLOSE` → persistent cookie. Existing sessions keep their old expiry; new logins only.
|
||||
|
||||
## SESSION_SAVE_EVERY_REQUEST (sliding window) is NOT available
|
||||
`/authentik/root/settings.py` reads `SESSION_COOKIE_AGE` from `CONFIG.get("sessions.unauthenticated_age","days=1")`, sets `SESSION_EXPIRE_AT_BROWSER_CLOSE=True`, and references **`SESSION_SAVE_EVERY_REQUEST` nowhere** → no `AUTHENTIK_*` env or config.yml key can flip it. Enabling a sliding window means patching the vendor image; instead raise the login-stage duration (absolute, supported).
|
||||
|
||||
## Audit the rest of the expiry surface (live values)
|
||||
- framework: `settings.SESSION_COOKIE_AGE`, `SESSION_SAVE_EVERY_REQUEST`
|
||||
- OAuth2 RPs: `apps.get_model("authentik_providers_oauth2","OAuth2Provider")` → `access_token_validity` / `refresh_token_validity` (this fleet: access 1h [embedded-outpost 24h], refresh 30d)
|
||||
- proxy claim cache: `authentik_providers_proxy.ProxySession.expires` (observed ~1–24h; silent re-auth, not a logout)
|
||||
- tenant: `authentik_tenants.Tenant.default_token_duration` (hand API tokens, not sessions)
|
||||
|
||||
## Notes
|
||||
- These stage/provider rows are DB state, NOT repo-managed → a blueprint re-import or fresh rebuild reverts them; drop a setup script/blueprint in the repo if reproducibility matters.
|
||||
- Don't probe the guarded vhost for the agent API and don't confuse Anubis' 168h challenge cookie with the authentik login session.
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
name: autoresearch-llm-bench-window
|
||||
description: "How to run an iterative config-tuning window on a Dank-fleet LLM endpoint: harness pattern, noise-floor-first discipline, bracket mapping, and the frank-primary laws measured in the 2026-10-06 segment."
|
||||
---
|
||||
|
||||
# Autoresearch bench window for a fleet LLM endpoint
|
||||
|
||||
Use when running (or re-running) an iterative config-tuning segment against a
|
||||
Dank-fleet LLM endpoint (frank-primary/frank-large/frank-small): build the
|
||||
harness, establish the noise floor, then bracket-map one flag at a time.
|
||||
|
||||
## Harness pattern (already built for frank-primary — reuse, don't rebuild)
|
||||
|
||||
Committed at `autoresearch.sh` (dev box) + `scripts/autoresearch-frank-primary.sh`
|
||||
(dankserver) in serverAppConfigs. The pattern generalizes to any fleet endpoint:
|
||||
|
||||
1. Dev-box wrapper scp's the candidate env file + bench script to the target
|
||||
host, then ssh-runs the bench. Never push the candidate config to origin
|
||||
(the scp dirties the host checkout on purpose).
|
||||
2. Host script, in order: stop the production unit (relinked units: `systemctl
|
||||
stop`, NEVER mask — mask refuses a symlinked unit file) → boot a scratch
|
||||
container (distinct name, e.g. `arbench`) on the SAME devices with the
|
||||
candidate env — mirror the production podman contract exactly, differing
|
||||
only in container name and a FRESH prefix-cache tier (wiped per run) so cold
|
||||
measurements stay cold → poll the health endpoint until it answers (the
|
||||
listener binds AFTER the full load, so a bare curl is honest; allow ~2-3 min
|
||||
cold boot, 720s timeout) → run ONE self-contained deterministic request →
|
||||
parse the engine's timings object into METRIC lines → tear down the scratch →
|
||||
restore production.
|
||||
3. The restore MUST be an EXIT trap (`restore() { podman rm -f scratch;
|
||||
systemctl start <unit>; }`) so a failed bench never leaves the endpoint
|
||||
down. Verify after EVERY run: `systemctl is-active`, health curl until it
|
||||
returns JSON (200 right after start is NOT enough — it can bind 60-90s
|
||||
later), and no scratch leftover.
|
||||
4. Bench request: fixed prompt built from a sorted repo-corpus glob with a
|
||||
fixed random seed (deterministic → stable prompt_n), greedy sampling
|
||||
(temperature 0) for a fixed decode length, sent locally (127.0.0.1) to
|
||||
remove LAN jitter. Read prefill/decode/draft-acceptance from the engine's
|
||||
own timings object — never roll a custom bench without asking.
|
||||
5. Scratch containers must die by teardown; never leave a script whose tail
|
||||
stops/removes a production unit (the stale-probe-tail lesson).
|
||||
|
||||
## Discipline (the laws that make the numbers mean anything)
|
||||
|
||||
1. **Noise floor FIRST.** Before iterating, replicate an identical config 2-3x
|
||||
and treat that spread as the floor. Fleet endpoints here: decode ±3%,
|
||||
prefill ±2%. A single-run "win" under the floor is NOT a win — this exact
|
||||
mistake cost a false 33.06 keep in the 2026-10-06 segment.
|
||||
2. **One flag change per iteration**; log every run honestly (keep/discard/
|
||||
crash); discards auto-revert the worktree.
|
||||
3. **Bracket-map, don't climb.** Test both directions of every lever around
|
||||
its deployed value, INCLUDING boot failures — crashes are envelope data
|
||||
("0.82/2048/wht6/40960 is the seqs=3-viable quartet" came from four crash
|
||||
walls). Test downward too: 40960 MiB host pool is a floor, not slack.
|
||||
4. **Repro before recording a mechanism.** An anomalous result next to an
|
||||
abandoned bench may be contamination (boot race). Re-run clean once; only
|
||||
then name a law. The MNBT 2560 collapse (2.6 t/s) was reproduced clean to
|
||||
rule this out.
|
||||
5. **Watch for truncation.** predicted_n < max_tokens means the generation
|
||||
died mid-run — that config is pathological regardless of the rate.
|
||||
6. **When the sweep ties, tie-break on real-traffic properties** (e.g. draft
|
||||
acceptance for multi-turn workloads), never on probe artifacts, and prefer
|
||||
the config that matches the documented runbook.
|
||||
|
||||
## frank-primary specifics (2026-10-06 segment, commit c80c250)
|
||||
|
||||
- MNBT=2048 is a RESONANCE POINT at MTP-on shapes, not an optimum: 2560
|
||||
collapses decode to 2.6-7.7 t/s (acceptance normal, verify stalls ~130ms/step,
|
||||
truncated output), 1536 truncates at 57/256, 4096 won't boot at seqs=3. Do
|
||||
not re-test.
|
||||
- HOST_POOL_MIB 40960 is the load-bearing floor (36864 crashes boot).
|
||||
- Spec depth 1-3 and seqs 2-3 all tie within noise; 0 and 4 are real
|
||||
regressions (29.57 / 24.57 t/s vs 31.72 baseline).
|
||||
- Levers known flat: GPU_HEADROOM, HOST_POOL upward, maxlen above 200k.
|
||||
- Final kept config = Frank's original (spec=3, seqs=2, maxlen=200k, wht6,
|
||||
0.82, 2048, 40960, fp8 KV, TP=2), baseline decode 31.72 t/s at 43.6k depth.
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
name: bash-cwd-drift-multi-worktree
|
||||
description: "Diagnose \"files keep reverting / phantom edits\" in multi-worktree sessions: the persistent bash shell's cwd silently reverted to another checkout, so probes and git commands ran against different files."
|
||||
---
|
||||
|
||||
# Multi-worktree bash cwd drift
|
||||
|
||||
## Symptom (all of these at once, in one session)
|
||||
- A file "keeps reverting" to an older version between edits (esp. test files).
|
||||
- `grep -n` line numbers disagree with the tool/test output (e.g. bats reports `line 637`, grep says the match is at 519).
|
||||
- `wc -l`/`md5sum` stable when sampled back-to-back, but state "flaps" across the session.
|
||||
- `git status` clean in one call, surprising in the next.
|
||||
- Probes against `src/` show code that doesn't exist in the branch you think you're on.
|
||||
|
||||
## Root cause
|
||||
The persistent bash shell resets to the session's ORIGINAL cwd when a call omits `cwd` (e.g. after a shell respawn at a turn boundary). With several git worktrees of the same repo checked out, consecutive calls silently alternate between two full copies of the tree — half the reads land in the old branch's files. Nothing is rewriting anything; two files are being inspected.
|
||||
|
||||
## Decision
|
||||
- In multi-worktree work, pass an **explicit `cwd` on EVERY bash call**. Never rely on persistence.
|
||||
- Use **absolute paths** for `read`/`edit`/`write` (already required) — file tools don't drift; only bash does.
|
||||
|
||||
## Check
|
||||
When state seems to flap: `pwd` in the same call as the probe. If two calls to the same relative path report different content, `git worktree list` and census each checkout separately — you have two files, not a gremlin.
|
||||
|
||||
## Repair pattern (if corruption got committed via the wrong tree)
|
||||
1. `git log --oneline --all -- <file>`; per candidate commit print `wc -l` + counts of a few marker strings to classify clean vs corrupted blobs.
|
||||
2. Restore the good blob with `git show <commit>:<path> > <path>` (no checkout — safe under worktree policies).
|
||||
3. Re-apply the intended additions as surgical edits; verify with targeted tests before the full suite.
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
name: bats-assertion-trust
|
||||
description: "When writing or reviewing bats shell tests, catch vacuously-passing assertions: grep -vq, sed range boundaries, unanchored section slices, and fixtures too small to trigger pipefail/SIGPIPE bugs."
|
||||
---
|
||||
|
||||
Prove a bats assertion can actually fail before trusting that it passes. These four traps each made a green suite lie in real work.
|
||||
|
||||
## 1. `grep -vq` proves nothing
|
||||
|
||||
`some_list | grep -vq 'x'` succeeds when ANY line lacks `x` — nearly always true. To assert absence, capture the text and test it:
|
||||
|
||||
```bash
|
||||
s="$(report_section)"
|
||||
[[ "$s" != *libfoo* ]] # absence
|
||||
[[ "$s" == *bun* ]] # presence
|
||||
```
|
||||
|
||||
Same trap in reverse: bash `[[ $s == *A*B* ]]` — `*` crosses newlines, so a substring check spanning two lines matches the wrong lines. Slice to one line first, then assert.
|
||||
|
||||
## 2. Section slicing helpers must have explicit boundary semantics
|
||||
|
||||
A helper used to scope assertions to one report section must define, and comment, both ends. The default you want: **start inclusive** (labels carry the count/names), **terminator exclusive** (a `sed -n "/A/,/B/p"` range includes B and leaks the next section's first line):
|
||||
|
||||
```bash
|
||||
# Call sites pass a bare anchored regex; both ends documented.
|
||||
section() { printf '%s\n' "$output" | awk -v s="$1" -v e="$2" '$0 ~ e {on=0} $0 ~ s {on=1} on'; }
|
||||
```
|
||||
|
||||
Two bugs this shape hides:
|
||||
- Helper prepends `^` while call sites already pass `^name` → pattern `/^^name/` → never matches → empty slice.
|
||||
- An **unanchored start address** matches the first line and the range runs to end-of-output, so every absence assertion becomes a vacuous pass. Symptom: tests that "verify X is not in section A" pass while X is printed later in the same output.
|
||||
|
||||
After fixing a helper like this, re-run the whole suite: tests that were passing vacuously suddenly fail, and that is the fix working.
|
||||
|
||||
## 3. Fixtures too small hide pipefail bugs
|
||||
|
||||
`set -euo pipefail` + `sort | head -1`: `head` exits, closes the pipe, `sort` dies of SIGPIPE, the command substitution returns `141`, `set -e` aborts the script. Small fixtures never overflow the 64 KiB pipe buffer, so the suite stays green and the first run against real data fails with no output.
|
||||
|
||||
- Take a minimum in one pass: `awk '{ if (m=="" || $1<m) m=$1 } END { print m }' file`
|
||||
- `awk 'NR==1{print; exit}'` is **not** a fix — exiting early closes the pipe exactly like `head`.
|
||||
- Pin it with a fixture larger than the pipe buffer: `seq 1 20000 | awk '{print "[2026-01-01T10:00:00-0700] [ALPM] installed pkgfiller (1-1)"}' > big.log` (use `seq`, not `yes | head`, or the fixture itself SIGPIPEs).
|
||||
|
||||
## 4. `tr` swallows the final newline and downstream tools preserve that
|
||||
|
||||
`tr '\n' ' ' < file` emits no trailing newline; `fold`, `sed`, and `cat` faithfully preserve the absence, so the *next* output line gets glued on. Breaks any tool that later slices output by line-start labels.
|
||||
|
||||
```bash
|
||||
{ tr '\n' ' ' < file; echo; } | fold -s -w 88 | sed 's/^/ /'
|
||||
```
|
||||
|
||||
Pin with: assert the following section's label does NOT appear inside the preceding section's slice.
|
||||
|
||||
## Debugging order when a bats assertion fails but the script output looks right
|
||||
|
||||
1. Reproduce the exact command outside bats (`bash -x` against the script under test) — if it works there, the bug is in the *test helper*, not the script.
|
||||
2. Dump the constructed argument: `printf '%s' "$sed_script" | od -An -c` inside the helper. That is how `/^^ repo /` (double anchor) got found.
|
||||
3. Keep stubs faithful to the real interface: if the stub answers `-Sqq` but the script calls `-Sl`, the test measures nothing. Make the stub exit `64` on unhandled args so a new real query fails loudly instead of silently returning empty.
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
name: bats-output-glob-red-proof-traps
|
||||
description: "Write red bats tests over command output: [[ ]] glob adjacency (quoted * is literal), run-capture vs test stdout in failure dumps, and proving red against pre-fix code before trusting green."
|
||||
---
|
||||
|
||||
Three traps hit while writing red-then-green bats tests for `remote-job.sh` (Refs #60, `dank/command-center` PR #61). Each one cost a detour because it makes a test look like a bug report when it is an assertion bug.
|
||||
|
||||
## 1. In `[[ $s == pat ]]` a quoted `*` is LITERAL — and it eats the wildcard beside it
|
||||
|
||||
```bash
|
||||
s=' x* DONE rc=0 1 lines'
|
||||
[[ "$s" == *"x*"DONE*"rc=0"* ]] # NOMATCH — and it looks like the tool under test broke
|
||||
[[ "$s" == " x*"*"DONE rc=0"* ]] # MATCH — quoted literals, UNQUOTED separators
|
||||
```
|
||||
|
||||
`*"x*"DONE*` means: anything, then the literal substring `x*DONE`, then anything. Quoting only makes the `*` inside the quotes literal; it does **not** keep the next unquoted token from butting up against it. When the subject has whitespace between the two facts, the pattern must carry an **unquoted** `*` between the quoted literals:
|
||||
|
||||
```
|
||||
"literal-a"*"literal-b" # a, then anything, then b
|
||||
```
|
||||
|
||||
Existing suites in the repo write `*"a"*DONE*rc=0*` — unquoted separators throughout — precisely for this reason. Copy that shape; do not quote for "clarity".
|
||||
|
||||
Discriminate before believing the test: run the pattern against BOTH the expected line and the line the buggy code produced (`correct-red` / `NOMATCH` on the second is the pass condition). A pattern that matches the old output was never a regression test.
|
||||
|
||||
## 2. A bats failure dump prints the TEST's stdout, not `$output`
|
||||
|
||||
```
|
||||
not ok 1 status treats a job name as a name, not a pattern
|
||||
# `[[ "$output" == *"x*"DONE* ]]` failed
|
||||
# started x*
|
||||
```
|
||||
|
||||
`started x*` is the stdout the test body itself printed (from a `start` call several lines earlier), **not** what `run` captured. Reading it as the captured value sends you to debug the tool. To see the real capture, echo it:
|
||||
|
||||
```bash
|
||||
run some-command
|
||||
echo "CAP:[${output}]"
|
||||
```
|
||||
|
||||
in a scratch copy of the test (`/tmp/dbg.bats`, same `setup`/harness) rather than adding diagnostics to the committed test.
|
||||
|
||||
## 3. Reproduce outside bats when green refuses to arrive
|
||||
|
||||
When a test fails but the same command works by hand, replicate the *exact* bats call shape in `/tmp` with the same env exports (`REMOTE_JOBS_DIR` etc.) and the same `run bash -c '…' _ "$A" "$B" "$C"` argument mapping. If the scratch copy passes and the committed one fails, the difference is the assertion, not the system — go back to trap 1.
|
||||
|
||||
## Red-proof rule
|
||||
|
||||
Run the new test against the pre-fix code and read *why* it failed. Both traps above produced a failure that looked meaningful while proving nothing (one matched the buggy output; one printed the wrong stream). Keep the test only when the failure reason is the defect you set out to pin.
|
||||
|
||||
Related: `bats-assertion-trust` (vacuous assertions), `red-verify-without-stash` (proving red in a shared worktree).
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: bench-arm-provenance-recovery
|
||||
description: "Before calling a leftover LLM bench drop-in a rogue/hijack, reconstruct whether Frank ordered and measured that arm: omp archive prompt timestamps + journald print_timing for the exact pid, then land the verdict in the repo."
|
||||
---
|
||||
|
||||
Use when a live service boots a payload you did not configure (stale `systemd` drop-in / `*_ENV_FILE` arm env) and the docs — or your own compaction-inherited context — call it "unfinished", "armed but never measured", or "hijack". Do NOT report that until proven: the arm may be an ordered, measured, approved configuration whose record lived only in `/tmp` + journald.
|
||||
|
||||
## Why this exists
|
||||
2026-10-18 `frank-zealot`: a `frank-zealot.service.d/ptq1.conf` drop-in made the cutover deploy boot a different model at a different shape. I reported it as an unfinished experiment. It was actually the 27B PTQ1_0 arm Frank ordered at 02:36, that served production 03:14:39→03:38:57, and whose numbers existed only in tool output that context compaction later dropped.
|
||||
|
||||
## Step 1 — recover the ORDER (the harness archive, not memory)
|
||||
`eval` with the `archive` prelude:
|
||||
|
||||
```python
|
||||
ps = await archive.prompts() # searchable prompt history, survives compaction
|
||||
# filter for the payload name / repo URL keywords, print p['at'] + p['text'][:90]
|
||||
r = await archive.session("<sessionId>") # full session record
|
||||
recaps = await archive.recaps() # recap stamps — check they postdate the window
|
||||
```
|
||||
|
||||
`archive.prompts()` keeps **user prompts with ISO timestamps** even when the middle of the transcript was dropped. Timestamp every instruction: ordered → measured → superseded. Convert to local (`UTC−7` here) before quoting. A recap older than the instruction proves the recap layer never covered the window.
|
||||
|
||||
## Step 2 — recover the MEASUREMENT (journal, per pid)
|
||||
The engine's own per-request timings are the record; they outlive `/tmp` env files.
|
||||
|
||||
```bash
|
||||
ssh dank@<host> 'journalctl -u <svc> --no-pager' | grep -F 'serve-<svc>.sh[<PID>]' \
|
||||
| grep -E 'prompt eval time|eval time' | awk '... median per task ...'
|
||||
```
|
||||
|
||||
- Pin the **pid** (`sh[NNNN]`): `--since` windows alone pull the previous process's numbers too.
|
||||
- llama.cpp format: `prompt eval time = 1234.56 ms / 100 tokens (…, 81.03 tokens per second)`; decode rows are `eval time = … ms / N tokens (…)`. Multiple spaces between `task`/`eval` break naive regexes — match `task\s+(\d+)\s+prompt eval time` exactly, or parse both labels separately.
|
||||
- Filter pathological rows: generations running concurrently on other slots, or ending in a 4-byte EOS, read ~4× low. Report clean single-stream probes (≥200 generated tokens) and label the rest.
|
||||
- Cross-check the boot banner for the payload identity (`type ptq1_0: 402 tensors`, KV ledger, slot/ctx lines) — that proves *which* shape that pid served.
|
||||
|
||||
## Step 3 — adjudicate, then land it in the repo
|
||||
Write the verdict in the same session you derive it: runbook deploy-ledger section, `log.md` dated section, the arm script's header comment, and the AGENTS.md law. `/tmp` and journald are not records — that absence is what makes the next agent misread an approved arm as a rogue one.
|
||||
|
||||
## Traps
|
||||
- `mask` fails on a relinked unit; never `systemctl disable`; retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload` (recoverable, not deleted).
|
||||
- `/health` 200, `/v1/models` echoing the alias, and a green e2e gate all fail to read the payload — check `systemctl show <svc> -p DropInPaths` (must be empty), then `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger.
|
||||
- Verification sequence that is honest: correction commit must quote commands you ran; state `[INFERENCE]` for anything the journal does not show.
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
name: bench-harness-fault-census
|
||||
description: Attribute benchmark failures to harness vs model via exception-signature census before trusting any score; python-grader fault signatures and fixes
|
||||
---
|
||||
|
||||
# Rule (from 2026-10-10 frank-intelligence-bench incident, 3 harness faults found)
|
||||
An LLM benchmark score is VOID until every failure signature is attributed to the MODEL. Run the census before quoting any number.
|
||||
|
||||
# Procedure
|
||||
1. Aggregate graded rows per eval; for each failed grade extract the LAST python exception signature:
|
||||
regex `(\w+Error|\w+Exception): (?:name '([^']+)' is not defined|No module named '([^']+)')?` → sig = `Type: detail`.
|
||||
2. Indict the harness (not the model) when:
|
||||
- `NameError: target` / `NameError: targets` in any count → expected-value bindings never reached the tests (gold data file missing/unstaged).
|
||||
- `ModuleNotFoundError: <grader package>` (e.g. `scicode`) → grading deps missing from the execution image.
|
||||
- ≥10 failures sharing ONE signature at >50% share → systematic fault; audit the protocol (e.g. markdown fences reaching an exec-based extractor — `SyntaxError` storm; strip ``` fences before feeding model text to graders that exec raw code).
|
||||
- >50% of generations fail → endpoint/transport fault.
|
||||
3. Real model failures look heterogeneous: AssertionError/AxisError/NameError of the model's OWN undefined names.
|
||||
|
||||
# Companion checks
|
||||
- Prove the grading path with a gold-solution integration test BEFORE trusting live scores: gold must PASS end-to-end (FIB_INTEGRATION=1 pattern).
|
||||
- Quarantine poisoned ledgers (.AGENTTRASH) rather than deleting; census them to root-cause.
|
||||
- Expect-score sanity: mixed PASS/FAIL with model-side signatures = real signal, even if the % looks low.
|
||||
@@ -1,69 +0,0 @@
|
||||
---
|
||||
name: bench-llama-server-params
|
||||
description: "Benchmark llama.cpp server-side parameters (--threads-batch, --ubatch-size, --spec-type) using HTTP API with max_tokens=1 for prefill, max_tokens=16384+ for decode"
|
||||
---
|
||||
|
||||
# Benchmarking llama.cpp Server-Side Parameters
|
||||
|
||||
## Problem
|
||||
`llama-bench` benchmarks the bench binary itself, not the running server.
|
||||
It cannot measure the effect of server-side parameters like `--threads-batch`,
|
||||
`--ubatch-size`, `--spec-type`, or `--parallel`.
|
||||
|
||||
## Solution: HTTP-Based Benchmarking
|
||||
|
||||
### Prefill throughput (prompt processing)
|
||||
1. Start the server with the target parameter value
|
||||
2. Send a large prompt via the chat API with `max_tokens=1`
|
||||
3. Measure `prompt_tokens / wall_time` from the response usage
|
||||
|
||||
`max_tokens=1` is correct here — you want only prompt processing time,
|
||||
not generation. The server crashes on `max_tokens=0`.
|
||||
|
||||
### Decode throughput (generation)
|
||||
1. Start the server with the target parameter value
|
||||
2. Send a prompt that generates a long response
|
||||
3. Use `max_tokens=16384` minimum (16k) — **never `max_tokens=1`** for decode
|
||||
benchmarking. One token tells you nothing about sustained generation rate.
|
||||
4. Measure `predicted_per_second` from the response `timings` object, or
|
||||
compute `completion_tokens / wall_time`
|
||||
|
||||
### Why at least 16k tokens for decode
|
||||
- Decode throughput varies over the generation (warmup, cache growth, MoE
|
||||
routing). A single token is noise.
|
||||
- Speculative decoding acceptance rates stabilize over thousands of tokens.
|
||||
- Short generations finish before the draft head reaches steady state.
|
||||
- 16k tokens gives a stable average at ~22 tok/s (≈12 min per run).
|
||||
|
||||
### Key Pattern
|
||||
```python
|
||||
# Decode benchmark — measure generation throughput
|
||||
body = json.dumps({
|
||||
"model": "<alias>",
|
||||
"messages": [{"role": "user", "content": "<prompt that generates 10k+ tokens>"}],
|
||||
"max_tokens": 16384, # 16k minimum for decode; use max_tokens=1 ONLY for prefill
|
||||
"temperature": 0, # deterministic for reproducibility
|
||||
"stream": False
|
||||
})
|
||||
# Measure wall time from request send to response receipt
|
||||
# Get predicted_per_second from timings, or completion_tokens / wall_time
|
||||
```
|
||||
|
||||
### When to Use
|
||||
- `--threads-batch` (prefill): use `max_tokens=1`
|
||||
- `--ubatch-size` (decode): use `max_tokens=16384+`
|
||||
- `--spec-type` / speculative decoding: use `max_tokens=16384+` (decode only)
|
||||
- Any server-side parameter that `llama-bench` cannot target
|
||||
|
||||
### Benchmarking policy (MANDATORY)
|
||||
1. **STOP the production service** before running any benchmark
|
||||
2. **Mask** the service (`systemctl mask`, never `disable` — on these hosts
|
||||
`disable` deletes the unit file from `/etc/systemd/system/`)
|
||||
3. Run the bench
|
||||
4. **Unmask + restart** the service when done
|
||||
5. **Verify health** before reporting results
|
||||
|
||||
### Example Scripts
|
||||
- `scripts/bench-threads-batch-http.py` — prefill sweep (max_tokens=1)
|
||||
- `scripts/bench-decode-threads-batch.py` — decode sweep (max_tokens=256+)
|
||||
- `scripts/bench-spec-decode.py` — speculative decoding config sweep
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
name: bounding-subagent-runtime
|
||||
description: "Cap and diagnose runaway or hung omp subagents: task.maxRuntimeMs, softRequestBudget, maxConcurrency, maxRecursionDepth (the 0-trap), per-provider in-flight ceilings."
|
||||
---
|
||||
|
||||
## When
|
||||
Subagent runs long, fan out too wide, or you must (re)declare the fleet's subagent limit policy. Also when asked "what limits can subagents have".
|
||||
|
||||
## Read first
|
||||
- Repo KB page `omp-docs/pi/subagent-limits.md` — the live policy of record (values + rationale + deliberately-not-set list). Do not re-derive; read it.
|
||||
- `cfg://task` and `cfg://providers/maxInFlightRequests` — effective values, not the stock defaults.
|
||||
|
||||
## The knobs (cfg:// writes; `/save` suffix persists)
|
||||
| Knob | Effect | Gotcha |
|
||||
|---|---|---|
|
||||
| `task.maxConcurrency` | session semaphore; excess queues | stock default 32 — "2" is not a default |
|
||||
| `task.maxRecursionDepth` | nesting cap | predicate is `depth < n`; **1 = children cannot spawn (correct for "no nesting"); 0 disables spawning for the MAIN session too — never reach for 0**; negative = unlimited |
|
||||
| `task.maxRuntimeMs` | hard wall clock, overrun = `aborted` | default 0 = off |
|
||||
| `task.softRequestBudget` | wrap-up notice at N, force-yield at 1.5N | scout/sonic may impose lower built-ins |
|
||||
| `task.agentIdleTtlMs` | park idle finished agents | `<= 0` disables parking |
|
||||
| `providers.maxInFlightRequests` | concurrent LLM requests per provider id | keys are the quoted block labels from `agent/models.yml` (e.g. `"llamacpp (dankclaw-3090)"`), flat map lookup; absent/≤0 = unlimited; scope = this machine's config root only — other workstations and non-omp clients are NOT counted; excess queues |
|
||||
| `tools.maxTimeout` | ceiling on any agent-requested tool timeout | kills `timeout: 0` runaways |
|
||||
| `task.isolation.enabled` | makes per-spawn `isolated` flag available | opt-in per spawn, not a blanket sandbox; needs git repo; off in plan mode |
|
||||
| `task.completionProbe` | periodic completeness side-requests | token sink on shared endpoints |
|
||||
|
||||
## Verify semantics from the bundle, not the label
|
||||
Setting descriptions lie by omission. `~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js`:
|
||||
- depth gate: `grep -o '.\{130\}xC\b.\{130\}'` finds `U_(n,s)`; `grep -o 'function U_(e,t){.\{0,60\}'` → `e<0||t<e` (n=cap, s=taskDepth).
|
||||
- in-flight lookup: `grep -o '.\{150\}axInFlightRequests.\{200\}'` → flat `[providerId]` lookup, non-number/≤0 = unlimited, clamp ≥1.
|
||||
|
||||
## Landing a policy change
|
||||
cfg `/save` writes persist immediately; `agent/config.yml` is machine-live tier (heartbeat commits it). Human-facing policy changes land as an `omp-docs` page update via branch → PR (docs tier), `git worktree add` from `origin/main` if the current checkout is branch-locked by workspace policy.
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
name: cfg-approval-timeout-recovery
|
||||
description: "Use when a cfg://.../save write fails with \"user did not answer the approval prompt\": the settings-approval dialog has a hardcoded 10 s timeout and no config key; recover via direct config.yml edit once the user has approved the change."
|
||||
---
|
||||
|
||||
# Recovering from cfg:// approval-prompt timeouts
|
||||
|
||||
## Facts (verified in bundle 2026-10-18)
|
||||
- Settings-write approval dialog timeout is **hardcoded 10 000 ms** (`timeout: JKl`, `JKl=1e4`, countdown class treats arg as ms) in the omp `dist/cli.js` bundle. **No config key exists** for it (checked full `cfg://` registry).
|
||||
- `ask.timeout` is a DIFFERENT gate (ask-tool auto-select; 0 = wait forever). It does not govern cfg approvals.
|
||||
- A timed-out approval silently auto-denies: session-only value stays applied, `config.yml` unchanged. Error says "nothing changed; it stays <value>. NEVER retry unless the user asks."
|
||||
- Related: `features.unexpectedStopDetection: smart` injects forced-continue `<system-injection>`s on text-only turns ending in a promise to act ("Task incomplete: MUST call the next concrete tool now"). Set `mechanical` (upstream default) or `none`; `smart` drives agents into junk subagent spawns to satisfy the nudge.
|
||||
|
||||
## Recovery procedure (only when the user has already explicitly approved the change)
|
||||
1. Do NOT re-issue `cfg://<key>/save` after a timeout unless the user explicitly asks again — the harness forbids it.
|
||||
2. Edit `~/.omp/agent/config.yml` directly:
|
||||
- `grep` the setting key to locate it; read surrounding lines.
|
||||
- surgical `edit` of the single line.
|
||||
- validate: `python3 -c "import yaml; yaml.safe_load(open('/home/blank/.omp/agent/config.yml'))"`
|
||||
3. State that the file was edited directly because the invisible 10 s dialog ate the user's approval, and offer the `omp config set <key> <value>` one-liner as the user-run alternative for future.
|
||||
|
||||
## Prevention
|
||||
- Before any `cfg://.../save`, warn the user a ~10 s dialog is about to appear and to answer immediately.
|
||||
- Bundle-fact lookups: `PKGDIR=$(dirname $(readlink -f $(command -v omp)))/..`, then grep `dist/cli.js` for the message string, walk to the call site, resolve minified constants with `grep -oE '[,;]CONST=[^,;)]+'` (`1e4` = 10000).
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: characterization-fixture-engine-cutover
|
||||
description: "Swap a path→service planner/mapping engine behind a golden fixture: capture, delta-reduce, document deltas, flip grep seams to behavioral, add render/drift gates, validate Caddyfile renders, Gitea PR URLs (serverAppConfigs cutover playbook)"
|
||||
---
|
||||
|
||||
Playbook for replacing a hand-maintained mapping engine (deploy planner, seam mapper, allow-lists) with a data-driven one without changing observable behavior. Proven on the 2026-10-13 `shared/services.yml` + `scripts/service-map.py` cutover (921 deltas → 17 intentional → green; 408/408 bats).
|
||||
|
||||
## 1. Capture the characterization fixture FIRST
|
||||
- Enumerate every path the old engine sees: all tracked files x all targets (`git ls-files`), plus hand-added edge paths (retired files, cross-host paths, shared networks).
|
||||
- Run the OLD engine on each, record `path<TAB>target<TAB>services` to `tests/fixtures/*-golden.tsv`. Old engine = current `deploy-plan.py` before edits.
|
||||
- This fixture is the contract; the new engine must reproduce it except documented deltas.
|
||||
|
||||
## 2. Cut over, then delta-reduce with the fixture
|
||||
- Diff new-engine output vs fixture; bucket every delta into: engine bug / intentional change / fixture staleness.
|
||||
- Engine bugs found this way (generic classes): `in` membership on dict-of-tuples, host-scoping of relative directory keys, riders/children resolved only in the changed file's own directory, stem collisions between unit files and generated artifacts (`.network` stem == service dir stem), `.network` files entering the unit branch by stem match.
|
||||
- Intentional deltas get documented individually in the commit message; NEVER batch-accept a delta count.
|
||||
- Regenerate the fixture LAST, from the new engine, and record the delta count + reasons in the commit.
|
||||
|
||||
## 3. Flip text-grep seams to behavioral calls
|
||||
- Old seams that grep a config file for the mapping (`grep -q 'dankclaw:hermes-webui'`) die silently on cutover. Replace with a call to the new engine: `run python3 scripts/service-map.py allows --host H --service S; assert_success`.
|
||||
- Vhost-scoped content greps (Caddyfile awk blocks) → grep the new canonical form (`import guarded`), not the inlined chain.
|
||||
|
||||
## 4. Renders + drift gates
|
||||
- Config artifacts that were hand-maintained mirrors (node-exporter quadlet pair, pihole addn-hosts) become `service-map.py render <name> --host H` artifacts; a drift test asserts tracked-file == render output.
|
||||
- Renderers must emit exactly what the consumer parses: dnsmasq `addn-hosts` treats inline `# comments` as hostnames — render no trailing comments.
|
||||
- Named-snippet collapse (Caddyfile guard chain): the snippet MUST be defined before the first site that imports it (put `(guarded)` right after the global options block), or real caddy reports `File to import not found`.
|
||||
|
||||
## 5. Suite + deploy gates
|
||||
- `./tests/run.sh` fully green before each rank commit; one atomic commit per rank.
|
||||
- Deploy is service-scoped: `deploy.sh <host> --service <svc>`; never a fleet drain for one service. The Playwright e2e gate runs inside deploy.sh against the live fleet.
|
||||
|
||||
## Repo-specific gotchas (serverAppConfigs)
|
||||
- Validate the Caddyfile with stock `caddy:latest` by DELETING the whole `acme_dns porkbun { ... }` block (`awk 'NR<53 || NR>56' dankclaw/Caddyfile`). A `sed s/acme_dns porkbun {/acme_dns acme {/` variant that drops the `{` swallows the following tokens into the global block and fakes an "import not found" on `(guarded)` — debug sed swaps with a minimal repro file before blaming the config. Stock-image `dns.providers.porkbun` unregistered + basicauth `REPLACE_WITH_HASH` base64 errors are the expected endpoints of a clean parse.
|
||||
- Gitea: branches with `/` 404 on `/compare/main...wt%2F...` — use the push-hook URL `…/pulls/new/<branch>`; private repo returns 404 to anonymous curl, which proves nothing about the page.
|
||||
- `test-changed.sh` contract: new seam selection ⊇ old selection per path; unmapped repo path must `die`, never pass silently.
|
||||
- Retire-to-trash: retired files (compose.yaml, dead templates, dead bats) → dated `.AGENTTRASH/YYYYMMDD-topic/` beside them; every repo sweep must exclude `.AGENTTRASH`.
|
||||
- bash `test -f a b c && ! test -e d` is invalid syntax (test takes one operand per operator); use `ls a b c` + separate `ls d` to census artifacts.
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: closed-engine-runtime-investigation
|
||||
description: "Recipe for diagnosing/tuning a closed-source LLM runtime (halogen-class) on this fleet: docs, env-lever enumeration from the binary, ledger scraping, cache-assisted A/B under the bench law."
|
||||
---
|
||||
|
||||
# Investigating a closed-source LLM runtime (halogen-class)
|
||||
|
||||
Use when a containerized closed-source engine (e.g. halogen on frank-large) underperforms and the repo docs don't explain it. Fleet law: stop the production unit BEFORE any GPU experiment (`sudo systemctl stop <unit>`; mask FAILS on relinked units — stop alone, redeploy after).
|
||||
|
||||
## 1. Ground truth before flags
|
||||
- Boot ledger + per-request timings live in `podman logs <container>` (container-lifetime only) UNLESS the serve script captures stdout on disk (foreground `podman run`). A `podman run -d ... > LOG` redirect captures ONLY the container ID — a one-line hex file is the tell.
|
||||
- `serve_api:`-style log lines carry per-request `prompt/prefill/rounds/commit/detok` — build the depth-vs-rate curve from REAL traffic before any bench. Shallow-healthy + deep-collapsed = a depth-linear term (KV reads), not a broken engine.
|
||||
- If the engine's public repo exists, read its FLAGS/docs (e.g. `docs/FLAGS.md`) AND the bench tool source (e.g. `tools/halogen-bench.py`) — bench tools reveal which regimes the published numbers never measured (halogen's tg rows are shallow-only).
|
||||
|
||||
## 2. Enumerate the undocumented levers
|
||||
The binary self-announces every env var it reads. The image often lacks `strings`:
|
||||
```
|
||||
podman run --rm --entrypoint /bin/sh <image> -c \
|
||||
'grep -aoE "<PREFIX>_[A-Z0-9_]+" /usr/local/bin/<engine> | sort -u'
|
||||
```
|
||||
Classify by kernel family (attention vs linear-state vs GEMV) against the suspected term. Only test levers with public precedent (e.g. issue #12 proved HALOGEN_FA_TQ) or an obvious name match; defaults are the publisher's measured winners — an A/B must show >=~15% to adopt.
|
||||
|
||||
## 3. Cache-assisted A/B (cheap reps)
|
||||
Boot each probe arm as its own container (env is read at startup) with the prompt cache ON (explicit MB >= one full-entry at ~70 KB/token): rep 1 pays the cold prefill, rep 2+ resumes (~2 s) — depth measurements then cost decode-time only. Publish the probe on host LOOPBACK (127.0.0.1:<scratch>), run the driver ON the host (python3 stdlib is enough), read the engine's own `timings` (prompt_ms/cache_n/predicted_ms/draft_n_accepted). Never benchmark beside a live engine on the same GPU.
|
||||
|
||||
## 4. Adopt or revert
|
||||
- Winner: env line into the serve env + test-seam update + docs, then `deploy.sh <target> --service <svc>` (restart restores everything; no unmask needed after stop-only).
|
||||
- Loser: `podman rm -f` probes, start the unit, revert nothing.
|
||||
- Either way: record the measured before/after in the runbook; report the depth behavior upstream if the publisher invites benchmark data — their bench gaps (depth, carve-topologies) are exactly what issues like #12 close.
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
name: command-center-bundle-lint-gate
|
||||
description: "Before committing docs in dank/command-center, lint the real omp-docs bundle directly — the bats suite only lints fixtures, so a green suite is not proof."
|
||||
---
|
||||
|
||||
# The bundle lint gate (`dank/command-center`)
|
||||
|
||||
Adding or editing pages under `omp-docs/`? `bats tests/` going green proves
|
||||
nothing about the pages you just wrote.
|
||||
|
||||
## Why this exists
|
||||
|
||||
`tests/kb-lint.bats` exercises `scripts/kb-lint.sh` against `mktemp -d` fixtures
|
||||
(`"conformant bundle passes"`, `"missing frontmatter is reported"`, …). For years
|
||||
nothing in the suite pointed the linter at the shipped tree, so a page with no
|
||||
frontmatter, an index that stopped listing a file, or a broken relative link
|
||||
committed under a green suite. (Two tests appended 2026-10-20 now run both
|
||||
checkers against `omp-docs/`; verify they are still present before assuming the
|
||||
suite covers it — `grep -n omp-docs tests/kb-lint.bats`.)
|
||||
|
||||
The general law: **a lint suite that tests only the linter does not lint your
|
||||
content.** Check which one a suite runs before calling green a gate.
|
||||
|
||||
## Gate a docs commit
|
||||
|
||||
```bash
|
||||
cd <worktree>
|
||||
bats tests/ ; grep -E '^1\.\.' /tmp/… # suite (count, failures)
|
||||
./scripts/kb-lint.sh omp-docs # want: kb-lint: 0 violation(s)
|
||||
python3 scripts/kb-coverage.py omp-docs # want: index-coverage: 0 issue(s)
|
||||
```
|
||||
|
||||
`kb-lint.sh <kb-dir>` (default `omp-docs`) = OKF v0.1 conformance **plus link
|
||||
integrity**, so it is the check that absolute bundle links like
|
||||
`/tools/gitea/client-access.md` resolve. `kb-coverage.py` = every page listed by
|
||||
its directory `index.md` (repo doctrine: nothing may appear in a directory its
|
||||
`index.md` does not list).
|
||||
|
||||
OKF shape that passes: every concept page carries frontmatter with a `type`
|
||||
(`type: reference` + `title`/`description`/`tags`/`timestamp`/`verified`);
|
||||
`index.md` files carry **no** frontmatter; new pages get an entry in their
|
||||
directory index in the same commit, plus a dated entry in `omp-docs/log.md`
|
||||
(one entry = what changed + the durable lesson, a few lines, never a narrative).
|
||||
|
||||
## Commit-message trap in this harness
|
||||
|
||||
A multi-line message written as a heredoc inside a chained `bash` command
|
||||
(`git commit -q -F - <<'MSG' … && …`) fails with
|
||||
`pi-natives:command: syntax error at end of input`. Write the message to
|
||||
`/tmp/msg-<topic>.txt` with the write tool and `git commit -q -F /tmp/msg-<topic>.txt`.
|
||||
Keep the `&&` chain so a failed gate stops the commit — hooks (secret gate) stay armed,
|
||||
never `--no-verify`.
|
||||
|
||||
## Land and reap
|
||||
|
||||
Merge is form-encoded `do=merge&merge_style=no-ff` and answers `200` with an
|
||||
**empty body**; never read that as ambiguous — verify by landing. Then
|
||||
`git -C ~/.omp merge --ff-only origin/main` (the checkout *is* the deployment;
|
||||
dirty `agent/memories/`, `agent/config.yml`, `APPEND_SYSTEM.md` are machine-tier
|
||||
live state and do not block an ff whose diff misses them), fast-forward the
|
||||
session's `wt/*` scratch branch the same way, then `git worktree remove <build-wt>`
|
||||
and `git branch -d <type>/<issue>-<slug>` (`-d` succeeding is itself proof of
|
||||
merge).
|
||||
|
||||
Stray files whose names start with `-` (e.g. `-c`, `-es` — vim takes a
|
||||
`--startuptime` flag name as the output filename) show as untracked dirt:
|
||||
`mv ./-c .AGENTTRASH/<date>-<meaning>.log`, never `rm`.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
name: corpus-voice-skill-rewrite
|
||||
description: "Rewrite a skill to emulate an author corpus's voice: full read, style-DNA distillation from writing-for-agents levers, positive-phrasing pass, sharpened completion criteria"
|
||||
---
|
||||
|
||||
# Corpus-voice skill rewrite
|
||||
|
||||
Rewrite a skill so it reads as if a chosen author wrote it. Done for `grilling` against the `mattpocock/skills` corpus (2026-10-18, command-center PR #20); behaviour preserved, voice swapped.
|
||||
|
||||
## Where the corpus lives
|
||||
|
||||
Installed third-party skills: `~/.agents/skills/<name>/SKILL.md` (the skills-CLI global store; `~/.agents/.skill-lock.json` names sources). Whole mattpocock corpus ≈ 2,800 lines — small enough to read inline in ~4 `cat` batches via bash; do NOT delegate the reading, style lives in the exact prose.
|
||||
|
||||
Read the author's own meta-skill FIRST if one exists (`writing-for-agents` for mattpocock — it states his levers explicitly), then the corpus practicing them.
|
||||
|
||||
## Style DNA checklist (mattpocock; generalise per author)
|
||||
|
||||
- **Leading words**: compact pretrained concepts repeated as tokens, never sentences (_red_, _seam_, _fog of war_, _brief_, _frontier_). Define each once, reuse everywhere. Prefer words already in the author's other skills so the corpus shares one language.
|
||||
- **Vocabulary-first**: bolded glossary terms, then used exactly; explicit "_Avoid_: component, service".
|
||||
- **Positive phrasing**: prohibitions rewritten as target behaviours; keep a prohibition only when no positive form exists (hard guardrails).
|
||||
- **Sharpened completion criteria**: fuzzy bounds ("understanding reached") → checkable + exhaustive ("frontier empty and nothing sits blocked", "one artefact per question, counted before sending").
|
||||
- **Zero history**: no provenance, attribution, or absorbed-from lineage in the body.
|
||||
- **No-op hunting**: delete harness defaults and anything the environment already confesses (formats, rendering, caps). Gotchas the environment CANNOT confess (exclusive dialogs, Esc-abort semantics) stay inline — they're the cache worth keeping.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Read entire corpus; tabulate levers with per-skill evidence before writing.
|
||||
2. Rewrite preserving settled behaviour 1:1 — voice only; diff the two versions for semantic drift.
|
||||
3. Audit pass: grep the new text for history words (extracted/absorbed/from/upstream), machine-specific constants, repo-specific paths.
|
||||
4. Run repo checks, commit as `style(<scope>)`, note the lever→instance mapping in the PR comment so the reviewer can grade the emulation.
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
name: dankclaw-boot-line-window
|
||||
description: "Change a Dankclaw kernel command-line parameter safely: tracked grub + installer, measured before/after A/B on live LLM traffic, post-boot gates."
|
||||
---
|
||||
|
||||
# Dank fleet kernel boot-line window (Dankclaw pattern)
|
||||
|
||||
Kernel command-line state is **host state the deployer never owns**. The repo law
|
||||
("all edits through git") still applies: fetch the live file, commit it, edit the
|
||||
tracked copy, apply with a script.
|
||||
|
||||
## Source of truth
|
||||
- `dankclaw/host/grub` — tracked copy of `/etc/default/grub`.
|
||||
- `scripts/set-dankclaw-bootline.sh` — the only door: diff → `sudo install -m 0644 -o root -g root` →
|
||||
`sudo update-grub` → print the **default `grub.cfg` entry** → reboot only on `--reboot`.
|
||||
Refuses any host whose `hostname -s` lacks `dankclaw`.
|
||||
- Decision record: `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
|
||||
- `deploy-plan.py` maps `dankclaw/*` subdirs explicitly; a new `dankclaw/host/` path plans nothing (verify with a `--dry-run`).
|
||||
|
||||
## Procedure
|
||||
1. **Capture the before-arm measurement first** — you cannot get it back after the reboot.
|
||||
2. Fetch provenance byte-exact: `ssh dankclaw 'sudo cat /etc/default/grub' > dankclaw/host/grub` (never retype; a transcription dropped a leading `0`).
|
||||
3. Commit provenance, edit the tracked copy, commit the flip, push (**Gitea lives on Dankclaw — pull on the host before rebooting**), then `ssh dankclaw 'cd ~/serverAppConfigs && git pull --ff-only origin main'`.
|
||||
4. Install **without** `--reboot` first: it prints the `grub.cfg` default entry; confirm the token there before the irreversible step.
|
||||
5. `ssh dankclaw 'sudo systemctl reboot --no-block'`.
|
||||
6. **Poll on `/proc/uptime` + cmdline, never on ping** — the NIC answers ICMP for a minute after shutdown and sshd reset looks like "reachable".
|
||||
Gate set: `systemctl --failed` count 0 · `systemctl is-system-running` running · `/proc/cmdline` token · `ls /sys/kernel/iommu_groups | wc -l` · `cat /sys/kernel/iommu_groups/*/type | sort | uniq -c` · `systemctl is-active frank-dragoon frank-zealot` · `curl 127.0.0.1:7770/health` (listener binds only after the full load — poll up to 10 min).
|
||||
|
||||
## Measurement discipline (this is where windows get lied about)
|
||||
- Both arms must be **byte-identical traffic**. Engine-reported token counts are the identity proof (e.g. 59,024 prefill / 21,568 needle both boots).
|
||||
- `/tmp` is **tmpfs**: the reboot deletes `/tmp/halogen-eval`. Rebuild with `scripts/stage-frank-dragoon-eval.sh` (corpus 2,891 lines / 1,287,656 B is the check) — and run it via `bash -lc`, `uv` is in `~/.local/bin`, absent from a non-login PATH.
|
||||
- The 51.2 GB halogen n-gram table is page cache (`Pin::None`). A fresh boot has it cold and a cold table throttles prefill → `cat` it once (~40 s) before measuring. `WARM=1` in the staging script does this.
|
||||
- **Sampled decode on this engine is a useless single-sample instrument**: three repeats of the same request gave 31.8 / 46.9 / 47.8 t/s. Only prefill (cold) and needle-hit carry signal; anything else needs ≥3 repeats.
|
||||
- Bench law still bites: nothing else on the GPU; measuring *through* the live service is how dragoon/zealot are measured (a scratch server cannot share the card).
|
||||
|
||||
## Traps
|
||||
- `edit` `PUT N.:` can **insert instead of replace** — verify the range shrank, then `CUT` the leftover.
|
||||
- Body rows beginning with a markdown bullet (`- `) get auto-prefixed and land as `+- `; rewrite the block with `write` when that happens.
|
||||
- A long reboot fires `LlmEndpointDown` → Signal; Alertmanager has **no persistence**, so every active alert gets a fresh `startsAt` after boot — check `query_range` history before blaming the window for an alert.
|
||||
- `sudo podman ps` shows nothing for Dankclaw's rootless stack; `systemctl is-active caddy` is meaningless for quadlets. Use the public endpoints or `systemctl --user` as `dank`.
|
||||
|
||||
## Worked example (2026-10-18)
|
||||
`amd_iommu=off` → `iommu=pt` to restore the XDNA2 NPU. Result: cold prefill 1,585.1 → 1,616.0 t/s (**+2.0 %**, i.e. no penalty), needle hit/hit, `amdxdna` bound `0000:c6:00.1` (group 26 identity, PASID width 10). The 13–16 % figure it replaced had never been measured locally. Deferred on purpose: `pp_dpm_fclk` pin (nothing co-runs yet) and PerfOpt (needs Linux 7.4; Ubuntu 26.04 caps at `7.0.0-38.38`).
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
name: dankclaw-expose-service
|
||||
description: "Expose a new service at *.franksdank.cloud on Dankclaw: quadlet container, Caddy site block, Pi-hole LAN record (custom.list), Porkbun public record; includes API vhost/volume gotchas proven on the ntfy deploy."
|
||||
---
|
||||
|
||||
# Expose a service on Dankclaw (franksdank.cloud)
|
||||
|
||||
Proven end-to-end 2026-10-11 with ntfy. Four layers, each with a known trap.
|
||||
|
||||
## 0. Survey first
|
||||
```bash
|
||||
ssh dankclaw 'ls ~/.config/containers/systemd/; grep -nE "franksdank|anubis" /home/dank/Containers/caddy/Caddyfile'
|
||||
```
|
||||
- Caddy runs as quadlet `caddy.container`; config at `/home/dank/Containers/caddy/Caddyfile` (NOT `/etc/caddy` on host).
|
||||
- Sites fronted by `anubis:8923` get bot-challenge. **CLI/API publishers (ntfy, webhooks, git) MUST bypass anubis** — use the `gitea`/`ai` pattern (direct `reverse_proxy`).
|
||||
|
||||
## 1. Quadlet container (user `dank`)
|
||||
Write `~/.config/containers/systemd/<name>.container`:
|
||||
```ini
|
||||
[Unit]
|
||||
Description=<name>
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
[Container]
|
||||
ContainerName=<name>
|
||||
Image=<image>
|
||||
Network=dankclaw_default
|
||||
Volume=<name>-cache:/var/lib/<name>
|
||||
[Service]
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
```
|
||||
`Network=dankclaw_default` ⇒ Caddy reaches it as `<name>:<port>`. Then `systemctl --user daemon-reload && systemctl --user start <name>.service`.
|
||||
Rootless podman: bare containers never boot-persist — quadlet is mandatory.
|
||||
|
||||
## 2. Caddy route
|
||||
Append site block (wildcard `*.franksdank.cloud` cert already issued via Porkbun DNS-01; no new cert work):
|
||||
```
|
||||
<name>.franksdank.cloud {
|
||||
reverse_proxy <name>:<port>
|
||||
}
|
||||
```
|
||||
Validate, then reload in-place:
|
||||
```bash
|
||||
podman exec caddy caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
|
||||
podman exec caddy caddy reload --config /etc/caddy/Caddyfile
|
||||
```
|
||||
|
||||
## 3. LAN DNS (Pi-hole v6, container `pihole`)
|
||||
Traps, in order of wasted turns:
|
||||
- API is NOT at `127.0.0.1:8083/api` on the host port path — requests need `-H "Host: pi.hole"`.
|
||||
- `[[dns.records]]` in pihole.toml and `gravity.db` `domain` table: absent/unused on this box.
|
||||
- **Working location:** `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list` (dnsmasq addn-hosts), format `IP FQDN`:
|
||||
```bash
|
||||
echo "192.168.0.143 <name>.franksdank.cloud" | sudo tee -a /home/dank/Containers/pihole/etc-pihole/hosts/custom.list >/dev/null
|
||||
podman exec pihole pihole restartdns
|
||||
```
|
||||
|
||||
## 4. Public DNS (Porkbun, for off-LAN clients)
|
||||
Keys: `/home/dank/Containers/caddy/env` (`PORKBUN_API_KEY`, `PORKBUN_API_SECRET_KEY`).
|
||||
```bash
|
||||
curl -sL -X POST https://api.porkbun.com/api/json3/domain/api/create/franksdank.cloud \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"apikey\":\"$K\",\"secretapi\":\"$S\",\"name\":\"<name>\",\"type\":\"A\",\"content\":\"71.227.227.172\",\"ttl\":\"600\"}"
|
||||
```
|
||||
- `json3` path + `-L` (302-redirects to v3). Public IP: 71.227.227.172 (router forwards 443 → Dankclaw).
|
||||
- Porkbun edge throws intermittent 503s; retry later or do it in the UI. LAN path works without this record.
|
||||
|
||||
## 5. Verify
|
||||
```bash
|
||||
# from Dankclaw (host): getent hosts <name>.franksdank.cloud → 192.168.0.143
|
||||
# workstation w/ stale resolved negative cache: use curl --resolve <name>.franksdank.cloud:443:192.168.0.143
|
||||
# workstation's own `sudo resolvectl flush-caches` needs a terminal (no passwordless sudo) — negative cache expires on its own
|
||||
curl -sf --resolve <name>.franksdank.cloud:443:192.168.0.143 https://<name>.franksdank.cloud/
|
||||
```
|
||||
Publish-style test for ntfy inside container (CLI flags vary by version; raw HTTP is version-proof):
|
||||
```bash
|
||||
podman exec <name> wget -qO- --header="Title: smoke" --post-data="up" http://localhost:<port>/<topic>
|
||||
```
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
name: dankclaw-homepage-asset-verify
|
||||
description: "Add/verify homepage assets on Dankclaw: Caddy @data allowlist, deploy copy step, e2e panel census, live content-type check."
|
||||
---
|
||||
|
||||
Every new top-level homepage asset directory (e.g. `shared/homepage/img/`, `shared/homepage/sfx/`) needs FOUR coordinated changes, all in one commit:
|
||||
|
||||
1. **Caddy allowlist** — `dankclaw/Caddyfile` `@data path` matcher must include `/<dir>/*`. Without it the post-auth `@home` catch-all answers with `index.html` (`200 text/html` for a PNG/OGG — silent breakage).
|
||||
2. **Deploy copy** — `scripts/deploy-remote.sh` `deploy_homepage()` copies `shared/homepage/<dir>/` with the `cp -r ... /srv/homepage/` merge pattern beside the fonts/img/sfx blocks.
|
||||
3. **Test pins** — `tests/homepage_caddy.bats` grep-pins the new `@data` path; if the asset changes panel rendering, the e2e census in `tests/e2e/specs/10-homepage.spec.ts` must count the new render style (it derives expected names from services.json: `.panel .name` for normal cards, `.mi-name` for fused Model Information cards — a new fused component breaks the panel census otherwise, which is exactly how the 2026-10-08 Protoss deploy failed its e2e gate).
|
||||
4. **Deploy both services** — `deploy.sh dankclaw --service homepage` and `--service caddy` (Caddyfile change), then live-verify from a browser tab: `fetch('/<dir>/<file>')` must return `200` with the correct `content-type` (`image/jpeg`, `audio/ogg`), not `text/html`.
|
||||
|
||||
Checker note: `status.json` names follow `services.json` automatically (the deployed checker re-reads it via the homepage deploy's restart of homepage-status.timer), so renaming cards needs no checker change.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: dankclaw-model-inventory
|
||||
description: "Read-only census of downloaded models on Dankclaw: GGUF store, ComfyUI container mount, dead-config trap"
|
||||
---
|
||||
|
||||
Answer "what models are on Dankclaw" without guessing roots. All probes are read-only ssh (`ssh -o BatchMode=yes dankclaw '…'`).
|
||||
|
||||
## Roots (in order)
|
||||
|
||||
1. **LLM GGUF store:** `/home/dank/models/` — one dir per model, main GGUF + `mmproj-*` (multimodal projector) + `mtp-*` (speculative-decode draft) satellites. `~/.share/models` does NOT exist on Dankclaw (that's the workstations' convention).
|
||||
2. **ComfyUI runtime store:** `/home/dank/comfyui-data/models/` — category dirs (`diffusion_models/`, `text_encoders/`, `vae/`, `loras/`, …). This is what the running stack actually uses.
|
||||
|
||||
## ComfyUI topology (the trap)
|
||||
|
||||
- Runtime = podman container `comfyui` (image `localhost/comfyui-rocm:gfx1151`, user unit `comfyui.service`). Mounts: `~/comfyui-data/{models,custom_nodes,input,output}` → `/opt/ComfyUI/...`.
|
||||
- The host checkout `~/ComfyUI/` is NOT the runtime. Its `extra_model_paths.yaml` (base_path `/home/dank/models` + category keys) is dead config from a pre-container setup — never read by the container. Do not report those empty category dirs as "ComfyUI has no models".
|
||||
|
||||
## Census commands
|
||||
|
||||
```bash
|
||||
# LLM store: sizes + names in one pass
|
||||
ssh -o BatchMode=yes dankclaw 'find ~/models -maxdepth 2 -type f \( -iname "*.gguf" -o -iname "*.safetensors" \) -printf "%10s %p\n" | sort -k2'
|
||||
|
||||
# ComfyUI: container mount truth
|
||||
ssh -o BatchMode=yes dankclaw 'find /home/dank/comfyui-data/models -type f -size +1M -printf "%10s %P\n" | sort -k2; du -sh /home/dank/comfyui-data'
|
||||
|
||||
# confirm runtime + mounts before trusting any path claim
|
||||
ssh -o BatchMode=yes dankclaw 'podman ps --format "{{.Names}} {{.Image}} {{.Status}}"; podman inspect comfyui --format "{{range .Mounts}}{{.Source}} -> {{.Destination}}{{\"\n\"}}{{end}}"'
|
||||
```
|
||||
|
||||
## Anomaly flags
|
||||
|
||||
- A `.gguf` of ~1 KB = git-LFS pointer / failed download, not a model (seen: MiMo-Q8_0 stub). Report as broken, offer refetch or `.AGENTTRASH`.
|
||||
- Size sanity vs Strix Halo 32 GiB unified memory: bf16 diffusion+TE pairs near 30 GiB can't co-reside with the serving LLM; int8/w4a8 quants are the working set.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: dankclaw-pihole-lan-dns-record
|
||||
description: "Add/repair a LAN DNS record for a new *.franksdank.cloud service on Dankclaw's Pi-hole v6 (hosts/custom.list + repo addn-hosts, plus the public Porkbun A record side) — and triage \"LAN names stopped resolving\" including the FTL boot-time record-load failure."
|
||||
---
|
||||
|
||||
# Dankclaw Pi-hole LAN DNS record
|
||||
|
||||
## What this fixes
|
||||
|
||||
A new `*.franksdank.cloud` service resolves publicly (via Porkbun → `franksdank.cloud` A record → Caddy on Dankclaw `192.168.0.143`) but **does not resolve on the LAN**. Pi-hole is the LAN DNS server (`dankclaw_default` bridge, published port 53) and serves local records only from files it imports.
|
||||
|
||||
## Live truth
|
||||
|
||||
- **The repo owns the records.** `dankclaw/pihole/addn-hosts` (hosts-style lines: `192.168.0.143 <name>`) is copied by `deploy-remote.sh` to `/home/dank/Containers/pihole/addn-hosts` and bind-mounted **read-only** into the container at `/etc/pihole/hosts/addn-hosts` (`Volume=/home/dank/Containers/pihole/addn-hosts:/etc/pihole/hosts/addn-hosts:ro` in `dankclaw/pihole/quadlet/pihole.container`). Add the name there, in the repo, then deploy — that is the durable path. `tests/pihole.bats` validates every line (IPv4, LAN range, name suffix, Caddyfile cross-check).
|
||||
- `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list` also exists on the host (older mechanism, mode `640` owned by the container uid, root reads it with `sudo`). It is **not** the repo's owner file — do not hand-edit it as the fix.
|
||||
- `pihole.toml` carries a `dns.hosts = [ "192.168.0.143 …", … ]` array too (FTL's own in-config records). Both sources exist; if LAN names fail, neither is being applied.
|
||||
- Pi-hole runs as a rootless quadlet named `pihole` — query it with `podman exec pihole …`, never `sudo podman` (per-user storage law).
|
||||
- `pihole.franksdank.cloud` resolves through Pi-hole itself (loop/edge), so a vhost record pointing at `192.168.0.143` is correct for every Caddy-fronted service.
|
||||
|
||||
## Procedure (repo-first)
|
||||
|
||||
1. Append the line to `dankclaw/pihole/addn-hosts`:
|
||||
```
|
||||
192.168.0.143 <name>.franksdank.cloud
|
||||
```
|
||||
One record per vhost; keep it in sync with the Caddyfile (the bats seam cross-checks the two).
|
||||
2. `./tests/run.sh` (the `pihole` seam validates the file), commit, push.
|
||||
3. Deploy so the copy step runs: `./scripts/deploy.sh dankclaw --service pihole`.
|
||||
4. Verify from the box: `getent hosts <name>.franksdank.cloud` → `192.168.0.143`.
|
||||
5. Public edge record (optional; skip when every subscriber is LAN-only): Porkbun `POST /api/v3/json` with `apikey`/`secretapikey`/`command=add`/`type=A`/`hostname=<name>`/`data=76.49.118.136` (see `dankclaw-porkbun-edge`).
|
||||
|
||||
Client-side proof (avoid the resolver trap):
|
||||
|
||||
```
|
||||
sudo resolvectl flush-caches # systemd-resolved otherwise keeps the negative cache
|
||||
curl -sv --max-time 5 https://<name>.franksdank.cloud 2>&1 | grep 'Connected to'
|
||||
```
|
||||
|
||||
Expected: `Connected to <name>.franksdank.cloud (192.168.0.143)` — LAN route, not the public IP.
|
||||
|
||||
## Triage: every LAN name broke at once (2026-10-19 law)
|
||||
|
||||
Signature: **all** `*.franksdank.cloud` return NXDOMAIN / Playwright `ERR_NAME_NOT_RESOLVED` while **external names still resolve**. The external-OK/local-fail split is the tell: FTL is answering external names from cache and holds **no local records at all** after a container/host restart — the mounts and files are intact and readable inside the container (`ls -l /etc/pihole/hosts/`, `grep -c . …/addn-hosts`), `dns.hosts` in `pihole.toml` is complete. The mechanism is unestablished; do not go hunting for a lost file.
|
||||
|
||||
```
|
||||
ssh dankclaw 'getent hosts grafana.franksdank.cloud' # empty => FTL, not the client
|
||||
ssh dankclaw 'podman exec pihole pihole restartdns reload' # fixes it in seconds
|
||||
```
|
||||
|
||||
Distinguish before acting: a **single** client failing while others work is FTL rate-limiting that client — `podman exec pihole grep Rate-limiting /var/log/pihole/FTL.log`; a `10.0.2.x` client address is a pasta container's DNS forwarder (SNAT), not a machine. Full causal page: `serverAppConfigs-docs/Infrastructure/Pi-hole.md` (**Records** pitfall + Failure modes table).
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do NOT insert local records into SQLite. v6's `gravity.db` `domain` table has NO `type=2` custom-DNS rows in this build.
|
||||
- `pihole restartdns reload` re-imports the hosts files **and** `dns.hosts`; that is the remedy for the boot-load failure above. Plain `pihole restartdns` also restarts the resolver — brief LAN-wide DNS blip.
|
||||
- `dig @<ip> <name>` from the workstation bypasses systemd-resolved — use it to check FTL's answer directly (`+short @192.168.0.143`).
|
||||
|
||||
## Cross-reference
|
||||
|
||||
- Fleet DNS flow: `dankclaw/Caddyfile` (Caddy binds `:443` on the host, serves `*.franksdank.cloud` vhosts)
|
||||
- Pi-hole service page: `serverAppConfigs-docs/Infrastructure/Pi-hole.md`, flow page `Infrastructure/LAN-DNS.md`
|
||||
- Public edge: managed skill `dankclaw-porkbun-edge`
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
name: dankvoice-bats-serialization
|
||||
description: "Run dankvoice bats suites without hangs or false failures: serialize dist rebuilds, never edit src mid-run, and rebuild before committing the tracked bundle."
|
||||
---
|
||||
|
||||
Use when running the dankvoice `bats tests/` suite on the shared box, when a suite run hangs or times out, or when the "committed dist is byte-identical to a fresh build" pin test fails after a commit.
|
||||
|
||||
## The two serialization hazards
|
||||
|
||||
### 1. `setup()` rebuilds `dist/` from `src/`
|
||||
Every test runs `cd "$REPO" && bun run build` in `setup()`. Consequences:
|
||||
|
||||
- **Never edit `src/` while a suite is running.** The next test's `setup()` compiles whatever is on disk at that instant, so an in-flight edit gets attributed to the run and produces a failure with no owner. Finish the gate, then write.
|
||||
- **Two suites on the same checkout race on `dist/`.** Serialize: run one, wait for its report, then start the next. Multiple worktrees of the repo each build their own `dist/` and do not collide.
|
||||
- Suite runtime is ~85 s for 55 tests; a backgrounded job with a 600 s deadline is the right shape (`timeout: 600`, `async: true`). Never poll it — `setup()` output lands in the conversation when the job ends.
|
||||
|
||||
### 2. `dist/` is tracked, and a pin test enforces it
|
||||
One test rebuilds and compares against the committed bundle. Therefore the commit order is:
|
||||
|
||||
```bash
|
||||
bun run build # BEFORE staging
|
||||
git add -A && git commit -m '…'
|
||||
bats tests/ # gate AFTER the commit
|
||||
```
|
||||
|
||||
Committing `src/` changes without rebuilding first fails the pin test (`not ok … byte-identical to a fresh build`) — the tracked bundle is the *previous* state. If that happens on an unpushed branch, fix it by amend rather than a follow-up commit:
|
||||
|
||||
```bash
|
||||
bun run build && git add dist && git commit --amend --no-edit
|
||||
```
|
||||
|
||||
Verify freshness any time the pin is in doubt:
|
||||
|
||||
```bash
|
||||
bun run build >/dev/null 2>&1 && git status --porcelain dist # empty = committed == fresh build
|
||||
```
|
||||
|
||||
## If a backgrounded gate job vanishes
|
||||
A crash can kill the job before its report; `read proc://bg_N` returning "not found" means it is gone, not that it failed. Reconstruct what landed before resuming — `git log --oneline -3`, `git status --porcelain`, then the freshness check above — and rerun the suite; the commit may well have succeeded while the gate never ran.
|
||||
|
||||
## Gotchas
|
||||
- `DV_HOLD` is seconds (`sleep "${DV_HOLD:-0.1}"`), so `DV_HOLD=3` really holds the stub for three seconds.
|
||||
- Isolation tests must not count shared state: any assertion that sweeps `$TMPDIR` or `~/.omp/cache` observes every other live session on the box (each narrating session creates `dankvoice-*` scratch dirs). Point `TMPDIR` at a private directory instead — see the `cutting off mid-utterance leaves no orphaned audio behind` test.
|
||||
- `bun run build` does not typecheck. Run `bunx tsc --noEmit` separately when moving code between modules; the bundler happily emits a bundle that references a deleted identifier.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
name: dankvoice-bundle-swap-red-proof
|
||||
description: "Prove a dankvoice behavior test is red against the pre-change bundle by pointing DV_EXT at git show HEAD:dist/index.js, with the driver extracted from the bats heredoc into a pinned scratch directory. Use before claiming a new bats assertion is red-capable."
|
||||
---
|
||||
|
||||
Prove a new `tests/behavior.bats` assertion is **red-capable** — that it fails against the *previous* build, not just passes against the new one — without stashing, branching, or editing `src/`.
|
||||
|
||||
## Why
|
||||
|
||||
`tests/behavior.bats` is a black-box suite: the plugin bundle is loaded through `DV_EXT`, so a *different bundle* is the red side. Stashing the fix would break the `dist` byte-identity pin test and the shared worktree.
|
||||
|
||||
## Working recipe
|
||||
|
||||
```bash
|
||||
cd ~/Projects/dankvoice # or the feature worktree: ~/Projects/wt-dv-*
|
||||
bun run build # current src = the green side
|
||||
git show HEAD:dist/index.js > /tmp/old-dist.js # the red side (pre-change bundle)
|
||||
|
||||
W=/tmp/dv-redproof # PINNED name; see traps below
|
||||
rm -rf "$W" && mkdir -p "$W/bin"
|
||||
cat > "$W/player.sh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "$*" >> "$DV_STUB_LOG"
|
||||
for arg in "$@"; do case "$arg" in *.wav) cp "$arg" "$DV_COPY" ;; esac; done
|
||||
sleep "${DV_HOLD:-0.1}"
|
||||
SH
|
||||
chmod +x "$W/player.sh"
|
||||
ln -s "$W/player.sh" "$W/bin/pw-play"
|
||||
printf '#!/usr/bin/env bash\nprintf "116\\talsa_output.usb-KT.0\\taudio/sink\\t*\\n"\n' > "$W/bin/wpctl"
|
||||
chmod +x "$W/bin/wpctl"
|
||||
# The driver lives inside the bats file as a heredoc: extract it verbatim.
|
||||
end=$(grep -n '^TS$' tests/behavior.bats | cut -d: -f1)
|
||||
sed -n "86,$((end - 1))p" tests/behavior.bats > "$W/driver.ts"
|
||||
|
||||
export DV_EXT="$PWD/dist/index.js" DV_STUB="$W/player.sh" DV_STUB_LOG="$W/player.log" \
|
||||
DV_COPY="$W/played.wav" DV_OMPCONFIG=/dev/null DANKVOICE_CONFIG="$W/settings.json" \
|
||||
DV_MODELS="$W/models.yml" DV_WPCTL=x DV_SELECT=""
|
||||
for ext in /tmp/old-dist.js "$PWD/dist/index.js"; do
|
||||
rm -f "$W/player.log" "$DANKVOICE_CONFIG"
|
||||
DV_EXT="$ext" bun "$W/driver.ts" <scenario> 2>&1 | grep -E '^PLAYS|^CALLS|^NOTIFY|^SETTINGS'
|
||||
done
|
||||
```
|
||||
|
||||
The scenario name is the driver's third argv (`speak`, `auto`, `repeat`, `humanizecycle`, `badurl`, …); the bats `-f` test body's env (`DV_PLAYER_BIN`, `DV_SETTINGS_JSON`, `DANKVOICE_VERBOSITY`, …) is passed the same way. `DV_SELECT` holds the menu answers, `|`-separated, consumed in order.
|
||||
|
||||
## Traps, each one cost a turn
|
||||
|
||||
- **The scratch directory must be pinned by name.** `ls -dt /tmp/tmp.*` picks up *other sessions'* temp dirs, and then `player.sh` and `driver.ts` live in different directories — the run silently does the wrong thing.
|
||||
- **The stub must append to `$DV_STUB_LOG`.** The driver counts player invocations from that file; a stub that only `sleep`s reports `PLAYS 0` for *both* bundles and the whole proof reads as "fix does nothing".
|
||||
- **`DV_EXT` is required.** Missing it yields `Cannot find package 'undefined'` from the driver, which looks like a broken driver, not a missing env var.
|
||||
- **`DV_SETTINGS_JSON` supports `__PORT__`** (the driver substitutes the fixture port), so a preseed can name a prefixed route — needed whenever the test is about *which* endpoint got resolved.
|
||||
- **Never run this while a `bats` run is in flight** in the same repo: `setup()` runs `bun run build`, so two runs clobber `dist/` and the pin test.
|
||||
|
||||
## Reading the result
|
||||
|
||||
Red side must show the *defect* (`PLAYS 0`, `644`, a chat `CALL` with the wrong prefix, `Invalid URL`), green side the fix — assert on printed markers only (`PLAYS`, `CALLS`, `CALL <url>`, `NOTIFY`, `SETTINGS`). `CALLS` counts `/audio/speech` calls only; the rewrite call appears as its own `CALL /…/chat/completions … sys=<full system prompt>` line, which is how prompt-level knobs (verbosity, heckle) are observed.
|
||||
|
||||
Prefer this over a synthetic leak: if the check already caught a real leftover/failure once, that event *is* the sensitivity proof — say so in the commit message instead of faking one.
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
name: dankvoice-epithet-pool-audit
|
||||
description: "Grow or audit the dankvoice 1000-name epithet pool: gender classes, individual eviction, typo sweep, safe regeneration into framing.ts."
|
||||
---
|
||||
|
||||
# Dankvoice epithet pool audit
|
||||
|
||||
Use when adding names to, reclassifying, or cleaning `src/framing.ts` `EPITHETS_*` in `dank/dankvoice`. The pool is law: exactly 1000 distinct names, curated at build time, never user-editable, "the commons, never the characters".
|
||||
|
||||
## Classification rules (proven traps in parentheses)
|
||||
- **man**: gendered lexicon (king, lord, priest, uncle...) + tokens ending `man|men|master` — but never `human|inhuman|nonhuman`. Compounds like `quartermaster`, `hangwoman` are single tokens: token-equality checks miss them, use suffix rules.
|
||||
- **woman**: lexicon (queen, sister, daughter...) + tokens ending `woman` + an EXPLICIT `-ess` title list (baroness, countess, prioress, canoness, chartress, heiress, priestess...). Never a bare `-ess` suffix match: `endless`, `harmless`, `fortress` all end in "ess".
|
||||
- **Possessives follow the head noun**: `lieutenant of the king's hull` is a lieutenant → neutral commons; `count's armsman` is an armsman → man. Do not gender by words inside `'s` phrases.
|
||||
- Modern-neutral offices stay commons: hero, governor, wizard. `witch/lass/bride` are female-coded; `warlock` male-coded.
|
||||
- Regex trap for absence checks: `he\b` matches inside `the`; token-split before matching.
|
||||
|
||||
## Individual eviction law
|
||||
Evict: named fictional/historical individuals (duke caladan, uncle toby, pyrrhus, usul, endymion, skynet, moya-born, medusa), named protagonist ships (rocinante), and real authors in universe clothing (strugatsky, niven).
|
||||
Keep: species, orders, houses, orgs/companies (sardaukar, jedi, cyberdyne), places (kessel; `falconer of the caladan eyries` is fine — the person problem was only `duke caladan`), institutions (zekma, karass).
|
||||
Replace each eviction with the generic office beside it, class-stable, and unique against the pool.
|
||||
|
||||
## Invariants tests pin (breaking any of these goes red)
|
||||
- 1000 names, all distinct.
|
||||
- Sentinel literals must survive: `hangman of the holiday` (man), `high priestess of the ear moon` (woman), `comrade operator` (other).
|
||||
- Pool firsts feed pool[0] probe tests — union order is MAN-first / WOMAN-first / OTHER; keep source order within classes so firsts stay: `chairman of the presidium of processors` / `hangwoman of the public convenience` / `comrade operator`.
|
||||
- Class counts (86/15/899 as of 2026-10) may churn but must be reported; tests never pin them.
|
||||
|
||||
## Regeneration
|
||||
Pool lives as three generated arrays between markers `/** Male-coded` and `/** The unions` in `src/framing.ts`. Splice new arrays between those markers (python splice of a generated `const NAME = [ "x", ... ] as const;` block is fine; do not hand-edit 1000 lines). Then `bun run build` (dist pin test byte-compares), `bun run typecheck`, and the full `bats tests/` — the class sweeps are exhaustive `Math.random`-cycled walks (each ~200 s; suite ~17 min; never two bats processes at once).
|
||||
|
||||
## Fleet cutover after merge
|
||||
main checkout ff-only → `omp plugin upgrade omp-dankvoice` → `cmp` `~/.omp/plugins/node_modules/omp-dankvoice/dist/index.js` against merged `dist/index.js`.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: dankvoice-menu-knob-workflow
|
||||
description: "Add a persisted /say knob to dankvoice and test its menu row through the bats driver (DV_SELECT picks, INPUT stub, dist-pin rebuild gate)."
|
||||
---
|
||||
|
||||
Adding a setting to `dankvoice` (`~/Projects/dankvoice`, omp speech extension) means five touchpoints plus one behavioral test. Skip one and TypeScript or a silent drop catches it; skip the test and the row ships dead.
|
||||
|
||||
## The five touchpoints (order matters)
|
||||
|
||||
1. `src/settings.ts` — field on `Settings` (optional) **and** a type-guarded line in `readSettings`. A field without a guard line is written to disk and never read back; tsc stays clean, so nothing else catches it.
|
||||
2. `src/config.ts` — `Config` field (required, defaulted), resolution line, entry in the returned object, and the knob documented in the file header comment (that header is the README's pointer for "full knob list").
|
||||
- Convention: explicit env wins over settings (`env.X === "1" ? true : env.X === "0" ? false : (settings.x ?? false)`); booleans use `1`/`0`, enumerations use the literal value.
|
||||
3. `src/index.ts` — `persistSettings()` key list (a knob missing there never survives the session) **and** the getter in the `new Speaker({...})` deps object.
|
||||
4. `src/utterance.ts` — getter on `SpeakerSettings` if the Speaker reads it at speak time.
|
||||
5. `src/menu.ts` — the row. Rows carry their own `run`; never dispatch on `choice === options[N]`.
|
||||
|
||||
Gate at the call site, not the prompt, when a mode must not apply to operator-typed text: `Speaker.speak(ctx, text, thinking, auto)` — `auto=true` only from `message_end`/`agent_end`; `/say <text>` passes nothing. Prompt-only gating leaks.
|
||||
|
||||
## Test the row behaviorally, never as a config assertion
|
||||
|
||||
`tests/behavior.bats` drives the built `dist/index.js` against a mock harness. Selects and typed text both come from one pipe-separated queue: `DV_SELECT="row label|sub choice|typed text"`, consumed in call order; a consumed-exhausted `select` returns `options[0]`, so always name the row by its **rendered label** (`"Verbosity: normal (change)"`, `"Intro: none (change)"`).
|
||||
|
||||
- `ui.input(title, placeholder)` in the driver mock logs `INPUT <title> <placeholder>` and takes the next queue entry. If the mock lacks `input`, no typed value was ever testable — add the stub before writing the test.
|
||||
- Scenario pattern that proves a pick works (`case "knobs"`): `say.handler("menu")` → `say.handler("on")` → two `agentEnd(answer(...))` with `settle()` between. Then assert on observable output, not on the settings file:
|
||||
- prompt-shaping knob: `[[ "$output" == *"ONE spoken sentence"* ]]` (directive reached the rewrite request)
|
||||
- framing: `[[ "$output" == *"input=Update, chief!"* ]]`
|
||||
- throttle/framing counting: `[[ "$output" == *"CALLS 1"* ]]`
|
||||
- A guard test whose negative case would pass trivially must also assert its premise, e.g. truncation: assert `CALLS 1` so "no outro" cannot be a green caused by nothing ever being cut.
|
||||
|
||||
Red first, in a batch: write all the new `@test` blocks, run `bats tests/behavior.bats -f "<pattern>"`, confirm the new ones fail, then implement.
|
||||
|
||||
## Gates and gotchas
|
||||
|
||||
- **Two `bats` runs at once corrupt each other**: `setup()` runs `bun run build`, so a concurrent run rebuilds `dist/` mid-suite and the byte-identical pin test fails spuriously. Serialize.
|
||||
- **Edit `src/` → rebuild before commit**: the pin test compares committed `dist/` to a fresh build. Sequence is `bun run build && bunx tsc --noEmit && bats -f <pattern>`, then `git add -A` (dist included). A failure there is normally a stale bundle, not a bad assertion.
|
||||
- `VAR=value some_function args` in bats does reach the child process, so knob tests need no `export`.
|
||||
- `edit` tool: inserting after a line whose content you know but that the current tag never displayed gets refused — the refusal echoes the real line; if it matches intent, re-issue the identical edit instead of re-reading.
|
||||
- Ship: branch `feat/<issue#>-<slug>` off fresh `origin/main`, one commit per knob, PR targets `main`, merge only on Frank's explicit OK (`do=merge`, `merge_style=no-ff`), then `omp plugin upgrade dankvoice` and `cmp` installed `dist/index.js` against the merged one.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
name: dankvoice-silence-triage
|
||||
description: "Triage silent or late dankvoice auto-speak on the fleet: arming key, /say grammar, endpoint/sink, event granularity, humanizer latency, plus the black-box bats driver."
|
||||
---
|
||||
|
||||
# Triage "dankvoice is silent" on the fleet
|
||||
|
||||
Symptom that recurs: auto-speak only speaks when `/say <text>` is typed
|
||||
explicitly. Work the gates in this order; each has an objective check.
|
||||
|
||||
1. **Armed?** `cat ~/.omp/cache/dankvoice.json | jq .auto` — must be `true`
|
||||
(or `DANKVOICE_AUTO=1`). Arming is read at **startup only** and `/say on`
|
||||
never persists, so a session started before the key was added is silent even
|
||||
though fresh sessions work. `/say start` is *not* a subcommand: unknown
|
||||
arguments are spoken verbatim (it read out the word "start"). `/say on` is
|
||||
the verb; `/say status` must print `auto-speak on`.
|
||||
2. **Endpoint + sink?** `/say status` must show a real endpoint (not
|
||||
`NO ENDPOINT`) and the saved sink must exist:
|
||||
`pw-dump | grep -o 'alsa_output\.[^"]*' | sort -u`.
|
||||
3. **Did it synthesize?** `pgrep -a pw-play` during a turn: one process per
|
||||
segment, `--target <sink>`, wav under `/tmp/dankvoice-*/say-<epochms>.wav`.
|
||||
One temp **directory per utterance chain** — count dirs to count utterances,
|
||||
pids to count segments.
|
||||
4. **Event granularity (the structural cause).** `agent_end` fires once per
|
||||
*prompt* when the loop settles and is skipped when the harness maps a
|
||||
non-terminal settle (`willContinue`) — long runs with background jobs or
|
||||
subagents therefore narrate nothing. Mid-run narration needs `turn_end`
|
||||
(fires per assistant message + tool results). Verify delivery, don't infer
|
||||
it: skill `omp-extension-event-probe`.
|
||||
5. **Latency is not silence.** Narration passes through the humanizer
|
||||
(frank-zealot) before synthesis, so an utterance lands several seconds after
|
||||
its turn; under endpoint load that stretches to tens of seconds.
|
||||
`/say humanize off` = mechanical voice, near-instant. Judge cadence from
|
||||
probe-log timestamps, never from when audio starts.
|
||||
|
||||
## Design rules the plugin now encodes (dank/dankvoice#2→#3)
|
||||
|
||||
- Narrate `turn_end` per turn **plus** `agent_end` for the settled answer;
|
||||
dedupe identical text per run (reset on `agent_start`) so the answer speaks once.
|
||||
- A turn landing while the player is alive is **dropped** — no queue, no
|
||||
interrupting; a tool loop would cut every sentence off after a word.
|
||||
Busy test: `state.child !== undefined && state.child.exitCode === null`.
|
||||
- Tool-only turns (no prose, no narration-flagged thinking) stay silent.
|
||||
- `print`/`json` modes stay silent, so benches never chatter.
|
||||
- Settings writes must **round-trip the file** (`{...onDisk, ...owned keys}`).
|
||||
Rebuilding from owned keys deleted `"auto"` on any menu choice and silently
|
||||
un-armed every later session — the classic cause of "it worked yesterday".
|
||||
|
||||
## Black-box test harness (repo `dank/dankvoice`)
|
||||
|
||||
`tests/behavior.bats` drives the built artifact through an in-process driver:
|
||||
`const say = DV_EXT(ctx, cfg)[0]`, handlers via `const events = new Map();
|
||||
const pi = { on: (n,h)=>events.set(n,h) }`, then `events.get("turn_end")?.({
|
||||
turnIndex:0, message:{role:"assistant",content:[{type:"text",text:"…"}]},
|
||||
toolResults:[]}, ctx)`. Assert on `CALL`/`PLAYS`/`NOTIFY` lines and
|
||||
`SETTINGS <file contents>`; stubs are `DV_PLAYER_CMD` (logs voice+input, honours
|
||||
`DV_HOLD`) and `DV_REWRITE_URL` (404 → falls back to mechanical text). `setup()`
|
||||
runs `bun run build` per test → serialize suites on the shared `dist/`.
|
||||
`tests/packaging.bats` requires `dist/` **committed** (compares to HEAD), so
|
||||
build → commit → the drift pin goes green.
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
name: defuse-shell-rc-secrets
|
||||
description: "Find literal API keys/tokens in shell rc files and replace them with 600-perm secret files + loader lines, syncing the canonical tool token file."
|
||||
---
|
||||
|
||||
# Defuse shell-rc secrets
|
||||
|
||||
Symptom: `export KEY=...` literal in `.zshrc`/`.bashrc` — world-readable in every process env, one `git add -A` from a dotfiles repo.
|
||||
|
||||
## Procedure (repeat per host)
|
||||
1. Sweep: `grep -n "export.*KEY\|export.*TOKEN" ~/.zshrc ~/.zprofile ~/.profile ~/.bashrc 2>/dev/null` (run per host; when batching hosts in one ssh compound command, label which output came from which host — misattribution wastes turns).
|
||||
2. Extract WITHOUT echoing: `mkdir -p ~/.config/secrets && chmod 700` it, then `sed -n 's/^export NAME="\(.*\)"$/\1/p' ~/.zshrc > ~/.config/secrets/<name>` (quote variant and unquoted variant are different sed patterns — verify byte counts with `wc -c`, a 0-byte file means the pattern missed).
|
||||
3. `chmod 600` every secret file; sync any canonical tool file the ecosystem expects (e.g. `~/.cache/huggingface/token`).
|
||||
4. Replace the rc literal (hashline `PUT N.=N:` range — never echo the secret in the op): `[ -f ~/.config/secrets/<name> ] && export NAME="$(cat ~/.config/secrets/<name>)"`.
|
||||
5. Verify: `zsh -n ~/.zshrc` (or `bash -n`); keep a dated `.bak` of the rc on remote hosts; shred any /tmp transfer copies immediately (`shred -u`).
|
||||
|
||||
## Laws
|
||||
- A token pasted into chat lands in transcripts and model context — treat chat as a leak channel; give the user the `read -rs` snippet instead.
|
||||
- scp of a secret FILE (encrypted channel, no argv) is fine; the token is never a command-line argument.
|
||||
- After replacing a legacy HF token, probe scope non-destructively: `HfApi().whoami()` (legacy read tokens show `type: access_token` without canWrite; the decisive probe is `create_repo(exist_ok=True)` on the real target repo).
|
||||
- Verify extraction byte-count before deleting/trusting; a successful-looking sed can write an empty file.
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
name: deployment-checkout-trunk-sync
|
||||
description: Use when the live deployment checkout (~/.omp main) must merge a moved origin/main while dirty tracked files block the merge and workspace policy blocks git checkout.
|
||||
---
|
||||
|
||||
---
|
||||
name: deployment-checkout-trunk-sync
|
||||
description: Use when the live deployment checkout (~/.omp main) must merge a moved origin/main while dirty tracked files block the merge and workspace policy blocks git checkout.
|
||||
---
|
||||
|
||||
# Sync a dirty live checkout to a deployment (checkout blocked)
|
||||
|
||||
Situation: the checkout IS the deployment; months of machine/hand writes sit
|
||||
uncommitted; `origin/main` has moved; `git checkout`/discard is BLOCKED by
|
||||
workspace policy ("edit forward"; exemption only during active conflict).
|
||||
|
||||
## Procedure
|
||||
1. `git fetch`; trial `git merge --no-commit --no-ff origin/main` to enumerate
|
||||
exact blockers (dirty files the merge would overwrite + untracked collisions).
|
||||
Untracked blockers: `cp -a` to `.AGENTTRASH/pre-merge-<date>/`, then `mv` aside.
|
||||
2. **Preservation commit:** `git add` ONLY the blocking paths (not the whole
|
||||
tree) and commit them as-is: `chore: adopt pre-existing working-tree state
|
||||
before trunk merge`. Nothing is discarded; the secret gate still runs.
|
||||
Include EVERY dirty deployment-state path (dotfiles/live tiers too) — a
|
||||
dirty file that is neither committed nor trashed is unprotected from step 3.
|
||||
3. `git merge --no-ff origin/main`. Conflicts are now resolvable — the policy
|
||||
exemption is active during a merge.
|
||||
4. Resolution law:
|
||||
- **live machine config** (`agent/config.yml`, `models.yml`, dotfiles live
|
||||
state like noctalia `settings.toml`) → `--ours`: the running box is its
|
||||
own authority; the remote copy is a snapshot.
|
||||
- **prose/law/docs** (`AGENTS.md`, KB) → `--theirs` (upstream canonical;
|
||||
local variants survive in the preservation commit).
|
||||
- **append-mostly logs** → union: cut only the three marker lines
|
||||
(`<<<<<<<`/`=======` range + `>>>>>>>`), keep both sides' entries.
|
||||
5. `git commit --no-edit`, push trunk, verify `main == origin/main`.
|
||||
6. Pre-merge byte-compare every dirty file vs `origin/main`
|
||||
(`git show origin/main:$f | diff -q - $f`): SUPERSEDED (identical → discardable),
|
||||
DIVERGED (resolve), untracked-in-upstream (collision).
|
||||
|
||||
## Gotchas
|
||||
- **NEVER `git stash` to unblock a deployment sync, NEVER `git stash clear`/
|
||||
`drop` afterward.** Observed 2026-10-18: a sync stashed the dirty tree, the
|
||||
stash entry was later destroyed, and months of GUI-built bar layout
|
||||
(noctalia settings.toml, never committed) vanished with it — recovered only
|
||||
via dangling-object archaeology (below). Step 2's preservation commit exists
|
||||
precisely so stashing is never needed.
|
||||
- **Stash-and-retry loops are a red flag**: three `WIP on main` stash commits
|
||||
within ~60s of each other means the operator was fighting the dirty tree,
|
||||
not resolving it.
|
||||
- **Recovery from destroyed stashes** (objects survive entry deletion until gc):
|
||||
`git fsck --lost-found` → `git log -1 --format='%ci %s' <dangling>` to date
|
||||
them → `git show <c>:<path>` to dump candidates; newest `WIP on` commit is
|
||||
usually the last good state. Validate before deploying (e.g.
|
||||
`noctalia config validate`), back up the clobbered file to `.AGENTTRASH/`,
|
||||
then restore through the symlink target.
|
||||
- Never wrap `git checkout` in `2>/dev/null` here: the policy-block error was
|
||||
swallowed and the loop silently no-op'd; the merge then failed identically.
|
||||
- `git add` a directory pathspec stages its untracked children (fine for
|
||||
allowlist dirs; never for prose dirs).
|
||||
- The live-sync heartbeat (#11) commits machine tiers hourly, so this sync is
|
||||
a rare operation post-deployment — mostly memories/dotfiles stragglers.
|
||||
Remember the heartbeat can lag up to an hour behind GUI edits: the tree
|
||||
between heartbeats is the ONLY copy of fresh GUI work.
|
||||
- **Frank's accepted risk posture (2026-10-18):** hourly sweep + the
|
||||
preservation-commit law + fsck recovery is defense-in-depth ENOUGH. Do not
|
||||
re-propose faster heartbeats or path/event triggers to shrink the
|
||||
GUI-edit-to-commit window; he chose status quo after measuring (heartbeat
|
||||
itself is 57 ms CPU — cost was never the objection; un-debounced
|
||||
DirectoryNotEmpty triggering was, and path-unit proposals are vetoed).
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
name: detached-fleet-job-launch
|
||||
description: Launch long-running jobs (fleet hosts or local workstation) so they survive ssh/logind/harness teardown; includes worker takeover law for queue daemons
|
||||
---
|
||||
|
||||
# Detached long-running jobs on Dank fleet hosts
|
||||
|
||||
## Law
|
||||
`ssh host 'setsid nohup CMD & '` is NOT detached. systemd-logind deactivates
|
||||
`session-N.scope` when the ssh command returns and SIGKILLs the whole cgroup:
|
||||
no journal entry, no OOM trace, python block-buffers stdout so the log stays
|
||||
0 bytes. setsid/nohup do not escape the cgroup.
|
||||
|
||||
## Canonical launch (remote)
|
||||
```
|
||||
ssh host 'export XDG_RUNTIME_DIR=/run/user/$(id -u); \
|
||||
systemd-run --user --collect --unit=NAME \
|
||||
-p SendSIGKILL=no \
|
||||
-p StandardOutput=append:/home/dank/logs/NAME.log \
|
||||
-p StandardError=append:/home/dank/logs/NAME.log \
|
||||
sh -c "CMD -u; echo \$? > /home/dank/logs/NAME.rc"'
|
||||
```
|
||||
- transient user unit = outside every session scope; survives ssh teardown
|
||||
(requires `Linger=yes`, true for `dank` on both hosts)
|
||||
- `echo $? > …rc` inside the unit = completion sentinel the unit itself writes
|
||||
- `-u` / unbuffered output so the log is live
|
||||
- verify launch in the same ssh call: `systemctl --user is-active NAME.service`
|
||||
|
||||
## Local workstation variant (verified 2026-10-10, blankpad)
|
||||
Harness background jobs get **tree-killed** at timeout — setsid/nohup children
|
||||
die too (survives the process-group kill, not the tree walk). Same cure as
|
||||
remote, run locally:
|
||||
```
|
||||
loginctl enable-linger $(whoami) # default Linger=no kills user units at session end
|
||||
XDG_RUNTIME_DIR=/run/user/$(id -u) systemd-run --user --collect --unit=NAME \
|
||||
-p StandardOutput=append:/abs/path/job.log \
|
||||
-p StandardError=append:/abs/path/job.log \
|
||||
/full/path/to/binary args...
|
||||
systemctl --user is-active NAME.service
|
||||
```
|
||||
Gotchas:
|
||||
- bash-tool env lacks `XDG_RUNTIME_DIR` → set it inline.
|
||||
- `--collect` GCs the unit when it **exits** — after a crash-exit,
|
||||
`systemctl --user restart NAME` fails "Unit not found"; relaunch with
|
||||
`systemd-run`. Redeploy code = `stop` (not `restart`) + relaunch.
|
||||
- point StandardOutput at the job's canonical log file (what the UI/SSE tails),
|
||||
not a sidecar — observed drift: worker.log vs per-run log.txt diverged.
|
||||
- foreground `(cmd &)` and `(cmd &)` subshells die with the tool call's group.
|
||||
|
||||
## Worker takeover law (queue daemons)
|
||||
A supervisor killed mid-run leaves `queue.json.active` set ⇒ every relaunching
|
||||
worker honors "one run at a time" and exits ⇒ silent queue deadlock. Cure:
|
||||
- heartbeat: run dir `worker.pid`, rewritten every ~5s while running, removed
|
||||
on completion;
|
||||
- claim rule: `active` set + pid dead (`process.kill(pid,0)` ESRCH; EPERM =
|
||||
alive) ⇒ resume; pid alive ⇒ exit.
|
||||
- resume must be append-only + completed-key skip so takeover is cost-free
|
||||
(observed: 386 completed samples skipped, zero re-requests).
|
||||
|
||||
## Watcher (one async job, never poll)
|
||||
```
|
||||
ssh host 'while [ ! -f /home/dank/logs/NAME.rc ]; do sleep 60; done; \
|
||||
echo RC=$(cat /home/dank/logs/NAME.rc); tail -25 /home/dank/logs/NAME.log'
|
||||
```
|
||||
Run it async with timeout 0; the result is injected on completion.
|
||||
|
||||
## Forensics when a detached job died
|
||||
1. `ps -p PID` — gone.
|
||||
2. Log 0 bytes → buffered stdout lost at SIGKILL.
|
||||
3. `journalctl --since … | grep -E "session-[0-9]+.*(Deactivated|Removed session)"` —
|
||||
timestamps matching the death window confirm the scope kill (vs OOM, which
|
||||
leaves kernel lines).
|
||||
|
||||
## Notes
|
||||
- zsh remote shells: `echo ===X===` fails (`=cmd` expansion); quote sentinels.
|
||||
- The HF upload of 2026-08 (113.7 GiB, hf_xet) died this way ~2 min in;
|
||||
relaunch as above succeeded.
|
||||
- Frank's machine: `systemctl --user terminate` is not a verb — use `stop`.
|
||||
@@ -0,0 +1,49 @@
|
||||
---
|
||||
name: docker-pin-latest-census
|
||||
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag census plus digest-equals-latest verification, no docker CLI needed."
|
||||
---
|
||||
|
||||
# Is my pinned image actually the latest?
|
||||
|
||||
Use when bumping a pinned container image tag (e.g. `lscr.io/linuxserver/*` quadlet `Image=` lines) and "latest" must be PROVEN, not assumed. Works from any box with curl — no podman/docker daemon required. `lscr.io` fronts Docker Hub; its tags are NOT on quay.io (querying quay returns nothing and reads like "no newer tags exist" — false).
|
||||
|
||||
## 1. Full tag census via the registry API
|
||||
|
||||
```bash
|
||||
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:<owner>/<repo>:pull" \
|
||||
| python3 -c "import json,sys;print(json.load(sys.stdin)['token'])")
|
||||
curl -s "https://registry-1.docker.io/v2/<owner>/<repo>/tags/list?n=2000" \
|
||||
-H "Authorization: Bearer $tok" | python3 - <<'PY'
|
||||
import json,sys,re
|
||||
tags=json.load(sys.stdin).get('tags') or []
|
||||
# linuxserver form: <version>-<sha>-ls<N>; group by version, keep newest ls per version
|
||||
v={}
|
||||
for t in tags:
|
||||
m=re.match(r'^(\d+(?:\.\d+)+)-([0-9a-f]+)-ls(\d+)$',t)
|
||||
if m: v.setdefault(tuple(map(int,m.group(1).split('.'))),[]).append((int(m.group(3)),t))
|
||||
for k in sorted(v)[-5:]:
|
||||
print('.'.join(map(str,k)),'-> newest:',max(v[k])[1]) # max on (ls#, tag) tuple
|
||||
PY
|
||||
```
|
||||
|
||||
- Sort versions as **tuples of ints**, never strings (`1.4.10 < 1.4.9` lexicographically).
|
||||
- Keep the max `ls` build per version, then the newest version — both can lag independently.
|
||||
- Trap already eaten: joining the tuple key to a string mid-loop then re-looking-it-up → `KeyError`. Keep keys as tuples; format only when printing.
|
||||
- Some channels have NO `latest` tag in `/tags/list` (or the endpoint truncates) — absence there is not evidence.
|
||||
|
||||
## 2. Digest-proof the candidate against `latest`
|
||||
|
||||
```bash
|
||||
for t in latest <candidate-tag>; do
|
||||
curl -sI "https://registry-1.docker.io/v2/<owner>/<repo>/manifests/$t" \
|
||||
-H "Authorization: Bearer $tok" \
|
||||
-H "Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.docker.distribution.manifest.v2+json" \
|
||||
| grep -i '^docker-content-digest'
|
||||
done
|
||||
```
|
||||
|
||||
Equal digests ⇒ the pinned tag IS what `latest` serves right now — the strongest "ensure latest" evidence, and recordable in the endpoint page's pin note (the dank-fleet law: *pinned; verified == the registry's `latest` manifest digest at pin time*). A multi-arch index digest matches only if both requests send the same Accept set — send all three types above.
|
||||
|
||||
## 3. Bump through git, not by hand
|
||||
|
||||
Edit the quadlet/unit `Image=` line + the vault page's pin row (one owner each), run the repo suite, PR, deploy `--service <svc>` only. Rollback stays "revert one line".
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: docs-dedupe-verify-campaign
|
||||
description: "Run an exhaustive docs dedupe + fact-verification campaign in dank/command-center: scout cluster ledgers, override scouts with evidence, gate every commit, and sweep scaffolding before PR."
|
||||
---
|
||||
|
||||
# Exhaustive docs dedupe + verification campaign (dank/command-center)
|
||||
|
||||
## Frame
|
||||
- Read `omp-docs/references/okf-spec.md` + root `omp-docs/index.md` first; inventory with `find omp-docs -name '*.md' | wc -l`.
|
||||
- Run `scripts/kb-lint.sh omp-docs` baseline; index-coverage gate is `scripts/kb-coverage.py omp-docs` (each dir's `index.md` must link every sibling file/subdir).
|
||||
- Branch off fresh `origin/main` in a NEW worktree (`git -C ~/.omp worktree add ~/.omp/wt/<name> -b docs/<issue>-slug origin/main`) — workspace policy blocks `git checkout -b` in session worktrees.
|
||||
- Open the Gitea issue first (multi-moving-parts law); branch name `docs/<issue#>-<slug>`.
|
||||
|
||||
## Scout fleet (overlap detection)
|
||||
- One `task` call, `tasks[]` batch, agent `scout`, read-only; one cluster per task (skills trees, llama trees, pi vs packages, niri, sleep/power, core, gitea, gitea+cross-cutting sweep). Cap 2 concurrent; queue rest.
|
||||
- Ledger format per cluster → `local://dedupe/<cluster>.md`: inventory table (file | lines | verdict KEEP|MERGE|DELETE|VENDOR|UPDATE), overlaps, facts-to-keep, inbound links, staleness.
|
||||
- **Scouts lie.** Verified misses in the 2026-10-19 campaign: fabricated ledger entries ("peer-reviewed BrowseComp" claim that exists in no file), a DELETE verdict for a generated artifact that is checked in by design AND under test (`tools/niri/keybindings.md`, see `scripts/gen-keybinds.sh` + `tests/gen-keybinds.bats`), and a KEEP for a leftover the scout misread its own changelog about. NEVER apply a scout verdict without re-checking the file, the tests, and the changelog yourself.
|
||||
|
||||
## Fact verification
|
||||
- Fleet facts: read-only probes over `ssh -o BatchMode=yes dankclaw|dankserver` (podman ps/inspect/top, stat, /etc/os-release, curl localhost APIs, `podman exec <c> <tool> --version`). Live state beats docs even when docs claim recent verification — the 2026-10-18 "audit" still drifted (zealot 27B→9B, build 10754→10777, container census 14→32).
|
||||
- External claims: web_search/web read yourself (no subagents per Frank's rule). Kill fabrications outright (invented distro `FreeToken`, nonexistent `ornith-llama-cpp-setup` tree, unattributable "44.5% peer-reviewed CUA") — delete/drop with an explicit note, never keep "plausible" numbers.
|
||||
- Policy drift (live ≠ tuned docs): flag in the doc + PR body as an operator decision; NEVER silently re-apply policy.
|
||||
|
||||
## Log compaction (log.md)
|
||||
- Entry law: dated entry = what changed + durable lesson + concept-page pointer; narratives recoverable via `git log --follow`. Record the law at the top of the log. Tag historical-era entries.
|
||||
|
||||
## Gates + landing
|
||||
- Gate every commit: `bash scripts/kb-lint.sh omp-docs && python3 scripts/kb-coverage.py omp-docs && bats tests/`. Pre-commit hook prints `error: command not found: caddy` on Caddyfile-touching commits — warning only, commit still lands.
|
||||
- Before opening the PR: `git diff --stat origin/main..HEAD | grep -i 'tmp\|scratch\|\.bak'` — scaffolding merged under a `.tmp-` name in the 2026-10-19 campaign and needed a follow-up PR to promote.
|
||||
- PR via API (skill `gitea-api-pr-operations` for token-extraction + merge payload); `xdg-open` the PR the moment it opens; comment the campaign ledger on the issue.
|
||||
- After Frank's merge OK: propagate deployment `git -C ~/.omp fetch && git merge --ff-only origin/main`; reap campaign worktree (`git worktree remove --force`) + delete branch local/remote; close issue with landing comment.
|
||||
|
||||
## Line-edit op syntax trap (bit three times)
|
||||
- Valid ops: `PUT N.=M:` (body `+text` lines), `CUT N.=M` / `CUT N.=M @reg`, `PUT <N @reg`, `PUT >N:` / `PUT <N:`. There is NO `PUT N+=:` form — it lands in the file as a literal garbage line and shifts the insert to the wrong position. For gap-insert use a lone `PUT >N:` or `PUT <N:` per call; pre-compute line numbers against the file map (ops reference original lines within one call, but a failed op poisons positions).
|
||||
- Recovery recipe when a literal op line lands: read exact bounds, `CUT N.=N` (junk), `CUT A.=B @block` (misplaced section), then `PUT <M @block` at the true anchor, re-read tail, add spacing blank if a `---` now abuts a list line.
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
name: docs-point-of-use-destruction
|
||||
description: "Documentation exists for point of use, not archival. Use before any dedupe/compact/verify-docs job, and whenever a census or \"only copy\" argument is being used to KEEP documentation: apply the purpose test and destroy obsolete measurements. Supersedes the preservation conclusions of vault-one-fact-one-owner-dedupe §3/§12."
|
||||
---
|
||||
|
||||
# Documentation exists for point of use
|
||||
|
||||
**Supersedes** `vault-one-fact-one-owner-dedupe` §3 ("port before cut") and §12
|
||||
("expect docs flat") wherever those conclude a page must be *kept*. Sole custody of a
|
||||
useless fact is still useless.
|
||||
|
||||
## The test, before you write and again before you keep
|
||||
|
||||
- Would the reader, doing a real task tomorrow, **act differently** because of this
|
||||
line? No → delete it.
|
||||
- Is it a measured rate (t/s, tok/s, ms, MiB-of-pool, acceptance %, a bench ladder) for
|
||||
a model / runtime / layout that is **not running now and not being compared now**?
|
||||
Destroy it. Not "file it in an Eras row", not "preserve it for history". AI moves
|
||||
faster than the vault: the number is wrong before anyone reads it, and a wrong number
|
||||
in a runbook gets believed.
|
||||
- Keep the **law** the measurement taught; drop the measurement. Qualitative causal
|
||||
stories survive; numbers do not.
|
||||
- What survives: live config and its exact numbers, procedures, failure signatures,
|
||||
security posture, interface contracts, rollback pointers.
|
||||
- **Losing information is an acceptable, usually correct outcome.** Frank said so
|
||||
explicitly on 2026-10-19 after rejecting a PR that had preserved 53 ledgers.
|
||||
|
||||
## Traps that lead agents to preserve junk
|
||||
|
||||
- **A census cannot decide what to keep.** `port-census.py`-style tools answer "where
|
||||
does this fact live", never "does anyone need it". "14–63 sole-record lines per page"
|
||||
was read as *irreplaceable* when the fact was *obsolete*. Ask usefulness, not custody.
|
||||
- **A census that includes the file under test proves the page is held by itself.**
|
||||
Exclude it, print the corpus size, assert absence; treat `0 unheld` on a long page as
|
||||
a tool bug until proven otherwise.
|
||||
- **"One fact, one owner" is a placement rule, not a preservation rule.** It says where
|
||||
a surviving fact lives; it never says the fact must survive.
|
||||
- **A change diary (`log.md`) is the largest archive in most vaults** (this one: 2,380
|
||||
lines, ~8 % of the vault). Git holds the chronology; a causal law belongs on the page
|
||||
that owns the thing it governs. Retire the diary.
|
||||
|
||||
## Execution that worked (2026-10-19, −13.7k lines)
|
||||
|
||||
1. Write the kill list as a reviewed artifact, one path per line, grouped by *why it is
|
||||
dead* — that list becomes the PR's argument and the trash manifest.
|
||||
2. Destroy whole pages first (`scripts/destroy-vault.py <list>`): move to
|
||||
`.AGENTTRASH/<date>-destruction/`, then delete every reference from survivors, whole
|
||||
line included when the dead link was its only content. Do not compare link targets
|
||||
with `pathlib.Path.stem` — it strips real suffixes (`sd.cpp`).
|
||||
3. Then strip numbers from what remains (`scripts/axe-measurements.py`): delete ledger
|
||||
sections (noun-LEDS the heading, nothing instructional after it), blank any table cell
|
||||
reporting a rate, fold Eras older than the current one into a single row pointing at
|
||||
git. Never touch a shape number or a Laws/Wire/Rollback section.
|
||||
4. Sweep indices (generator must exclude the trash dir, drop rows whose page cell became
|
||||
empty), re-lint, re-check links.
|
||||
5. Rewrite the governing policy in the same PR, or the next agent re-grows the vault:
|
||||
project `AGENTS.md` *Documentation Policy* + the global agent settings.
|
||||
|
||||
## Report honestly, per area
|
||||
|
||||
docs / scripts / operator-doc separately, measured against the merge-base. The tools are
|
||||
a cost that must be named, not hidden inside a headline "net" number — that is what got
|
||||
the previous pass rejected.
|
||||
|
||||
## Verify the repair tools themselves
|
||||
|
||||
One repair pass matched "line ends with a comma" and deleted the trailing comma from 660
|
||||
wrapped prose lines. Restore by matching lines that are byte-identical to `HEAD` except
|
||||
for trailing punctuation (`restore-punctuation.py`), keep the destructive rule out of the
|
||||
tool, and record the incident in its docstring.
|
||||
@@ -0,0 +1,85 @@
|
||||
---
|
||||
name: docs-refresh-scout-summaries
|
||||
description: "Refresh a stale documentation cluster without delegating the writing: scout subagents produce per-line verdict tables (holder verified by comma-insensitive grep), the parent makes every edit, and a port-before-cut census proves nothing was lost. Use for dedupe-our-docs, fix-the-stale-runbooks, pointers-instead-of-copied-numbers jobs."
|
||||
---
|
||||
|
||||
# Refresh stale docs with scout summaries, parent-side writes
|
||||
|
||||
Use when a cluster of documentation pages is stale and/or restates facts another page owns, and
|
||||
the fix touches many files. **Scouts read and classify; the parent writes.** Sibling skill
|
||||
`vault-one-fact-one-owner-dedupe` covers collapsing era pages into owner pages - its section 3
|
||||
(orphan census), section 7 (edit-tool hazard) and section 9 (verify/PR) apply unchanged.
|
||||
|
||||
## Why not writer subagents (measured cost)
|
||||
|
||||
Writer agents produce plausible output and cannot be trusted on the one claim that matters - the
|
||||
owner page already holds this fact. In one run they asserted every cut was grep-confirmed while
|
||||
FOUR cuts left pointers to pages that did not contain the fact (118->238 GiB, 16,095 tokens,
|
||||
0.739/0.554/0.427/0.340, PR #23897). Each became a broken promise the next reader trusts.
|
||||
Classification is the only part that parallelizes; writing is where taste (ownership, era
|
||||
naming, what counts as a law) lives. So fan out read-only scouts and keep the pen.
|
||||
|
||||
If writer agents were already spawned and are no longer wanted: `write agent://all` to stop
|
||||
them, then `git status` the target tree - `proc://` does not list agent jobs, and a finished
|
||||
worker may already have edited files.
|
||||
|
||||
## Pipeline
|
||||
|
||||
1. **Cluster** targets into groups of 3-4 pages sharing subject matter (one cluster per scout).
|
||||
Know the owner map first (endpoint/service/host pages, plus the repo documentation law).
|
||||
2. **Wave of 2 scouts** (harness cap is `task.maxConcurrency`, usually 2). Scout contract:
|
||||
|
||||
> READ-ONLY, no writes of any kind. Per file, one table row per candidate fact:
|
||||
> `| lines | verbatim figure + 6 words | holder elsewhere (file:line) or none | VERDICT |`
|
||||
> Verdicts: `CUT -> [[Owner]]` (holder verified), `KEEP(owned here)` (no holder, this page is
|
||||
> the record), `PORT(parent)` (rightful owner lacks it - name page and section), `STALE`
|
||||
> (contradicted by current state; quote the contradicting line), `INDEX` (locator page).
|
||||
> Verify every holder claim with a COMMA-INSENSITIVE grep: a holder writing 1,350,182
|
||||
> satisfies a page writing 1350182 and vice versa; 16,095 does NOT satisfy a check for 16095
|
||||
> unless commas are stripped. Never name a holder line you have not read.
|
||||
> End with a per-file tally: kept / cut / port / stale.
|
||||
|
||||
Put the law, the owner map and the verification rule in the batch-level `context`, never per
|
||||
task.
|
||||
3. **Write while the next wave runs**: edit cluster N from wave N, launch wave N+1 in the same
|
||||
turn. Never launch a wave you have nothing to write.
|
||||
4. **Verify each CUT yourself** before cutting - read the holder line. A scout row is a
|
||||
hypothesis, not evidence.
|
||||
5. **PORT items are your edits**: add the fact to the owner page section (Eras row / Laws / spec
|
||||
table), then cut it from the technique page in the same commit.
|
||||
6. **STALE items: check live state**, not another document - `ssh <host> systemctl is-active
|
||||
<svc>`, the endpoint `/health`, the env file. Then rewrite with a date, or drop transient
|
||||
live state an owner page already records.
|
||||
|
||||
## The ownership test that stops wasted ports
|
||||
|
||||
Two pages carrying one figure is the defect - a figure with exactly one holder is fine wherever
|
||||
it lives. A technique page that is the sole record of a campaign owns that campaign's arm
|
||||
tables; do not port such ledgers into an endpoint Eras table. Write the clause into the repo
|
||||
documentation law so the next pass stops porting single-owner facts. Corollary: a worked example
|
||||
may repeat a live number if it declares itself dated and names the owner (a snapshot, not fleet
|
||||
state - recompute from the live files).
|
||||
|
||||
## Census after each cluster (catches your own slips too)
|
||||
|
||||
Run the sibling skill's section 3 census over `git diff -U0` for the cluster (deleted lines x
|
||||
distinctive numeric keys vs the whole vault **plus AGENTS.md**), comma-insensitively; drive the
|
||||
count to zero or list each accepted loss in the commit message.
|
||||
|
||||
```python
|
||||
flat = corpus.replace(',', '')
|
||||
pres = lambda n: n in corpus or n.replace(',', '') in flat
|
||||
toks = lambda l: [n for n in re.findall(r'\d[\d,]*(?:\.\d+)?', l)
|
||||
if len(n.replace(',', '').split('.')[0]) >= 4
|
||||
or (len(n.split('.')[-1]) >= 2 and len(n.replace(',', '')) >= 5)]
|
||||
orphan = [l for l in deleted if toks(l) and not any(pres(n) for n in toks(l))]
|
||||
```
|
||||
|
||||
## Verify and land
|
||||
|
||||
- Vault link seam must print `0 unresolved links`; hard-wrapped `[[Two Word Page]]` links split
|
||||
across lines are the usual cause - unwrap them.
|
||||
- Full suite once at the end, not per cluster; a docs seam running while scouts are live may
|
||||
read half-written files, so re-run after the last edit.
|
||||
- Commit per cluster (docs-only means **no deploy**); push, open the PR through the API with a
|
||||
600-perm curl config, `xdg-open` the PR URL immediately.
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
name: dotfiles-adoption-audit
|
||||
description: "Use when deciding which userspace configs on a Dank box belong in the command-center dotfiles tier — separates tracked intent from generated expansion, app state, secrets, and config owned by another repo. Not for deploying one known file."
|
||||
---
|
||||
|
||||
# Dotfiles adoption audit
|
||||
|
||||
Decides, with evidence, which files under `~/.config` (and top-level dotfiles) belong in
|
||||
`dank/command-center`'s `dotfiles/` tier. The repo law being applied: **config in, state
|
||||
out**, and **one fact, one owner**.
|
||||
|
||||
## The four gates (all four must pass)
|
||||
|
||||
1. **Installed?** `command -v <prog>` — config for software absent from the box is a prune
|
||||
candidate, never an adoption target. This eliminates most of the clutter.
|
||||
2. **Authored, or the expansion of something else?** Generated files must stay out even
|
||||
though they look like config.
|
||||
3. **Does another repo or installer already write it?** If a script or package owns the
|
||||
file, tracking it here creates a second owner that loses every argument.
|
||||
4. **Could a secret ever live in it?** If yes: template + quarantine, or leave it out.
|
||||
|
||||
## Gate 2 is the one people get wrong — prove generation from mtime
|
||||
|
||||
A generator walking the app list writes dozens of files in the same sub-second window.
|
||||
Compare timestamps across unrelated programs:
|
||||
|
||||
```bash
|
||||
cd ~/.config && stat -c '%y %n' */themes/noctalia* */noctalia.* */colors/noctalia* 2>/dev/null | sort
|
||||
```
|
||||
|
||||
Identical seconds + microsecond progression across ten different programs = generated.
|
||||
Re-check one file days later: a new mtime with no human edit proves regeneration, so the
|
||||
tracked file must be the *source* (palette/config the generator reads), never its output.
|
||||
The correct pattern is a tracked config that *imports* the generated path.
|
||||
|
||||
## Classify the whole tree per file, not per directory
|
||||
|
||||
The tracked unit is the **file symlink**, so a directory can be "real" while containing only
|
||||
tracked symlinks. Never classify by directory:
|
||||
|
||||
```bash
|
||||
cd ~/.config && for d in */; do d=${d%/}; tr=0; rl=0; rlfiles=""
|
||||
while IFS= read -r f; do
|
||||
case "$(readlink -f "$f" 2>/dev/null)" in
|
||||
*/.omp/dotfiles/*) tr=$((tr+1));;
|
||||
*) rl=$((rl+1)); rlfiles="$rlfiles ${f#./}";;
|
||||
esac
|
||||
done < <(find "$d" -maxdepth 2 \( -type f -o -type l \) 2>/dev/null \
|
||||
| grep -vE 'Cache|cache|Cookies|storage|sessions|Crashpad|Default|thumbnails|extensions')
|
||||
[ $((tr+rl)) -eq 0 ] && continue
|
||||
printf '%-22s tracked=%-3s box-only=%-3s %s\n' "$d" "$tr" "$rl" "$(echo $rlfiles | cut -c1-90)"
|
||||
done
|
||||
```
|
||||
|
||||
Then read the headers of suspicious files — installers often confess ("Installed by
|
||||
`infra/setup-dev.sh` into `~/.config/containers/systemd/`") and unit files name another
|
||||
project's `WorkingDirectory`/`ExecStart`, which means that project owns them.
|
||||
|
||||
## Ownership before adoption
|
||||
|
||||
```bash
|
||||
git -C ~/.omp ls-files dotfiles # packages deploy.sh sees
|
||||
grep -rl "<unit-or-file-name>" ~/Projects/<proj> # does a project repo own it?
|
||||
systemctl --user is-active <unit> # dead config is not load-bearing
|
||||
```
|
||||
|
||||
## Secret-gap check (do this before adopting anything env-shaped)
|
||||
|
||||
The ignore list may cover only `.env`, so `frank-job-hunt.env` / `postgres.env` fall
|
||||
through, and a token-shape secret gate (`sk-`, `ghp_`, PEM, JWT) does not match a
|
||||
low-entropy `POSTGRES_PASSWORD=…`. Test both explicitly:
|
||||
|
||||
```bash
|
||||
cd ~/.omp && git check-ignore -v --no-index 'dotfiles/x/postgres.env' || echo "NOT IGNORED"
|
||||
sed -n 's/^patterns=//p' .githooks/pre-commit
|
||||
```
|
||||
|
||||
Fix = add `*.env` to the quarantine block **and** a password-assignment pattern to the gate,
|
||||
with a red test that stages a fake `POSTGRES_PASSWORD=` file.
|
||||
|
||||
## Land it
|
||||
|
||||
Packages are home-relative trees under `dotfiles/<pkg>/`; per-host overrides go in
|
||||
`zz-$(hostname)` (stowed last). `dconf` is loaded via `dconf load /`, `systemd-user` gets
|
||||
`daemon-reload` + conditional `enable --now`. Sequence: `scripts/deploy.sh --check` (dry run,
|
||||
reports divergents) → adopt → `deploy.sh` → confirm live paths are symlinks into the repo →
|
||||
`bats tests/` (includes the real-bundle `kb-lint`/coverage gate) → branch off `origin/main`,
|
||||
PR to `main`, never merge without Frank's OK.
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
name: dotfiles-deployment-truth-audit
|
||||
description: "Answer \"is my X config actually tracked and deployed?\" per package: classify every file by readlink, expose tracked-then-ignored debt, and re-derive census numbers from the command printed on the page."
|
||||
---
|
||||
|
||||
Answer to "does this repo still track my <pkg> settings?" must be a measurement, never a remembered count. The tracked unit is the **file**, not the directory: a live directory can be a real directory and still be fully tracked, because every file inside it is a symlink into the repo.
|
||||
|
||||
Run in the deployment checkout (the checkout IS the deployment).
|
||||
|
||||
## 1. Tracked set, per package, both tiers
|
||||
|
||||
```bash
|
||||
cd ~/.omp # or the repo that owns dotfiles/
|
||||
git ls-files dotfiles/<pkg> # base tier
|
||||
git ls-files dotfiles/zz-$(hostname) | grep <pkg> # host override tier — counted separately
|
||||
```
|
||||
|
||||
The `zz-*` tier overrides base files by the same relative path, so a per-package count
|
||||
that ignores it under-reports what a machine actually runs.
|
||||
|
||||
## 2. Live wiring — classify every tracked file (never by directory)
|
||||
|
||||
```bash
|
||||
for f in $(git ls-files dotfiles/<pkg>); do
|
||||
rel=${f#dotfiles/<pkg>/}; live="$HOME/$rel"
|
||||
if [ -L "$live" ]; then printf 'link %-50s -> %s\n' "$rel" "$(readlink "$live")"
|
||||
elif [ -e "$live" ]; then printf 'REAL %-50s (NOT a link)\n' "$rel"
|
||||
else printf 'absent %s\n' "$rel"; fi
|
||||
done
|
||||
```
|
||||
|
||||
- `REAL` = the repo copy is a fiction: live state diverged, deployment never reached it.
|
||||
- `absent` = tracked relic, nothing consumes it (candidate for `git rm --cached`).
|
||||
- Spaces in filenames break the `for` loop (`palettes/Gruber Red.json` splits into two
|
||||
rows). Use `git ls-files -z | tr '\0' '\n'` + `while read -r` when names can hold spaces.
|
||||
- Reverse direction finds untracked-but-load-bearing files:
|
||||
`find ~/.config/<pkg> -maxdepth 2 \( -type f -o -type l \) | while read -r p; do case "$(realpath "$p")" in $PWD/dotfiles/*) ;; *) echo "not from repo: $p";; esac; done`
|
||||
|
||||
## 3. Live state that must stay untracked
|
||||
|
||||
Generated palette output and app state are expansion, not authorship. Distinguish by
|
||||
mtime cluster, not by name: `find ~/.local/state/<app> -type f -printf '%TY-%Tm-%Td %p\n'` —
|
||||
a same-second mass write is a generator; single later re-stamps are re-runs. Pointers a
|
||||
switcher script writes (e.g. `cfg/display.kdl -> desk-1-home.kdl`) are machine state too.
|
||||
|
||||
Then check the hazard this creates: grep the entrypoint for includes of those generated
|
||||
files. A **non-optional** include of a correctly-untracked generated file bricks a fresh
|
||||
box (`include "noctalia.kdl"` with no `optional=true` = niri cannot start until the theme
|
||||
pipeline has run once). Report it as a fresh-box failure, not a style note.
|
||||
|
||||
## 4. Tracked-then-ignored debt (invisible to every normal check)
|
||||
|
||||
```bash
|
||||
git ls-files dotfiles | grep AGENTTRASH # trash committed inside the tree
|
||||
git check-ignore --no-index -v <that path> # does an ignore rule match it?
|
||||
```
|
||||
|
||||
`.gitignore` patterns without an interior slash (`.AGENTTRASH/`) match at **any depth**, so
|
||||
the rule was fine all along — the file simply entered history *before* the rule existed, and
|
||||
an ignore rule never reaches an already-tracked path. Only repair: `git rm --cached <path>`
|
||||
plus moving the working copy to the root `.AGENTTRASH/` (never `rm`). `git check-ignore`
|
||||
without `--no-index` prints nothing for tracked paths and reads as "not ignored" — that
|
||||
false negative is the trap.
|
||||
|
||||
## 5. A census number is only valid with its command
|
||||
|
||||
If a page prints a classifier command, every number on it must be reproducible by running
|
||||
that command. A row that the command does not reproduce was written from memory — wrong, not
|
||||
merely stale. Fix the row, and state the correction in the commit message rather than
|
||||
silently editing the number.
|
||||
|
||||
## Verification before landing
|
||||
|
||||
`bats tests/`, `scripts/kb-lint.sh omp-docs`, `scripts/kb-coverage.py omp-docs`, and
|
||||
`scripts/deploy.sh --check` **from the worktree** — `--check` is read-only; never run the
|
||||
real deploy from a worktree, it repoints live symlinks into the worktree the heartbeat reaps.
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
name: dotfiles-tier-live-audit
|
||||
description: "Audit the command-center dotfiles tier before running deploy.sh: prove per-file tracked/deployed state, refuse generated theme output, catch stale zz overrides, wrong-box dconf keys, and unresolvable systemd ExecStart specifiers."
|
||||
---
|
||||
|
||||
# Deploy-tier audit (`~/.omp` command center)
|
||||
|
||||
Run **before** any real `scripts/deploy.sh` on a box with a live session, and whenever a
|
||||
symlink, palette, wallpaper or unit behaves oddly. Facts and laws live in
|
||||
`omp-docs/tools/dotfiles-deploy.md` + `omp-docs/inventory/home-config-census.md` — this page
|
||||
is only the probe order. Do not restate the laws in new files; the vault owns them.
|
||||
|
||||
## 0. Classify per file, never per directory
|
||||
|
||||
```bash
|
||||
cd ~/.omp
|
||||
for f in $(git ls-files dotfiles/<pkg>); do t=$(readlink -f "$HOME/${f#dotfiles/<pkg>/}")
|
||||
printf '%-50s %s\n' "${f#dotfiles/<pkg>/}" "$([ -L "$HOME/${f#dotfiles/<pkg>/}" ] && echo LINK || echo real)"
|
||||
done
|
||||
```
|
||||
A live directory can be real and still fully tracked (every file a symlink). `niri` 8/2,
|
||||
`gtk-3.0` 3/0. `zz-$HOST` files count in the `zz` tier's row, not the base package's.
|
||||
|
||||
## 1. The plan, and how to read it
|
||||
|
||||
```bash
|
||||
./scripts/deploy.sh --check # writes nothing; read the last line, not the RELINK lines
|
||||
```
|
||||
`--check` prints `RELINK` for **every** path the `zz` tier carries even when the live link
|
||||
already points at the override (base is walked first, intermediate owner change reported).
|
||||
Only `actions=0` is a real summary; confirm a pending change with `readlink`.
|
||||
|
||||
## 2. Generated theme output must never be tracked or deployed
|
||||
|
||||
```bash
|
||||
git ls-files dotfiles | grep -Ei 'noctalia\.(kdl|toml|json|conf|css)|themes/noctalia'
|
||||
```
|
||||
Hit = generated palette output (same-millisecond mtime cluster proves it). Untrack
|
||||
(`git rm --cached` + `.gitignore` patterns); `deploy.sh` refuses the class (`SKIP … generated
|
||||
theme output`). Tracking one means the next sync quarantines the user's live palette and
|
||||
restores the committed colours.
|
||||
|
||||
## 3. Stale `zz` overrides
|
||||
|
||||
For each path present in both base and `zz-$HOST`: `diff` them, then compare each file's
|
||||
`git log -1 --date=short`. A `zz` copy older than the base change is a time bomb — sync
|
||||
relinks cross-package owners **silently** (`nothing to quarantine`). Delete the `zz` copy in
|
||||
the same commit that base absorbed.
|
||||
|
||||
## 4. dconf: compare key by key, never with `diff`
|
||||
|
||||
`dconf load /` writes only keys present, so a text diff misleads both ways. Parse both sides
|
||||
into `[section]/key → value` and report: value conflicts, keys the dump would newly write,
|
||||
and any value containing `/home/` (foreign-box state). File-chooser `last-folder-path`,
|
||||
`window-state`, `picture-uri*` are state, not preferences — delete them from the dump and
|
||||
clear the live keys if they were already deployed here.
|
||||
|
||||
## 5. Units: state, then resolvability
|
||||
|
||||
```bash
|
||||
for u in <units in dotfiles/systemd-user>; do printf '%s %s %s\n' "$u" \
|
||||
"$(systemctl --user is-enabled $u)" "$(systemctl --user is-active $u)"; done
|
||||
```
|
||||
`is-enabled` exits **1** for `linked` exactly as for `disabled` — read the printed word.
|
||||
Never start a unit that is already installed: `swayidle-lock` sits `enabled`/`inactive` and
|
||||
`--now` arms a screen lock mid-session. Specifiers are systemd's, not the shell's: probe
|
||||
`command -v "${bin//%h/$HOME}"` before believing `ExecStart … absent on this host` — the
|
||||
literal `%h/…` string made this repo's own heartbeat look foreign on every box.
|
||||
|
||||
## 6. Landing it
|
||||
|
||||
`bats tests/` (deploy contract runs `scripts/deploy.sh` in a scratch repo with `DEPLOY_REPO`,
|
||||
`DEPLOY_HOST`, stub `HOME`, stub `systemctl` on `PATH`) + `scripts/kb-lint.sh omp-docs` +
|
||||
`scripts/kb-coverage.py omp-docs`. Branch off `origin/main`, PR, never merge without Frank's
|
||||
OK (`gitea-merge-form-encoded`: form-encoded `do=merge`, `merge_style=no-ff`).
|
||||
|
||||
Then deploy from the **deployment root** `~/.omp` only — never from a worktree (live symlinks
|
||||
would point into a checkout the heartbeat reaps in 7 days). Acceptance: `actions=N
|
||||
quarantined=0`, two consecutive `--check` runs both `actions=0`, no `ENABLE … started` for a
|
||||
unit that was already installed.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: e2e-port-squat-triage
|
||||
description: "When an e2e seam on a fixed port fails weirdly (404s on routes that exist, stale behavior, health probe passes but app seems ancient), audit for leftover dev/debug servers squatting the port before believing any red or green."
|
||||
---
|
||||
|
||||
# E2E fixed-port squat triage
|
||||
|
||||
## Symptom classes (all seen with a stale server squatting the seam's fixed port)
|
||||
- Routes that exist in current `dist`/source return 404 ("Cannot POST /api/...") while unit tests through the same app factory pass.
|
||||
- A seam that is *green* but exercises old code: its setup curls `http://host:PORT/health` and gets a response from **whoever owns the port** — a leftover server, not the one it just spawned (the spawned one dies or idles with EADDRINUSE, sometimes still showing in `pgrep`).
|
||||
- Behavior matches an older build (missing mounts/guards) despite a fresh build step.
|
||||
|
||||
## Triage recipe (before touching code)
|
||||
1. Who owns the port:
|
||||
```bash
|
||||
ss -tlnp 2>/dev/null | grep :PORT
|
||||
```
|
||||
2. List every server process with **start times**:
|
||||
```bash
|
||||
for p in $(pgrep -f "path/to/server-entry"); do ps -o pid,lstart,cmd -p $p --no-headers; done
|
||||
```
|
||||
3. Compare start times against the build artifact timestamps (`ls -la dist/...`). A server older than the dist you just built is a liar — its in-memory modules predate the code under test.
|
||||
4. Kill all leftovers, then re-check: `pgrep -f "server-entry" || echo all-dead`.
|
||||
5. Rerun the seam. Only now is red/red and green/green meaningful.
|
||||
|
||||
## Why it happens
|
||||
- `kill %1` in **non-interactive** bash fails silently (no job control) — use explicit PIDs or `pkill -f <pattern>`.
|
||||
- Seam setups that "wait for health" cannot distinguish their own server from a squatter on the same port; a pre-flight "port must be free" assertion is the permanent fix (assert *before* spawn).
|
||||
|
||||
## Hardening (if you own the seam)
|
||||
Add to setup, before spawning: fail fast if the port is already listening — turns the silent liar into a loud error:
|
||||
```bash
|
||||
ss -tln "sport = :PORT" | grep -q LISTEN && { echo "port PORT busy — kill leftovers first" >&2; return 1; }
|
||||
```
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: faillock-sudo-lockout-recovery
|
||||
description: "Diagnose and recover from a pam_faillock sudo lockout on Frank's Arch workstations, typically caused by an agent's non-interactive sudo retries"
|
||||
---
|
||||
|
||||
# Faillock Sudo Lockout Recovery
|
||||
|
||||
## Symptom
|
||||
User reports "sudo password rejected / changed"; journal shows
|
||||
`pam_faillock(sudo:auth): User <u> is temporarily locked out due to 3 consecutive failed login attempts`.
|
||||
|
||||
## Diagnose (read-only, one round)
|
||||
1. `stat -c '%n mtime=%y ctime=%z' /etc/shadow` — unchanged mtime = password was NEVER changed.
|
||||
2. `journalctl --since '<window>' | grep 'sudo\['` — lines with `PWD=` / `COMMAND=` name the spammer (agent harness loops show `PWD=/home/.../.omp`, `COMMAND=...yay/pacman...`, no `TTY=`).
|
||||
3. `faillock --user <user>` — shows the valid failure entries.
|
||||
|
||||
## Root cause law
|
||||
In the omp harness sudo has NO TTY: each `sudo <cmd>` = 3 pam_unix failures. Arch pam_faillock defaults deny=3 with ~600s unlock_time, and each retry re-arms it — the user's real interactive sudo is locked while the loop runs.
|
||||
|
||||
## Recover
|
||||
1. Kill the loop: `ps -eo pid,ppid,etime,cmd | grep -E 'sudo|yay|makepkg'` → kill offending PIDs; stop/kill the responsible agent session.
|
||||
2. `faillock --user "$USER" --reset` — works unprivileged for your own user (dir /run/faillock/<user> is 0700 owner).
|
||||
3. User verifies with `sudo -v` in a real terminal.
|
||||
|
||||
## Prevent
|
||||
- Agents NEVER run sudo/yay/pacman/pkexec; no SUDO_ASKPASS/expect workarounds.
|
||||
- Root-requiring steps are handed to the user as exact copy-paste commands in the final report.
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
name: fandom-wiki-media-fetch
|
||||
description: Fetch audio/image media from a Fandom wiki via its MediaWiki API when the wiki HTML pages 403 curl or hang headless browsers.
|
||||
---
|
||||
|
||||
Goal: get direct media URLs (and files) from a starcraft.fandom.com-style Fandom wiki.
|
||||
|
||||
Why this route: Fandom HTML pages return a ~5 KB JS shell to `curl` (403 via `read ...:raw`) and time out headless Chromium (`browser.open`/`goto` >30 s on quotation megapages). The MediaWiki API is plain JSON and answers plain curl.
|
||||
|
||||
Steps:
|
||||
1. Find the content page's wikitext:
|
||||
`curl -sL -A "Mozilla/5.0" "https://<wiki>.fandom.com/api.php?action=parse&page=<Page_Name>&prop=wikitext&format=json&formatversion=2"`
|
||||
Top-level pages are often tabber/transclusion shells — follow the `{{:Subpage}}` / `{{TemplateName}}` references with the same call (e.g. `StarCraft_unit_quotations` → `/Protoss` → `Template:QuotesSC1Zealot`).
|
||||
2. Extract media names from the wikitext. StarCraft quotes use `{{sm2|File.ogg|displayed quote}}` inside `{{UnitQuoteBox|select=...}}` fields — match the quote text to pick the exact file.
|
||||
3. Resolve direct CDN URLs (works for images too):
|
||||
`curl -sL -A "Mozilla/5.0" ".../api.php?action=query&prop=imageinfo&iiprop=url|size&format=json&formatversion=2&titles=File:A.ogg|File:B.jpg"`
|
||||
→ `query.pages[].imageinfo[0].url` = `https://static.wikia.nocookie.net/...` — downloadable with plain `curl -sfL`.
|
||||
4. To discover art on a unit page: `action=query&prop=images&imlimit=100&titles=<Page>`, filter titles (icon/art/portrait/Head/Render), then resolve the keepers via imageinfo. Note splash/head art naming: `File:Zealot SC2 Art1.jpg`, `File:Archon SC2 Head1.jpg`; some obvious guesses (SC1-era units' SC2 splash) do not exist — verify with imageinfo, absent pages return `missing` with no `imageinfo`.
|
||||
5. Verify downloads (`file sfx/*.ogg` → `Ogg data, Vorbis audio`) — a 403 body saves as a success with `-o` and no `-f`.
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: fixture-delay-window-race-test
|
||||
description: Pin a race/abort bug in a fixture-server test by delaying only the Nth request so the interruption lands inside the in-flight window; assert via a driver-installed unhandledRejection reporter.
|
||||
---
|
||||
|
||||
# Reproducing a timing-window bug against a fixture server
|
||||
|
||||
Use when a defect only appears in a window — a promise created before it is awaited, a kill/abort landing mid-flight, a prefetch interrupted — and a fixture HTTP server answers instantly, so the window never opens. Earned in `dankvoice` issue #7 (`[Unhandled Rejection] AbortError` from `/say stop` during a pipelined synthesis).
|
||||
|
||||
## Why naive attempts pass vacuously
|
||||
|
||||
1. **Uniform delay on the fixture route** (`DV_LATENCY_MS` for every request) makes *all* requests resolve at the same time, so by the time the interruption fires nothing is in flight. The test goes green against the broken code — worse than no test.
|
||||
2. Asserting on a log line count that also counts interruption noise: `dankvoice`'s driver prints `PLAYS <n>` as the number of **player log lines**, and the stub appends `KILLED` on SIGTERM — so a single playback with one interrupt reports `PLAYS 2`. Count the payload lines instead (`grep -c 'PLAYER .*\.wav$'`).
|
||||
|
||||
## The shape that works
|
||||
|
||||
- **Delay only requests after the first.** The first response gets the pipeline into playback; the second stays in flight, which is the window:
|
||||
```ts
|
||||
const spoken = received.filter((c) => c.url?.endsWith("/audio/speech")).length; // already pushed
|
||||
const delay = spoken > 1 ? Number(process.env.DV_SYNTH_MS ?? 0) : 0;
|
||||
if (delay > 0) setTimeout(answer, delay); else answer();
|
||||
```
|
||||
- **Make the interruption land inside it.** Player holds long (`DV_HOLD=2`), interruption at a fixed offset (`DV_STOP_AT=400`) with the delayed response far later (`DV_SYNTH_MS=1200`): margins wide enough that the abort is neither before the request nor after the response.
|
||||
- **Turn unhandled rejections into observable output** in the test driver instead of letting the runtime crash on them — that is also exactly what the real harness does to the user:
|
||||
```ts
|
||||
process.on("unhandledRejection", (reason) => {
|
||||
console.log(`UNHANDLED ${reason instanceof Error ? `${reason.name}: ${reason.message}` : String(reason)}`);
|
||||
});
|
||||
```
|
||||
Then assert `[[ "$output" != *"UNHANDLED"* ]]` plus the concrete behavior (segments actually played, player killed).
|
||||
- **Leak checks measure a delta, not an absolute:** count `dankvoice-*` temp dirs before and after the run and compare; an absolute `== 0` fails on another test's leftovers.
|
||||
|
||||
## Order
|
||||
|
||||
Write the scenario first and watch it fail **on the rejection assertion specifically** — if it fails on some other assertion (a count, a kill line), the window is wrong, not the bug. Then fix, then confirm red→green on that one assertion before running the full suite.
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: fleet-host-process-service-wiring
|
||||
description: "Add a host-process (non-quadlet) systemd USER service/exporter to serverAppConfigs — every file that must change together, with the signal-relay precedent and the Prometheus gateway/LAN reach rules."
|
||||
---
|
||||
|
||||
A host-process service (runs `python3`/a binary on the host, NOT a container) is the right shape when the thing must read host-only state (DRM/cgroup sysfs, `shells to nvidia-smi`, signal-cli) that a container cannot reach without passthrough. Precedent to mirror: `scripts/signal-relay.{py,service}` + `install_signal_relay` in `deploy-remote.sh`. A new one landed 2026-10-20: `scripts/gpu-exporter.{py,service}` + `deploy_gpu_exporter`.
|
||||
|
||||
## The unit is a USER unit, not a quadlet
|
||||
- Hand-written `scripts/<svc>.service`: `[Service]` `ExecStart=/usr/bin/python3 /home/dank/serverAppConfigs/scripts/<svc>.py` (runs straight from the git checkout — a pull updates it, no copy step), `Restart=always`, `[Install] WantedBy=default.target`.
|
||||
- It CAN be `systemctl --user enable`-ed (generated quadlet units cannot on this host's podman generator). With `Linger=yes` for `dank`, `[Install]`+`enable` is the whole boot story.
|
||||
|
||||
## Every file that MUST change together (a lone change on one side fails silently)
|
||||
1. `scripts/<svc>.service` — the unit. Base binds **loopback only**; the LAN address is added by a deploy **drop-in**, so the checked-in unit is never a wildcard.
|
||||
2. `scripts/deploy-remote.sh`:
|
||||
- a `deploy_<svc>()` function: `cp` the unit to `~/.config/systemd/user/`, write a `<svc>.service.d/bind.conf` drop-in with the per-host `Environment=..._BINDS=127.0.0.1,<lan>` (map LAN from `$TARGET`: dankclaw=192.168.0.143, dankserver=192.168.0.235 — do NOT hardcode one host if it runs on both), `daemon-reload`, `systemctl --user enable`+`restart`, then a **loop health gate** (`for i in $(seq 1 15); curl -fsS -m5 http://127.0.0.1:<port>/metrics && return; sleep 1`) — a one-shot curl races the bind and once failed a healthy deploy.
|
||||
- a dispatcher branch `<svc>) deploy_<svc> ;;` in the central `case "$service"` (near the observability branch).
|
||||
- an **allowlist entry on EVERY target** it runs on: `dankclaw:<svc>` and/or `dankserver:<svc>` (the allowlist `case` around the top of the file).
|
||||
3. `scripts/deploy-plan.py`: add `"<svc>"` to `SERVICE_ORDER`, to `DANKCLAW_SERVICES`/`DANKSERVER_SERVICES` (whichever it runs on), and a `scripts/<svc>.*` path-map in the `parts[0] == "scripts"` block: `if name in {"<svc>.py","<svc>.service"}: _add(services, "<svc>")`. `target_services` then filters per target.
|
||||
4. `scripts/test-changed.sh`: a `case` routing `scripts/<svc>.py|scripts/<svc>.service` → its seam + `deployment_scoping.bats` (place before the generic fallbacks). `tests/run.sh` runs all `*.bats` regardless, so the mandatory gate is covered even if this route is missing — but add it for focused runs.
|
||||
5. `shared/prometheus/prometheus.yml`: one scrape job per host, using the **reach rules** (below).
|
||||
6. Dashboard JSON + the owning page under `serverAppConfigs-docs/Infrastructure/Observability.md` (one owner per fact).
|
||||
|
||||
## Prometheus reach rules (the whole cross-file contract)
|
||||
Prometheus is a **bridge container**. On this netavark host a bridge container CANNOT dial the host's own LAN IP at a host-bind (only at a *published* port). Therefore:
|
||||
- **Same host (Dankclaw):** scrape `host.containers.internal:<port>` (netavark delivers it to the host LAN bind). Scraping `192.168.0.143:<port>` reads DOWN forever even though the endpoint is healthy.
|
||||
- **Different host (Dankserver):** plain LAN `192.168.0.235:<port>` reaches it.
|
||||
- Never target a `*.franksdank.cloud` vhost — Prometheus cannot solve an Anubis challenge; it is red by construction.
|
||||
- The exporter must bind BOTH loopback and the host LAN (like `node-exporter`/`cadvisor`), so `host.containers.internal` and the LAN both land.
|
||||
|
||||
## Firewall
|
||||
Leave it LAN-open like node-exporter/cadvisor (unauthenticated metrics on the LAN is the repo's exporter posture). Only signal-relay gets an iptables allowlist (loopback+host+`10.89.0.0/16`+DROP) because it is an unauthenticated message-SENDING webhook — do not copy that lock onto a read-only exporter.
|
||||
|
||||
## Seam test (tests/<svc>.bats) pattern
|
||||
- Black-box the binary with injectable inputs (fixture dirs / stub commands), assert only consumer-visible output; assert **absent means absent** (anchor guards to a sample line `^metric{`, not the always-emitted `# TYPE` header — grepping the bare family name matches the header and false-fails).
|
||||
- Cross-file contracts (grep the other files): the unit's `ExecStart`/bind posture, the deploy function + allowlist + dispatch + health gate, the planner selection (run `deploy-plan.py --target T --paths scripts/<svc>.py`, assert output == `<svc>` per target), the prometheus targets, the dashboard `json.load` + that its `targets[].expr` reference the emitted families.
|
||||
|
||||
## Gotchas hit while building gpu-exporter
|
||||
- Values are float-normalised (`397.40`→`397.4`); a test pinning the literal fails — assert the normalised value. Derive expected byte counts from real units (MiB×1048576, sysfs µW/1e6, temp1_input/1000), don't guess the multiply.
|
||||
- `xdg-open <PR-URL>` after opening the PR (repo law) — don't make Frank hunt in Gitea.
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
name: fleet-image-pull-freshness-audit
|
||||
description: "Answer \"when was the last time we pulled/updated engine?\" for digest-pinned dank-fleet images: git-history pin dating vs host podman state, pull-vs-restart distinction."
|
||||
---
|
||||
|
||||
Answer "when was the last time we pulled/updated <engine image>?" for a digest-pinned fleet endpoint (radiance/halogen/llama.cpp images on dankserver/dankclaw).
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Locate the pin.** `grep` the repo for the image digest prefix (e.g. `4c3f484564`) — pins live in `scripts/<svc>.env` (`FRANK_ARCHON_IMAGE` etc.) and are restated in `AGENTS.md` / runbooks.
|
||||
2. **Date the pin in git.** `git log --all --format='%h %ad %s' --date=short -S '<digest-full-prefix>'` — the OLDEST hit is the cutover that pulled it (renames and service-name sweeps produce later hits; ignore them).
|
||||
3. **Check the host state.**
|
||||
`ssh <host> 'sudo podman images --format "{{.Repository}}:{{.Tag}} {{.ID}} {{.Digest}} {{.CreatedTime}}"'`
|
||||
Compare the on-disk digest to the pin. `CreatedTime` is the VENDOR build stamp, not the pull time — it proves which build is on disk; the git commit dates the pull.
|
||||
4. **Distinguish pull from restart.** Digest pinning means the serve script NEVER auto-pulls; a fresh `ActiveEnterTimestamp` (`systemctl show <svc> -p ActiveEnterTimestamp`) is a restart, not an update.
|
||||
5. **Sweep side pulls.** A newer engine version may have been pulled for a one-off (calibration, KLD, bench) and removed after. `podman images` census + `git log -S` for the second digest; note when the side image is gone from disk (answer: "pulled <date>, no longer on disk").
|
||||
|
||||
## Facts to carry (radiance example)
|
||||
|
||||
- frank-archon pin `stilldeadcode/radiance@sha256:4c3f484564…` (1.0.13) pulled 2026-10-05 (cutover commit); reused unchanged at the 2026-10-08 w4dn8 cutover.
|
||||
- Engine 1.2.0 was pulled 2026-10-08 for calibration only; image absent from disk afterwards.
|
||||
- Cross-pin compatibility (e.g. RADGRAM2 grams byte-identical across pins) is documented in the runbook — cite it when explaining a calibration/serving pin split.
|
||||
|
||||
## Answer format
|
||||
|
||||
Table: pin-first-introduced commit+date (the pull), later cutovers reusing the same digest (no pull), side-image pulls, last container start (restart, explicitly labeled not-a-pull). Offer upstream freshness scout as next step; a digest change for an LLM endpoint requires the stop-and-mask bench-window law.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
name: fleet-judge-swap-calibrate
|
||||
description: "Swap the frank-intelligence-bench judge default and prove judge quality: fleet alias→weights map, known-truth calibration via scripts/judge-calibration.ts, per-backend reasoning-off dialect traps."
|
||||
---
|
||||
|
||||
# Fleet judge swap + calibration (frank-intelligence-bench)
|
||||
|
||||
## Fleet alias → weights (verified 2026-10-10 via ps on dankclaw; endpoints only serve aliases)
|
||||
- frank-archon = Qwen3.8-Flash-Next quant (radiance TP=2, dankserver:7770)
|
||||
- frank-dragoon = qwen38-flash-next-v2-swift15-abliterated.hgn, 180B MoE (halogen container `frank-large-halogen`, dankclaw:7770) — check `podman inspect frank-large-halogen --format {{json .Config.Env}} | grep HALOGEN_CHECKPOINT`
|
||||
- frank-zealot = Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K (llama.cpp, 3090, dankclaw:7771) — check `ps aux | grep <port>`
|
||||
|
||||
## Swap procedure
|
||||
1. `config/hosts.yml`: target host gains `roles: [target, judge]`; set `judge_default`. No-self-judge law: `judge == target` throws at enqueue; rotation = next fleet model.
|
||||
2. Judge load = AA-LCR + HLE only (SciCode/Terminal-Bench are unit-tested, judge-free).
|
||||
3. Mid-run: `run.json` pins its judge (provenance); default swap affects future runs only.
|
||||
4. Docs: `docs/endpoints.md` judge law section; commit with gates.
|
||||
|
||||
## Calibration — the ONLY accepted proof
|
||||
`bun scripts/judge-calibration.ts [n] [judges...]` — known-truth floor test: gold answers must be judged CORRECT, deterministically mutated (first number +7) must be INCORRECT. Reports accuracy, mean s/verdict, projected AA-LCR/HLE judge hours. Safe beside a live run (judge-only traffic).
|
||||
- Known-truth is a FLOOR test: catches broken judges, not lenient ones. Gold-vs-mutation is easy mode; small-model leniency on model-vs-gold edges needs real-verdict spot audits.
|
||||
|
||||
## Traps (all fired at least once)
|
||||
- **False-accuracy via transport death:** a 400-dying judge scores ~50% on this test (all verdicts collapse to INCORRECT, matching the mutation half by luck). ALWAYS inspect `err` on misses before trusting any agreement/accuracy number. Fix: if all misses share one `err=HTTP 4xx`, it is the harness, not the judge.
|
||||
- **Reasoning-off dialect split:** llama.cpp/radiance want `chat_template_kwargs:{thinking:false}`; halogen/vLLM-style servers 400 on `thinking` and honour `enable_thinking`. HostSpec field `reasoning_off: thinking|enable_thinking` (default `thinking`; dragoon needs `enable_thinking`). Wire-test via mock `bodies` capture.
|
||||
- **No historical judged rows?** Quarantined ledgers may pre-date judge usage (census first: `failureCensus` rows/phase), then use known-truth synthesis instead of re-judging.
|
||||
|
||||
## Baselines measured (for future swaps)
|
||||
- zealot (9B): 25/25 known-truth, 13 s/verdict → AA-LCR 1.1h, HLE 9.3h; leniency risk on hard edges (small judge + compliance tune).
|
||||
- dragoon (180B): 2 s/verdict → AA-LCR 0.2h, HLE 1.4h; family-bias caveat when target shares its base family.
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: fleet-served-model-id-rename
|
||||
description: "Rename the served model id (alias) on a dank-fleet LLM endpoint and re-key every consumer that keys on it, with live verification."
|
||||
---
|
||||
|
||||
# Fleet served-model-id rename (wire rename)
|
||||
|
||||
Renames the id an endpoint announces on the wire. The service, unit, script and env
|
||||
FILE names keep the old name (container/unit identity is stable) — only the served
|
||||
alias moves. Done this way on 2026-10-09: frank-primary→frank-archon,
|
||||
frank-large→frank-dragoon, frank-secondary→frank-zealot.
|
||||
|
||||
## 1. The knob, per engine
|
||||
| Endpoint | Knob |
|
||||
|---|---|
|
||||
| frank-primary (radiance, dankserver :7770) | `FRANK_PRIMARY_SERVED_NAME` in `scripts/frank-primary.env` → `--served-model-name` |
|
||||
| frank-large (halogen flash-server, dankclaw :7770) | `HALOGEN_MODEL_ID` in `scripts/frank-large.env` |
|
||||
| frank-secondary (llama.cpp, dankclaw :7771) | `ALIAS` in `scripts/frank-secondary.env` → `--alias` |
|
||||
|
||||
Also flip the serve-script defaults (`: "${ALIAS:=…}"`, the `:?` gate messages).
|
||||
`scripts/deploy-remote.sh` has no alias references — the deployer needs no edit.
|
||||
|
||||
## 2. Consumers that key on the id (all in ONE change)
|
||||
- Seams: `tests/frank_{secondary,large,primary}_unit.bats` — fixture env value,
|
||||
the argv assertion (`--alias`), the env grep pin, the `-e HALOGEN_MODEL_ID=` render
|
||||
assertion.
|
||||
- OpenWebUI (Dankclaw, rootless quadlet, `webui.db`): `model.id`+`name`,
|
||||
`access_grant.resource_id` (`resource_type='model'`), `openai.api_configs`
|
||||
per-backend `model_ids`. Backup first (`webui.db.bak-YYYYMMDD`; the DB is WAL-mode).
|
||||
No sqlite3 binary in the image → `podman exec -i open-webui python3 -` with a
|
||||
script on stdin. Then `systemctl --user restart open-webui.service`. A stale id
|
||||
fails SILENTLY: OpenWebUI intersects the backend `/v1/models` with `model_ids`, so
|
||||
the picker just goes empty.
|
||||
- Hermes (`/home/dank/Containers/hermes/data/config.yaml`, root-owned → sudo):
|
||||
`model.default`, the provider `models:` key, every `auxiliary:` route naming the
|
||||
endpoint. Backup, surgical `sudo sed -i`, sweep `grep -rln <old-id>` over the data
|
||||
dir (session JSONs are historical — leave), restart `hermes.service
|
||||
hermes-webui.service` (user units).
|
||||
- Dev harnesses: `"model": "frank-…"` literals in `scripts/bench-*.py`,
|
||||
`deep-probe.py`, `calib-push.py`, `fleet-probes-swift15.sh`, `autoresearch*`.
|
||||
- Docs: `shared/homepage/docs.html` (headings, api-id annotations, the
|
||||
curl/opencode/Pi examples), endpoint runbooks, `AGENTS.md` Model Config,
|
||||
`serverAppConfigs-docs/Agent/OMP Harness.md`, `Dankclaw/OpenWebUI.md`.
|
||||
- Harness side (`~/.omp/agent/models.yml` on the dev box) — ids and `modelRoles`
|
||||
must name the same string; two ports on one host = two providers.
|
||||
|
||||
## 3. Deploy + verify
|
||||
`./scripts/deploy.sh <host> --service <name>` per endpoint; keep the tree clean
|
||||
(untracked scratch files inside the repo fail the whole-tree gate — put probes in
|
||||
/tmp with a `node_modules` symlink to `tests/e2e/node_modules`).
|
||||
|
||||
Verify on `/v1/models`, NEVER on a chat echo: radiance echoes the REQUESTED model
|
||||
name (a 200 proves nothing), llama.cpp and halogen ignore the requested id.
|
||||
|
||||
```bash
|
||||
for u in 192.168.0.235:7770 192.168.0.143:7770 192.168.0.143:7771; do
|
||||
curl -fsS "http://$u/v1/models"; done
|
||||
```
|
||||
|
||||
OpenWebUI acceptance = a real browser as a family account: the welcome screen names
|
||||
the model and `POST /api/v1/chat/completions {"model":"<new-id>"}` answers.
|
||||
`GET /api/v1/models/list` returning `{"items":[],"total":0}` for that same account
|
||||
is a probe artifact, not a failure. Playwright notes: `locator.isVisible({timeout})`
|
||||
does NOT wait (returns immediately) — use `waitFor({state:'visible'})`; the composer
|
||||
is a contenteditable div, not a textarea; stable anchors are
|
||||
`getByRole('button', {name: /selected model/i})` and "How can I help you today?".
|
||||
|
||||
## 4. Trap: green deploy, old config
|
||||
`deploy.sh` plans from repo paths and gates on the port. A systemd drop-in under
|
||||
`/etc/systemd/system/<unit>.service.d/` can override `Environment=FRANK_*_ENV_FILE`
|
||||
and `ExecStart` to a staging worktree, so the deploy restarts a DIFFERENT tree and
|
||||
reports healthy (this bit frank-primary for a day after a quantization campaign).
|
||||
After an LLM deploy, prove the running shape:
|
||||
|
||||
```bash
|
||||
systemctl cat <unit> # drop-ins print at the bottom
|
||||
podman inspect <container> --format '{{json .Config.Cmd}}' | tr , '\n'
|
||||
```
|
||||
|
||||
Before retiring such a drop-in: diff staging vs checkout for serve script/unit/env,
|
||||
and `git merge-base --is-ancestor <staging-HEAD> HEAD` so no campaign tuning is
|
||||
lost. Move it to `~/.AGENTTRASH/YYYYMMDD-topic/` (never `rm`), `daemon-reload`,
|
||||
restart, re-read `/v1/models`.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
name: fleet-service-rename-cutover
|
||||
description: "Rename a Dank-fleet LLM service end-to-end (repo trio, deployer keys, tags, hosts): the 2026-10-10 frank-archon/dragoon/zealot cutover procedure."
|
||||
---
|
||||
|
||||
# Fleet service rename cutover (serverAppConfigs)
|
||||
|
||||
Procedure proven 2026-10-10 renaming the three LLM services to their wire
|
||||
aliases (repo `55f87fc`). For any future full service rename.
|
||||
|
||||
## Repo codemod
|
||||
1. `git mv` every tracked file whose path carries the old token
|
||||
(`frank-<old>` → `frank-<new>`, plus `frank_<old>` bats seams).
|
||||
2. Content sweep over all tracked text files:
|
||||
`git grep -Ilz '' | xargs -0 sed -i` with expressions for
|
||||
`frank-<old>`, `FRANK_<OLD>`, `frank_<old>` → new forms; add the
|
||||
`LLM_<OLD>` / `llm_<old>` / handler-name forms in the same pass.
|
||||
3. Known casualties to record: `scripts/patches/*.patch` context lines get
|
||||
rewritten (patches stop applying byte-exact); historical prose adopts
|
||||
the new name (one service, one name — Frank's rule).
|
||||
4. Fix sed-made-contradictory lore by hand: AGENTS.md Model Config
|
||||
paragraph, any "keeps the old names" sentences. Do NOT enumerate the
|
||||
retired names anywhere — the acceptance test is
|
||||
`git grep -cE '<old forms>'` == 0 (case-insensitive too).
|
||||
5. Rename rollback tags in lockstep (docs quote tag names):
|
||||
`git tag NEW OLD; git tag -d OLD`, push new, delete old refs on origin.
|
||||
6. Sweep the harness config comments too: `~/.omp/agent/models.yml`.
|
||||
|
||||
## Deployer keying (why the sweep is safe)
|
||||
`scripts/deploy-plan.py` SERVICE_ORDER/host sets + `services_for_path`
|
||||
trio mapping; `scripts/deploy-remote.sh` `service_is_allowed` case,
|
||||
`deploy_service` dispatch, `relink_unit <service>` (symlink
|
||||
`/etc/systemd/system/<name>.service` → `scripts/<name>.service`),
|
||||
markers keyed by service name under `~/.local/state/serverAppConfigs/
|
||||
deploy-state/`. `add_pending_services` AUTO-TRASHES markers whose service
|
||||
is retired — no marker surgery needed if you skip a full drain.
|
||||
|
||||
## Host cutover per endpoint (order matters)
|
||||
1. `sudo systemctl stop <old>`; rename `~/logs/frank_<old>.log` → new;
|
||||
`podman rm -f` the old container name(s);
|
||||
`sudo rm` the unit symlink + `multi-user.target.wants/<old>.service`;
|
||||
`sudo systemctl daemon-reload`; trash the marker to
|
||||
`~/.AGENTTRASH/<date>-fleet-rename/` with a NEUTRAL filename.
|
||||
2. `deploy.sh <host> --service <new-name>` (run from the main checkout;
|
||||
refuses on a worktree branch). The handler relinks under the new name,
|
||||
enables, restarts, health-gates.
|
||||
3. Do the harness-default-model endpoint LAST on its host, and never stop
|
||||
an endpoint while a deploy on the other host is approaching its e2e
|
||||
gate (the gate probes the live fleet).
|
||||
|
||||
## Acceptance
|
||||
`/health` 200 + `/v1/models` shows the new id (radiance echoes requested
|
||||
names — verify on `/v1/models` only); unit active+enabled; new container
|
||||
name running; `git grep` zero on every checkout; e2e gate green;
|
||||
Prometheus job up (note: llama.cpp needs `--metrics` in the serve script
|
||||
for the scrape — absence is a pre-existing gap, not a rename regression).
|
||||
Out of scope: retired endpoint names (`frank-medium/small/npu`) and git
|
||||
history — old strings there are expected.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: fleet-service-runtime-census
|
||||
description: "Answer \"what is service running right now?\" for a dank-fleet service: live endpoint probe plus host units, image ids, live-vs-repo env, and drop-ins."
|
||||
---
|
||||
|
||||
# Fleet service runtime census
|
||||
|
||||
Answer the question "what is `<service>` running currently?" with evidence, not from docs. Docs say what *should* run; the host says what *does*. Run all five probes, then report one table.
|
||||
|
||||
## 1. Probe the published port (works from any LAN box, no ssh)
|
||||
|
||||
```bash
|
||||
curl -s -m 10 http://<host-ip>:<port>/health; echo
|
||||
curl -s -m 10 http://<host-ip>:<port>/v1/audio/voices # per-endpoint extras: /v1/models, /metrics
|
||||
```
|
||||
|
||||
The extra route matters more than `/health` for identity claims:
|
||||
- llama.cpp and radiance **echo the requested model name** — a 200 proves nothing; read `/v1/models`.
|
||||
- halogen 404s `/completion`, `/ping`, `/props`, `/tokenize`.
|
||||
- The frank-tts façade answers `/health` **without dialing the engine**, so "façade healthy, engine down" is a real state; the voice list in that same response is the honest census.
|
||||
|
||||
## 2. Units + containers on the host (ssh as the deploy user, no sudo)
|
||||
|
||||
```bash
|
||||
ssh <host> 'systemctl --user status <svc>.service <engine>.service --no-pager \
|
||||
| grep -E "^(●| Active| Loaded)"; \
|
||||
podman images --format "{{.Repository}}:{{.Tag}} {{.ID}} {{.Created}}"; \
|
||||
podman ps -a --format "{{.Names}}\t{{.Status}}\t{{.Image}}"'
|
||||
```
|
||||
|
||||
- `active (running) since …` + `Up N hours (healthy)` is the answer; `activating (auto-restart)` with a ~50 ms exit is a **preflight refusal**, not an engine fault.
|
||||
- **NEVER prefix these podman calls with `sudo`.** Containers live in the deploy user's rootless storage; `sudo podman inspect frank-tts-engine` returns `no such object` while the plain form succeeds. Treating that as "container missing" is a false finding.
|
||||
|
||||
## 3. Drop-ins — the override that no health gate sees
|
||||
|
||||
```bash
|
||||
ssh <host> 'systemctl --user show <svc>.service -p DropInPaths; \
|
||||
systemctl --user show <engine>.service -p DropInPaths'
|
||||
```
|
||||
|
||||
`DropInPaths=` must be empty. A leftover bench drop-in (`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`) makes the next deploy boot a different model at a different shape while `/health` still answers 200 and the e2e gate passes.
|
||||
|
||||
## 4. Live env vs repo env — the only truth test
|
||||
|
||||
```bash
|
||||
# repo side (local checkout)
|
||||
grep -vE '^\s*#|^\s*$' scripts/<svc>.env
|
||||
# host side
|
||||
ssh <host> 'sudo cat /home/dank/serverAppConfigs/scripts/<svc>.env' | grep -vE '^\s*#|^\s*$'
|
||||
```
|
||||
|
||||
Diff them. Per repo law the env file beside the serve script **is** the authority; a doc page that disagrees with it is the bug, and host-side drift means an uncommitted edit. Also check `git log --oneline -3 -- scripts/<svc>.env` to confirm the host isn't ahead of git.
|
||||
|
||||
## 5. Report
|
||||
|
||||
Table of: unit state + since, image repo:tag@short-ID + age, backend/alias actually served (from step 1, not from the name), drop-ins, and whether live env == repo env. State explicitly when a fact came from docs rather than from a probe.
|
||||
|
||||
## Traps that produce false findings
|
||||
|
||||
- Digest-pinned images: the local image ID's *creation date* answers "is this stale" better than the tag; `latest` + `Pull=never` is a locally built image, not a registry pull.
|
||||
- `ssh` user context changes what podman sees (root vs rootless) — see step 2.
|
||||
- Two ports on one host are two providers in `models.yml`; a port mismatch is a rename or drift, not a config typo.
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
name: fleet-status-daily-report
|
||||
description: "Answer \"status update / what happened yesterday\" for the dank fleet: git-window record on origin/main plus a live per-host census (endpoints, units, RAM, DNS), with the reboot-attribution check that stops false incident causes."
|
||||
---
|
||||
|
||||
# Fleet status report (Dankclaw + Dankserver)
|
||||
|
||||
Use for "what's the state of the fleet", "what happened yesterday", or a post-incident sweep. Read-only.
|
||||
For one service's live state, prefer `fleet-service-runtime-census`; this is the fleet-wide sweep.
|
||||
|
||||
## 1. The day's record (repo is the ledger)
|
||||
|
||||
```bash
|
||||
git fetch origin --quiet
|
||||
git log --no-merges --since="YYYY-MM-DD 00:00" --until="YYYY-MM-DD+1 00:00" \
|
||||
--date=format:'%H:%M' --pretty='%ad %h %s' origin/main
|
||||
git log --merges --since=... --pretty='%ad %h %s' --date=format:'%H:%M' origin/main # PR numbers
|
||||
```
|
||||
Merge lines carry the PR number — that is the review trail; the non-merge log is what actually changed.
|
||||
Group commits into streams (endpoint names lead the subject) rather than listing 20 subjects.
|
||||
|
||||
## 2. Live census (parallel: one call per host + one endpoint sweep)
|
||||
|
||||
Endpoint sweep from any LAN box — `/health` is honest on all four (archon binds only after full load):
|
||||
|
||||
```bash
|
||||
for u in 192.168.0.143:7770 192.168.0.143:7771 192.168.0.235:7770 192.168.0.143:7772; do
|
||||
printf "%s %s\n" "$u" "$(curl -s -m 12 -w ' [%{http_code}]' http://$u/health | head -c 160)"; done
|
||||
curl -s http://192.168.0.143:7770/v1/models # dragoon lists EVERY engine it serves, incl. NPU-lane models
|
||||
```
|
||||
|
||||
Per host: `uptime -s; uptime -p`; `systemctl show <llm units> -p ActiveEnterTimestamp -p NRestarts -p DropInPaths`
|
||||
(`sudo` for the root LLM units, `--user` for the quadlets); `list-units --failed`; `free -h`;
|
||||
`journalctl -k --since "<evening>" -p err`; the host's own engine log tail.
|
||||
|
||||
## 3. Attribution rules (where reports go wrong)
|
||||
|
||||
- **Never call a host reboot from a container's start timestamp.** Check `uptime -s` first; if uptime spans the
|
||||
event, the trigger was a **container restart** — find it with
|
||||
`journalctl --user -u <name>.service --since … --until …` + `podman inspect <c> --format '{{.State.StartedAt}}'`.
|
||||
(This exact mistake mislabelled a 2026-10-19 Pi-hole incident as boot-caused; the law's remedy belongs after
|
||||
*any* restart, which is a different action item.)
|
||||
- `podman stats` is a 28-container wall — sort/limit it; RSS of the LLM tenants is what matters.
|
||||
- This podman's inspect template has **no `.Config.PodName`, no `.Config.GroupAdd`, and `.HostConfig.Devices`
|
||||
can read `[]` even when a quadlet `AddDevice=` worked**. Prove GPU use from the engine's own log
|
||||
(`ggml_vulkan: Found 1 Vulkan devices: … Radeon 8060S (RADV GFX1151)`), not from inspect.
|
||||
- Kernel lines: **count** occurrences (`journalctl -k --since '7 days' | grep -c "NETDEV WATCHDOG"`) before
|
||||
calling a one-off a trend.
|
||||
- Failed units: `systemctl show <u> -p ExecMainExitTimestamp -p ExecMainStatus --value` separates
|
||||
"broken since boot" from "broke yesterday" — and a nightly failure (e.g. logrotate) is a live risk, not noise.
|
||||
|
||||
## 4. Report shape
|
||||
|
||||
Stream table (what landed) → live-state table (what is true now) → **corrections to my own earlier record** →
|
||||
open items that need Frank's decision, each with the evidence line and the one action. Flag anything that would
|
||||
touch host config outside the repo as a decision, never a silent fix.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
name: fork-locked-gguf-endpoint-trial
|
||||
description: "Trial a fork-locked GGUF (custom tensor types: ternary/PQ2_0/PTQ1_0, vendor Hadamard metadata) as a replacement payload on a fleet llama.cpp endpoint: identify the fork, close provenance of local variants, build the right binary, and run one trap-guarded census window that measures VRAM, per-slot window, tool/vision/reasoning channels, and long-context stability."
|
||||
---
|
||||
|
||||
# Fork-locked GGUF trial on a live endpoint
|
||||
|
||||
Use when a candidate GGUF carries **non-stock tensor types** (custom type ids, vendor metadata such as `prism.hadamard.*`) so mainline llama.cpp refuses it or loads a sibling `F16` file silently with garbage. Pairs with `gguf-header-vram-fit-analysis` (paper pricing); this skill is the *measure-it* half.
|
||||
|
||||
## 1. Header census decides everything downstream
|
||||
Range-parse `resolve/main/<file>.gguf` bytes 0–48 MiB (metadata is ~11 MiB at 250k vocab; GGUF v3 strings are **u64**-length). Extract: `general.architecture`, `block_count`, `full_attention_interval`, `attention.head_count_kv`, `key_length`/`value_length`, `context_length` (target window > native ⇒ YaRN), `<arch>.nextn_predict_layers` (absent ⇒ no drafter, ever), tensor-type histogram, and per-tensor byte sizes from consecutive `offset` deltas ⇒ exact bpw.
|
||||
- `KV = full_layers × head_count_kv × key_length` elements/token **per cache**; only full-attention layers bill per token, linear/DeltaNet layers bill a fixed state per slot.
|
||||
- Ternary packs shrink weights only. At 262,144 tokens KV (16 full layers, 4×256) = 16.0 GiB f16 / 8.5 GiB q8_0 / 4.5 GiB q4_0 — 3× the 6.71 GiB of weights. **The cache is the wall, never the pack.**
|
||||
|
||||
## 2. Fork identification and tag ancestry
|
||||
- Map custom type id → name from the vendor fork (`ggml/include/ggml.h` on its branch, e.g. `GGML_TYPE_PQ2_0 = 142`, `PTQ1_0 = 143`).
|
||||
- `GET /repos/<fork>/compare/<tag>...<branch>` → `ahead_by` + commit subjects tells whether the newest **release** contains a fix, without reading history. Release workflows lie by omission: grep the workflow for the flag you need (e.g. prebuilt CUDA lacked `GGML_CUDA_FA_ALL_QUANTS` → quantized/mixed KV silently runs flash attention on CPU, ~2× slower; vendor issue #267).
|
||||
|
||||
## 3. Close provenance of payload variants already on disk
|
||||
`sha256sum` the local file (7.2 GB ≈ 15 s NVMe), then search the Hub (`/api/models?search=<name>`, `/api/models/<repo>?blobs=true` → sibling `lfs.sha256` + size). Equal size + different sha = re-weighted variant, not a corrupt copy. Known houses: `dealignai/*-CRACK-*` (uncensored), `ManniX-ITA/*-MTP-GGUF` (appended MTP head ≈ +450 MB, restores a drafter the official file lacks). Same size/geometry ⇒ the VRAM verdict carries across variants; behaviour (refusals, tool calls) does not.
|
||||
|
||||
## 4. Build recipe (Dankclaw, RTX 3090)
|
||||
`/usr/local/cuda` does not exist for non-interactive shells; `/usr/bin/nvcc` is a Debian wrapper. Mirror the known-good cache instead of guessing:
|
||||
```
|
||||
cmake -G Ninja -B build-cuda-prism -DCMAKE_BUILD_TYPE=Release \
|
||||
-DCMAKE_CUDA_COMPILER=/usr/bin/nvcc -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13 \
|
||||
-DCMAKE_CUDA_ARCHITECTURES=86-real -DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON
|
||||
```
|
||||
nvcc 12.4 refuses gcc 15 (`gcc-13`). FA template instances dominate the build (~150/804 in 12 min); park it with `pkill -f "[r]un-<build>"` + `pkill -x ninja` before timing anything — the bracket stops `pkill -f` matching your own ssh cmdline.
|
||||
|
||||
## 5. One-window census (never two outages)
|
||||
Fetch through a committed script (`-fsSL --fail-with-body`, byte count + LFS sha, no resume, no token in argv), then stage `/tmp` throwaways: candidate env fed to the **repo serve script** via its `*_ENV_FILE` override (never edit remote config), and a runner whose `trap restore EXIT` always kills the candidate by pid and `systemctl start` the production unit + health-polls to 200.
|
||||
Order inside the window: copy the production log aside (serve scripts truncate it) → stop unit → wait `memory.used < 1500` → boot candidate → `/health`, boot ledger (`KV buffer`, `compute buffer`, `flash`), `nvidia-smi --query-compute-apps`, `/v1/models`, `/props`, `/slots` (per-slot `n_ctx` is the harness contract, `pool/parallel`), chat probe (does thinking land in `reasoning_content` or `content` — a vendor `--reasoning-in-content true` default breaks the harness channel), tool-call probe (name + valid-JSON `arguments`), 512 px solid-colour vision probe, then a ~190k-token `/completion` fill reading `timings`. Kill the server, then `llama-bench` dtype arms (`-c <pool> -np <slots> -ctk/-ctv -b 2048 -ub 2048 -p 4096 -n 128 -r 1`) under a 1 Hz `nvidia-smi` sampler, `peak_gpu_mib=$(sort -n | tail -1)` per arm. Skip `-mmp 1` unless `-mmproj` is also passed.
|
||||
|
||||
## 6. Gates that outvote the memory table
|
||||
Vendor `KNOWN_ISSUES.md` + open issues (quantized-KV resets near 200k, inert ngram/`--cache-reuse` on hybrid memory, malformed `tool_calls`, prompt-cache loss from re-rendered reasoning); base alignment (an uncensored endpoint replaced by a censored base is a fleet-wide capability loss); and which harness roles point at the endpoint (`models.yml` + `modelRoles`) — tool-heavy roles fail loudly on a `tool_calls` defect. Repo seams pinning `-ctk/-ctv` literals must be re-keyed to read the env file, so a retune moves one file, not two.
|
||||
@@ -0,0 +1,52 @@
|
||||
---
|
||||
name: frank-bench-run-ops
|
||||
description: "Operate frank-intelligence-bench runs on the dank fleet: status census, full void + fresh launch, judge cutover, supervisor restart law, watchdog timer, fault-autopsy snippets."
|
||||
---
|
||||
|
||||
Runbook for /home/blank/Projects/frank-intelligence-bench (Bun/TS). Companion to frank-bench-supervisor-recovery (row-voiding for poison); this skill is the ops loop around it.
|
||||
|
||||
## Status census (run from repo root)
|
||||
```bash
|
||||
export RID=$(ls -t runs | grep -m1 frank-archon) # live run id
|
||||
tail -2 runs/$RID/log.txt
|
||||
python3 - <<'PYEOF' # latest-wins cohort census (mirrors score.ts semantics)
|
||||
import json, os, collections
|
||||
led = f"runs/{os.environ['RID']}/ledger.jsonl"
|
||||
best = {}
|
||||
for l in open(led):
|
||||
r = json.loads(l)
|
||||
if r["phase"] != "grade": continue
|
||||
k = (r["eval"], r["instance"], r["repeat"])
|
||||
if r["ok"] and (k not in best or r["ts"] > best[k]["ts"]): best[k] = r
|
||||
elif k not in best: best[k] = r
|
||||
by = collections.defaultdict(lambda: [0,0])
|
||||
for k,v in best.items(): by[k[0]][0]+=bool(v["grade"]["correct"]); by[k[0]][1]+=1
|
||||
for e,(c,n) in sorted(by.items()): print(f"{e}: {c}/{n} = {100*c/n:.1f}%")
|
||||
PYEOF
|
||||
```
|
||||
`pgrep -f "cli.ts _exec"` = supervisor; `systemctl --user is-active fib-watchdog.timer` = watchdog; last ledger ts older than ~5 min = suspect stall.
|
||||
|
||||
## Launch laws (each earned by an outage)
|
||||
- Code fix + running supervisor = FIX IS NOT LIVE. Restart supervisor: `pkill -f "cli.ts _exec"`, then relaunch.
|
||||
- Correct detached form from harness bash: `cd runs/$RID && setsid nohup bun run ../../src/cli.ts _exec run.json >> log.txt 2>&1 < /dev/null & disown`. NEVER the subshell form `( ... & )` — reaped by cgroup within minutes.
|
||||
- Preferred launch = `bun src/cli.ts run HOST --judge JUDGE --detach` (in-process spawn + unref, proven 6 h+).
|
||||
- Watchdog = systemd user timer `fib-watchdog.{service,timer}` (~/.config/systemd/user), 10 min, Persistent, runs `bun src/cli.ts worker` (idempotent: exits when heartbeat fresh, takes over stale runs via 30 s mtime gate).
|
||||
|
||||
## Void all results + fresh run (model/endpoint change)
|
||||
1. `pkill -f "cli.ts _exec"`; sleep 1.
|
||||
2. `mv runs/<id> .AGENTTRASH/run-voided-<reason>-$(date +%s)`; `mv reports/<model>/* .AGENTTRASH/`.
|
||||
3. queue.json: set `runs: []` (python json edit).
|
||||
4. New run: `bun src/cli.ts run frank-archon --judge frank-dragoon --detach`.
|
||||
5. Log the void in docs/log.md (reason, quarantined scope) and commit — never silently erase scores.
|
||||
|
||||
## Judge cutover (zealot→dragoon pattern)
|
||||
Void ONLY LLM-judged evals' `phase=grade` rows (keep ALL gen rows — target output is judge-independent); set `judge` in run.json; restart supervisor; resume rejudges automatically (grade keys incomplete). SciCode = pytest, judge-independent, never voids for judge changes.
|
||||
|
||||
## Fault-autopsy snippets
|
||||
- Engine-fault census: grade rows with `ok:false` + empty text = engine, not model (signature-default trap: "AssertionError × N" with empty text = grades never executed).
|
||||
- Runaway-chain detector (HLE loops): gen rows with `finish=="length"` AND completion_tokens==max AND empty text = reasoning degenerated into repetition; measure pre/post any quantization change to separate model behavior from engine artifacts.
|
||||
- Census-by-eval: `collections.Counter((r["eval"], r["phase"], r["ok"]) for r in rows if r["ts"] > <cutover>)`.
|
||||
- Verdict-parser bugs: probe the live judge on a minimal case (`judgeAalcr`/`judgeHle` via `bun -`), inspect raw head+tail; llama.cpp thinking models may return content:"" with everything in reasoning_content, and prose verdicts when their JSON scaffold breaks.
|
||||
|
||||
## Reporting etiquette (Frank's rules)
|
||||
Quote score as: per-rep %, test avg@1 / avg@3, dev, index-lite; label provisional until all reps land; publish the runaway/parse-rate census next to any handicapped number. Dashboard :8787; `bun src/cli.ts report` renders the trio; `analyze` prints failure census.
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
name: frank-bench-supervisor-recovery
|
||||
description: "Recover a frank-intelligence-bench run after engine faults: detect poison grade rows, void them, restart supervisors with fixed code, judge cutover, watchdog for multi-day runs."
|
||||
---
|
||||
|
||||
Recover frank-intelligence-bench runs on the dank-fleet after engine faults (poison graders, judge swaps, supervisor deaths).
|
||||
|
||||
## Laws (friction-earned 2026-10-10/11)
|
||||
1. **A code fix is only real after the supervisor restarts.** Bun loads code at spawn; an on-disk fix never reaches the running process. Kill `_exec`, relaunch — takeover law makes resume cheap.
|
||||
2. **A judge cohort is all-or-nothing.** Never mix judges on one eval's grade rows. Void ALL grade rows for LLM-judged evals (`phase == "grade"`), KEEP all `gen` rows (target output is judge-independent) — resume rejudges from cache.
|
||||
3. **Poison census signatures:** grade rows with empty text + forced FAIL = engine, not model. `unparseable verdict` storm = judge output class (empty `content` + chain in `reasoning_content`; or 2048-tok truncation before verdict JSON). Default census sig `AssertionError` masked a poison shift — treat mass-default sigs as red flags.
|
||||
4. **Detached launch form matters:** from a short-lived harness shell, `(setsid nohup cmd & )` (subshell) gets cgroup-reaped within minutes — silent death, stale `worker.pid`, no log line. Proven form: `setsid nohup cmd >> log.txt 2>&1 < /dev/null & disown` as the tail of the same shell (survived 6 h overnight).
|
||||
|
||||
## Void-and-resume procedure
|
||||
```bash
|
||||
pkill -f "cli.ts _exec"
|
||||
python3 - <<'PYEOF'
|
||||
import json, pathlib, shutil, time
|
||||
led = pathlib.Path("runs/<RUN_ID>/ledger.jsonl")
|
||||
rows = [l for l in led.read_text().splitlines() if l.strip()]
|
||||
keep, void = [], []
|
||||
for l in rows:
|
||||
r = json.loads(l)
|
||||
if r["eval"] in (<EVALS>) and r["phase"] == "grade" and r["ts"] > "<CUTOVER_ISO>":
|
||||
void.append(l)
|
||||
else:
|
||||
keep.append(l)
|
||||
shutil.copy(led, f".AGENTTRASH/ledger-voided-{int(time.time())}.jsonl")
|
||||
led.write_text("\n".join(keep) + "\n")
|
||||
print(f"voided {len(void)}; kept {len(keep)}")
|
||||
PYEOF
|
||||
cd runs/<RUN_ID> && setsid nohup bun run ../../src/cli.ts _exec run.json >> log.txt 2>&1 < /dev/null & disown
|
||||
```
|
||||
- Judge cutover variant: void ALL `phase==grade` rows for the judged evals (no ts filter), edit `run.json` `judge` field, relaunch.
|
||||
- Validate the judge endpoint first: live probe both directions (correct + incorrect candidate); a good judge returns bare verdict JSON fast.
|
||||
- Resume proof = a specific known step flipping FAIL→PASS (or vice versa) in the log within minutes.
|
||||
|
||||
## Watchdog for multi-day runs
|
||||
No cron on the Arch workstations — systemd user timer, idempotent (`cli.ts worker` exits instantly when supervisor heartbeat is fresh; takes over via pid-file 30 s freshness gate):
|
||||
`~/.config/systemd/user/fib-watchdog.service`: Type=oneshot, WorkingDirectory=repo, `ExecStart=/usr/bin/bun src/cli.ts worker`, stdout/stderr `append:.../runs/watchdog.log`.
|
||||
`fib-watchdog.timer`: OnBootSec=5min, OnUnitActiveSec=10min, Persistent=true, WantedBy=timers.target.
|
||||
`systemctl --user enable --now fib-watchdog.timer`; verify `loginctl show-user <u> -p Linger` = yes (survives logout/reboot). Prove idempotence: log line "has a live supervisor — worker exits".
|
||||
|
||||
## Scoring truth after repair
|
||||
Report renders from ledger with latest-wins via `ok` filter (no explicit dedup): ok=false rows drop, recovered key counts once. UI `done` counts unique `(eval,instance,repeat)` completed keys. SciCode denominators: 341 steps/rep (not 1120/3-rep arithmetic traps).
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
name: frank-job-hunt-grilling-state
|
||||
description: "Resume frank-job-hunt: repo is BUILT and deployed (agents live on Dankserver, ntfy on Dankclaw); grilling closed, design v2 shipped; lists pending ops items and invariants."
|
||||
---
|
||||
|
||||
## Status (2026-10-11): design grilling CLOSED (design v2 signed); repo built, deployed, self-verifying.
|
||||
|
||||
Repo: ~/Projects/frank-job-hunt (workstation) — Gitea `dank/frank-job-hunt` on Dankclaw; Dankserver clone runs timers (sweep 07:30, digest 08:15, Linger=yes). Pre-commit hooks: bun test + tsc + docs-link + secret scan. Tests: 69, strict TS (no `any`; Records for static tables; named contract types).
|
||||
|
||||
## Machine
|
||||
- `agents/sweep.ts`: registry → greenhouse/lever/ashby/workday public APIs → gates → frank-zealot score → `jobs/*/inbox.json|gated.json`; `--dry --ats=` flags; per-site fetch dedupe; per-entry error census.
|
||||
- `agents/digest.ts` → ntfy (ntfy.franksdank.cloud, LAN; topic secret in `~/.config/frank-job-hunt.env` 600); degrades to file + exit 1.
|
||||
- `agents/review.ts`: funnel metrics (weekly rate 8-10, callback floor 4%).
|
||||
- `agents/verify-registry.ts --write`: monthly ATS-locator probe/re-attribute (14 scout mis-attributions repaired).
|
||||
|
||||
## Truths earned in build
|
||||
- Registry truth: **21 live API sites, ~169 manual lane** (vanity Workdays, self-built ATS incl. majors, neogov). Scouts' ats attribution ≈ guess — verify before trusting counts.
|
||||
- Gates do the discriminating; zealot only ranks survivors (9B inflates everything: recruiters scored 91 before family gate).
|
||||
- Gate law: role FAMILY gates unconditionally (recruiter/sales/people-ops/TPM/solutions, no infra override); topic negatives get infra-token precedence ("full-stack platform engineer" passes). Comp NEVER gated; unstated location passes to human.
|
||||
- Metro regex = Puget Sound list only (Tuolumne CA leak fixed — WA localities, no CA decoys).
|
||||
- ATS tokens: first DNS label (greenhouse.harpereng.com→harpereng); vanity workday URLs are NOT CXS-resolvable → manual lane.
|
||||
- Zealot (llama.cpp thinking model): send `chat_template_kwargs:{enable_thinking:false}` + max_tokens ~600; verdict in `content` then; `reasoning_content` fallback kept.
|
||||
|
||||
## Pending (user-side / next session)
|
||||
1. Porkbun API 503'd: public A record `ntfy → 71.227.227.172` still missing (LAN works; add via UI when API recovers, for off-LAN phone).
|
||||
2. Frank: phone ntfy app subscribe (topic in env file); LinkedIn CSV export → `contacts/`.
|
||||
3. First production digest = tomorrow 08:15; check `journalctl --user -u frank-job-digest.service` + phone.
|
||||
4. bg_6 (last session): full dry-sweep proof of family gate (top-10 must be engineering-only) — check `agent://`/proc result if pending.
|
||||
5. Resume Matcher compose (`agents/resume-matcher/compose.yaml`) not deployed — needs image digest pin on Dankserver.
|
||||
6. `prep/` STAR/LP bank content fill (daily-loop.md scaffolded).
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: frank-job-hunt-maintenance
|
||||
description: "Maintain frank-job-hunt agents: live-census gate hardening loop, registry locator verification, and the workstation/Dankserver/Dankclaw deploy-sync ritual."
|
||||
---
|
||||
|
||||
# frank-job-hunt maintenance
|
||||
|
||||
Repo `~/Projects/frank-job-hunt` (Gitea `dank/frank-job-hunt` on Dankclaw 192.168.0.143:3000). Production: Dankserver user timers `frank-job-sweep.timer` 07:30, `frank-job-digest.timer` 08:15 (`Linger=yes`). ntfy relay: Dankclaw quadlet `ntfy.service`, route `ntfy.franksdank.cloud` (no anubis), LAN record in `hosts/custom.list` (Pi-hole v6 file at `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list`; v6 API paths are hostile — edit the file, then `podman exec pihole pihole restartdns`). Secrets: `~/.config/frank-job-hunt.env` (600, workstation + Dankserver).
|
||||
|
||||
## Law (validated 2026-10-11)
|
||||
- The fleet judge (`frank-zealot`, dankclaw:7771) is a **ranker, never a gate** — it inflates anything reaching it (non-technical roles scored 88-91). All discrimination lives in deterministic gates (`agents/lib/gates.ts`); zealot only ranks survivors.
|
||||
- Thinking-model fault: verdict lands in `reasoning_content` with `content:""` → judge sends `chat_template_kwargs:{enable_thinking:false}` (0.7s clean JSON).
|
||||
- **Family beats topic**: role family (recruiter/sales/finance/PM/designer/people-ops/revops/…) gates `gate:scope` unconditionally — no infra-token override ("DevOps Recruiter" is recruiting). Compound technical titles keep the `INFRA_TITLE` override via `NEG_TITLE` branch.
|
||||
- **gate:region**: remote is US-only (visa reality). "Remote - Bangalore" = onsite abroad → gate; `US` regex pass wins over `NONUS` match ("U.S. & Canada" passes).
|
||||
|
||||
## Gate hardening loop (live-census, TDD)
|
||||
1. `bun agents/sweep.ts --dry` (full lane ≈ 30–90 s; 852 postings ≈ 62 pass / 790 filed).
|
||||
2. Classify leaks in the top-10: family gap / region gap / scoring tail. Accept long tail (Director/Founder edge cases) — digest + human curation is the design.
|
||||
3. Token fix in `gates.ts` regexes (`NEG_FAMILY` = no override; `METRO`/`US`/`NONUS` for arrangement). NEVER dedupe-check by memory — read file first; the file has bitten line-offset edits repeatedly.
|
||||
4. Pin every leak with a named regression test in `agents/test/gates.test.ts` before/with the fix.
|
||||
5. `bun test agents/test/ && bun x tsc --noEmit` — hooks re-run these at commit.
|
||||
6. Ship: `git commit` (pre-commit = tests + tsc + docs-link + secret scan) → `git push` → `ssh dankserver 'cd ~/Projects/frank-job-hunt && git pull'` (timers pull from master; Dankserver drift = tomorrow's run is stale).
|
||||
|
||||
## Registry verification (monthly, `agents/verify-registry.ts`)
|
||||
Scouts' `ats`/`ats_identifier` are guesses (14/84 were wrong). Ritual:
|
||||
1. `bun agents/verify-registry.ts` (dry, ~70 s serial) → REPAIR census.
|
||||
2. `--write` applies; repairs with live jobs preferred over empty boards (0-job boards 404 later).
|
||||
3. Unresolved = honest manual lane (vanity Workdays, self-built ATS incl. majors, neogov) — NOT a code bug; per-entry error census in sweep output is the same signal.
|
||||
4. Update `targets/README.md` census lines + `docs/log.md`; commit/push/pull (same ship step).
|
||||
|
||||
## Pitfalls
|
||||
- `--ats=<x>` smoke filters the census line — "N API-sweepable" reflects the filter, not the registry; don't diagnose a phantom bug.
|
||||
- Workday vanity careers URLs (`careers.uw.edu`) are NOT CXS-API-resolvable; tenant-only identifiers are (site path required).
|
||||
- Root `README.md` vs `targets/README.md` name collision in edit anchors: use full path headers.
|
||||
- Zealot endpoint probe: `curl http://192.168.0.143:7771/v1/models`.
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: frank-model-bench-grilling-state
|
||||
description: "Resume the frank-model-bench design grilling: AA Intelligence Index v4.3.2 research brief, fleet endpoint facts, dataset gating, and all settled design decisions for /home/blank/Projects/frank-model-bench."
|
||||
---
|
||||
|
||||
# frank-model-bench — Grilling State (design settled 2026-10-09, awaiting build go-ahead)
|
||||
|
||||
Target dir `/home/blank/Projects/frank-model-bench` is EMPTY (no git). Init from zero.
|
||||
|
||||
## Settled design decisions
|
||||
- **Scope**: Tiered reproduction of AA Intelligence Index v4.3.2 (10 evals, 4 categories; Agents 30 / Coding 20 / General 30 / Science 20).
|
||||
- **Tier A (local, self-graded)**: SciCode, Terminal-Bench 4.0 (podman on Dankclaw via ssh one-shot), AA-LCR, HLE incl. image questions (vision towers).
|
||||
- **Tier B (external dep)**: GDP.pdf (LiteParse+vision+judge).
|
||||
- **Tier C (never run; AA reference rows only)**: AA-Briefcase, GDPval-AA, AutomationBench-AA, AA-Omniscience (public set is 600/6000 — demoted), CritPt (key-gated grader API 10/24h — demoted).
|
||||
- Locally-run weight ≈ 45% of AA index weight. Headline = "Index-lite" (category-weighted over run evals, formula on-page).
|
||||
- **Stack**: Bun/TS orchestrator (UI, queue, ledger, reports) shelling out to commit-pinned vendored uv venvs per upstream harness (inspect_ai, Harbor, SciCode/HLE scripts). `evals.yml` manifest pins git URL+SHA+entrypoint+args+repeats per eval.
|
||||
- **UI**: local web dashboard, localhost bind only, SSE live tail, start wizard; CLI `frank-bench`.
|
||||
- **Endpoints**: registry `hosts.yml` (name, base_url, api_key_ref, model, ctx, backend, role, cap). Discovery = probe known fleet hosts ports 7770-7799 `/v1/models` + `/props`, confirm-then-append. Fleet: frank-archon dankserver:7770 (radiance TP=2 R9700, 200k), frank-dragoon dankclaw:7770 (halogen, 200k×2), frank-zealot dankclaw:7771 (llama.cpp 3090, 200k×2). All OpenAI-compat + API key + vision.
|
||||
- **Judge**: self-hosted fleet model ONLY, default frank-zealot; hard no-self-judge rotation rule. Scores labeled `local-judge`, drift vs AA hosted graders expected/documented.
|
||||
- **Execution**: per-endpoint runs, strictly serial queue, one request in flight per eval (per-endpoint cap), temp 0.6 reasoning ON, max_tokens cap 32,768, AA-style 30-attempt retry, per-sample timeout = ctx ÷ measured decode rate.
|
||||
- **Durability**: per-sample JSONL ledger is canonical artifact; resume mid-eval; reports commit to git (reports/<model>/<run-id>/{index.html,data.json,hf-card.md,manifest+ledger SHA-256}), ledgers gitignored.
|
||||
- **AA comparison**: free Data API key → `bench sync-aa` pins headline indices; per-benchmark refs hand-seeded in `references/aa-scores.yml` (source URL + fetched-at + index version per row). Free=100 req/24h headline-only; Pro=500 req/24h per-benchmark (stubbed for later).
|
||||
- **Datasets**: `bench init-data` checklist (hf auth login + accept terms cais/hle, surgeai/GDP.pdf, Harbor Hub); checksum cache ~/.cache/frank-bench/datasets; gated evals refuse with fix-URL and continue suite.
|
||||
- **Methodology updates**: version-pinned manifests + immutable history; update = one commit: evals.yml bump + docs/methodology rewrite + log.md + resync snapshot. AA cadence ~2-3 minor versions/week; changelog artificialanalysis.ai/changelog.
|
||||
- **KB**: `docs/` inside repo, OKF v0.1 (type required; index.md per dir; log.md newest-first; .okf/config.yaml from omp-docs); Obsidian-wikilink convention per serverAppConfigs-docs.
|
||||
- **Git**: Gitea on Dankclaw remote, master base, worktree branch per milestone, pre-commit: bun test + shellcheck + OKF validate. Secrets: ~/.config/frank-bench/secrets.env 600.
|
||||
- **Tests**: bun test w/ scripted mock OpenAI endpoint (full run→ledger→HTML offline); Playwright dashboard e2e; `bench smoke` manual real-endpoint.
|
||||
- **Milestones**: M1 scaffold+registry+mock; M2 SciCode E2E w/ real report trio; M3 rest of Tier A + queue/resume; M4 dashboard UI + discovery wizard; M5 GDP.pdf + AA snapshot + peer tables.
|
||||
- **Reports**: single-page HTML, offline-openable, no external assets; per-benchmark rows: local | AA ref | Δ | repeats | n | graded-by; Tier C rows "AA reference only"; speed strip captured (no AA speed compare).
|
||||
|
||||
## Key research facts
|
||||
- AA Data API: base https://artificialanalysis.ai/api/v2, `x-api-key` header, OpenAPI at /api/v2/openapi; free = /language/models/free headline indices.
|
||||
- Eval repos: SciCode1/SciCode (HF, Apache), cais/hle (gated, MIT, 2500 q, ~0.27GB), ArtificialAnalysis/AA-LCR (public, Equality Checker prompt public), surgeai/GDP.pdf (gated, Gemini 3.5 judge, inspect_ai), harbor-framework/terminal-bench 4.0 (66 tasks, CONTAINERS_BACKEND=podman, Harbor Hub gated), AA-Omniscience-Public 600q, CritPt public grading API 10/24h.
|
||||
- AA org repos: Stirrup (MIT agent harness), aa-agentperf-local, aiperf, optima, StirrupJS.
|
||||
- AA settings: temp 0/0.6, 16k+ max tokens, 30 retries, 1-5 repeats pass@1; Omniscience index [-100,100] accuracy−hallucination.
|
||||
- Search chain web/kagi→web/searxng (search.franksdank.cloud) verified working 2026-10-09.
|
||||
|
||||
## Accepted risks (user-informed)
|
||||
- Elo-pairwise agentic evals unreproducible locally; self-hosted judge drift; Omniscience (top quant-damage signal) not run; HLE vision inclusion diverges from AA text-based posture.
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
name: frank-primary-decode-tuning
|
||||
description: "Tune frank-primary (radiance TP=2) decode throughput: benchmark harness usage, flag levers, known optima and boot-failure walls."
|
||||
---
|
||||
|
||||
# frank-primary decode tuning
|
||||
|
||||
Harness: `bash autoresearch.sh` (dev box) scp's `scripts/frank-primary.env` + `scripts/autoresearch-frank-primary.sh` to dankserver, stops prod (`systemctl stop frank-primary`, NEVER mask — relinked unit), boots scratch container `arbench` on :7770 (fresh prefix-cache tier at `$STATE_DIR/kvcache/arbench` for cold prefill), polls health 720s, sends ONE greedy completion (fixed 140k-char repo-corpus prompt -> 43,627 tokens, 256-token decode), parses `timings` into METRIC lines, tears down, restores prod via EXIT trap. ~4 min/run; decode noise ~0.3%.
|
||||
|
||||
Metric: `METRIC decode_tps` = `timings.predicted_per_second` at 43.6k depth.
|
||||
|
||||
## Optima (measured 2026-10-06, keep these)
|
||||
- `FRANK_PRIMARY_SPEC_TOKENS=2` (unimodal: 1->31.89, 2->32.40, 3->31.72, 4->24.57 t/s; acceptance collapses past 3)
|
||||
- `FRANK_PRIMARY_MAX_SEQS=3` with `MAX_MODEL_LEN=200000` (600k KV; +4.2% total; 4 slots regresses)
|
||||
- `FRANK_PRIMARY_EXPERT_VS_CACHE=0.82` bracket-confirmed (0.78 fails boot, 0.84 -8.9%, 0.87 collapse)
|
||||
- `TP_WIRE=wht6`, `MAX_BATCHED_TOKENS=2048` — REQUIRED at seqs=3: bf16 wire and MNBT 4096 both fail boot (VRAM)
|
||||
|
||||
## Non-levers (decode-neutral, don't spend runs)
|
||||
GPU_HEADROOM_MIB (48 = neutral), HOST_POOL_MIB 49152 (neutral), MAX_MODEL_LEN any change (regresses: bigger KV steals resident weights), prefix-cache tiers (prefill-side).
|
||||
|
||||
## Rules
|
||||
One flag per iteration; keep only > +1% decode over best; a boot failure is `crash` (trap restores prod). Constraint floor: MAX_SEQS*MAX_MODEL_LEN >= 400000.
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
name: frank-tts-add-cloned-voice
|
||||
description: "Add or repair a cloned voice on the dank fleet: pack builder, the reference.wav contract, the boot warm hook (argv-cap and env-quote laws), and how to prove a voice really answers."
|
||||
---
|
||||
|
||||
# Adding (or repairing) a cloned voice on the dank fleet
|
||||
|
||||
Repo law lives in `serverAppConfigs-docs/Dankclaw/Voice Packs.md` and
|
||||
`Frank TTS Engine.md` — read those for the pack format. This skill is the
|
||||
**operating sequence plus the traps that cost real time on 2026-10-18**.
|
||||
|
||||
## Stack, fast
|
||||
|
||||
- `frank-tts-engine` — arm A `qwentts.cpp tts-server`, container-internal `:8880`,
|
||||
publishes **no port**. Reach it with `podman exec frank-tts-engine curl …`.
|
||||
- `frank-tts` — OpenAI façade on host `:7772`; resolves voice **by name**.
|
||||
- Packs: `/home/dank/voices/<name>/` with `reference.wav` (+ manifest, `.spk`,
|
||||
`.rvq`). Builder: `scripts/build-voice-pack.sh` (crop 19.84–25 s, first 10 s of
|
||||
any source is forbidden).
|
||||
- Weights: `/home/dank/models/Qwen3-TTS-GGUF/` —
|
||||
`scripts/download-frank-tts-engine-qwentts.sh [--verify-only]` (1.4 GiB, ~18 s
|
||||
on the LAN; verifies byte count + Hub LFS sha256).
|
||||
|
||||
## Sequence
|
||||
|
||||
1. Build the pack (dry-run first — sizes are arithmetic and asserted).
|
||||
2. Deploy the engine only if its files changed:
|
||||
`./scripts/deploy.sh dankclaw --service frank-tts-engine`
|
||||
(first run builds the image: ~10 min recursive source clone).
|
||||
3. Register/warm: `podman exec frank-tts-engine /app/warm-voices.sh`
|
||||
— registers **every** pack each run (the engine replaces a name it already
|
||||
holds; `tools/tts-server.cpp:148`), then synthesizes twice per voice.
|
||||
4. Prove from the consumer side, not the config:
|
||||
`curl -s localhost:7772/v1/audio/voices` then a real
|
||||
`POST /v1/audio/speech` and listen to the bytes.
|
||||
|
||||
## Trap 1 — the kernel's argument cap (this broke the first deploy)
|
||||
|
||||
`/usr/bin/curl: Argument list too long` in the warm log ⇒ a request body was
|
||||
passed as **one argv string**. Linux caps a single argument at
|
||||
`MAX_ARG_STRLEN = 131 072 bytes`; a ~1 MB `reference.wav` base64's to ~1.27 MB.
|
||||
curl never runs, and hooks report it as `HTTP no response` while `/health` is
|
||||
green. Fix = write the body to a file and use `--data-binary @file`; feed the
|
||||
payload through a redirection, not `$(base64 …)`.
|
||||
Regression-test shape: **real** curl + real listener + a fixture >128 KB, so the
|
||||
kernel (not a stub) produces the failure. Any pack over ~96 KB trips it.
|
||||
|
||||
## Trap 2 — two readers of the env file disagree about quotes
|
||||
|
||||
`bash set -a; source` strips `"…"`. podman's `--env-file` (how the unit really
|
||||
reads it) does **not**. A value interpolated into a JSON body therefore arrives
|
||||
quoted and invalidates it. Test both readers, not just `source`.
|
||||
|
||||
## Trap 3 — locally built images skip rebuilds on a stale hash
|
||||
|
||||
`build_compose_image` compares a `cfhash` label; it must cover every file the
|
||||
`Containerfile` `COPY`s, or an edited `app.py`/hook deploys "successfully" while
|
||||
the container runs old code.
|
||||
|
||||
## Measured numbers (2026-10-18, deployed build — retire the trial's ladder)
|
||||
|
||||
| thing | value |
|
||||
|---|---|
|
||||
| first synthesis of a freshly registered voice | 1 883 ms |
|
||||
| later short requests | 1 226–1 283 ms |
|
||||
| ~110-word request (repeat) | 9.4 s / 9.8 s — tracks generated audio, not state |
|
||||
| whole boot warm, 3 packs | 15.2 s |
|
||||
| engine resident | ~4.0 GiB |
|
||||
|
||||
The trial's `6.2–8.0 s cold → 0.5 ms warm` claim does **not** reproduce; warming
|
||||
is for registration (registry is memory-only, re-register ≠ warm) plus ~0.6 s.
|
||||
|
||||
## Ops notes
|
||||
|
||||
- `systemctl --user stop` **removes** the quadlet container — `podman start` then
|
||||
finds nothing. To run the engine without its `ExecStartPost` hook, use a
|
||||
separate container, not stop+start.
|
||||
- `podman cp` of a fixed script into a running container is a **trial only**: the
|
||||
layer dies with the container. Merge + redeploy to make it durable.
|
||||
- Nothing on the voice pair exposes `/metrics`; Prometheus can't scrape it (a
|
||||
blackbox HTTP probe on 7772 is the only honest option).
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
name: frank-tts-consumer-voice-triage
|
||||
description: "Triage a consumer (OpenWebUI, omp, open-notebook) rejecting or ignoring a frank-tts voice name: prove whether the server or the browser-side Kokoro.js engine produced the error."
|
||||
---
|
||||
|
||||
# frank-tts consumer voice triage
|
||||
|
||||
A consumer says a voice name is invalid, speaks in the wrong voice, or is
|
||||
silent. `frank-tts` (`192.168.0.143:7772`, shim
|
||||
`dankclaw/voiceservice/app.py`) resolves `voice=<name>` against
|
||||
`/home/dank/voices/<name>/reference.wav` and answers **400 + the list of
|
||||
voices that exist** for a name it cannot resolve — never a silent fallback.
|
||||
Any error naming a *different* voice list did not come from it.
|
||||
|
||||
**Signature that decides the case immediately:** an enum of ~29 names
|
||||
`af_heart, af_alloy, … am_*, bf_*, bm_* … bm_fable` is **Kokoro's fixed pack**,
|
||||
the voice list of **Kokoro.js running in the browser** (OpenWebUI ships it
|
||||
in its own front end as TTS engine value `browser-kokoro`, with a personal
|
||||
"Kokoro.js dtype" setting; model `onnx-community/Kokoro-82M-v1.0-ONNX`).
|
||||
`narrator1/2/3` are frank-tts cloned voices. Kokoro cannot clone.
|
||||
|
||||
## Instrument ladder (stop at the first that answers)
|
||||
|
||||
1. **What the endpoint actually accepts** (never infer from a consumer's text):
|
||||
`curl -s http://127.0.0.1:7772/v1/audio/voices` → JSON array of pack names.
|
||||
2. **Speak with the consumer's exact body, from inside the consumer's
|
||||
container** — same network, same shape. Write the JSON to a file and
|
||||
`--data-binary @file` (nested `sh -c` quoting silently sends an empty
|
||||
body and yields a bogus `400 malformed request body`):
|
||||
```
|
||||
printf '%s' '{"model":"frank-tts","input":"probe","voice":"narrator1","response_format":"mp3","speed":1.0}' > /tmp/p.json
|
||||
podman cp /tmp/p.json <consumer>:/tmp/p.json
|
||||
podman exec <consumer> curl -sS -o /tmp/o.bin -w 'http=%{http_code} type=%{content_type} bytes=%{size_download}\n' \
|
||||
-X POST http://host.containers.internal:7772/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' --data-binary @/tmp/p.json
|
||||
```
|
||||
`200 audio/mpeg|audio/wav` ⇒ the server path is healthy; the failure is elsewhere.
|
||||
3. **Did the request arrive at all?** `podman logs --tail 400 <consumer> | grep -c audio/speech`.
|
||||
A server-path failure leaves a log line **and** a frank-tts error body; a
|
||||
client-side engine leaves **neither**. This contrast is the discriminator.
|
||||
4. **Where the rejection string lives** (proves the emitter without guessing):
|
||||
```
|
||||
podman exec <consumer> sh -c 'grep -ho ".\{200\}Should be one of.\{80\}" /app/build/_app/immutable/chunks/*.js | head'
|
||||
podman exec <consumer> sh -c 'grep -ho ".\{60\}af_heart.\{200\}" /app/build/_app/immutable/chunks/*.js | head -2'
|
||||
```
|
||||
Matched inside the front-end bundle ⇒ browser-side validator, not the API.
|
||||
5. **Config authority sweep** (OpenWebUI): admin config is the `config`
|
||||
table (one key per row, JSON `value`; expected
|
||||
`audio.tts.engine=openai`, `audio.tts.model`, `audio.tts.voice`,
|
||||
`audio.tts.openai.api_base_url`); per-user overrides are the `settings`
|
||||
JSON column of table `user` (singular, no `is_active`; some rows hold the
|
||||
string `null` → guard it). `audio.stt.*` and `audio.tts.*` are separate.
|
||||
No `sqlite3` CLI in the image or on the host: `podman exec -i <c> python -`
|
||||
with the stdlib module. **`podman exec` needs `-i` to accept a heredoc
|
||||
script** — without it the script is silently dropped and you get no output.
|
||||
|
||||
## Known traps
|
||||
|
||||
- OpenWebUI is a **rootless** quadlet: `sudo podman …` reports *no such
|
||||
container*. Read it rootless.
|
||||
- `podman ps` in this podman build has no `.ImageName` field — use `{{.Image}}`.
|
||||
- The in-browser engine choice is **per-browser storage**: empty
|
||||
`user.settings` + correct admin config + zero backend hits is the normal
|
||||
shape of this bug. Fix is UI: Settings → Audio → TTS Engine → **Default**.
|
||||
- `GET /api/config` publishes no `audio` block, and nonexistent `/api/...`
|
||||
paths return the SPA HTML with `200` — never conclude "config missing" from
|
||||
an HTML body.
|
||||
- TTS is memory-registry sensitive: the engine's voice registry is
|
||||
memory-only, so "every name returns 400" after a restart points at the warm
|
||||
pass, not at the caller (see [[frank-tts-add-cloned-voice]]).
|
||||
|
||||
## Local playback side (omp `/say`, manual probes)
|
||||
|
||||
If synthesis succeeds (`200`, sane WAV bytes) but nothing is audible, the
|
||||
fault is the PipeWire graph, not the endpoint:
|
||||
`wpctl status` (default sink marked `*`, and *Default Configured Devices* —
|
||||
a remembered-but-offline `bluez_output.*` there is the classic silent
|
||||
route), `wpctl get-volume @DEFAULT_AUDIO_SINK@`, `pw-play -v <file>` (expect
|
||||
`stream drained` + exit 0), and target a device explicitly with
|
||||
`pw-play --target <node-id>` to identify which hardware is live. A one-word
|
||||
clip (~0.7 s) is easy to miss — probe with a multi-second sentence.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: frank-tts-repoint-voice-consumers
|
||||
description: "Rotate/rename a frank-tts voice across all consumers (repo DEFAULT_VOICE, Open Notebook speaker profiles over REST, docs) after packs change."
|
||||
---
|
||||
|
||||
## When
|
||||
frank-tts voice packs on `/home/dank/voices` (Dankclaw) changed — packs added, removed, or renamed — and old voice names (e.g. `narrator1/2/3`) no longer resolve. Consumers still sending a dead name now 400 with `unknown voice 'x'; available voices: ...`. This is a maintenance sweep, NOT a bug; confirm with the operator whether the change is intentional before "fixing".
|
||||
|
||||
## Facts that make this non-obvious
|
||||
- frank-tts (:7772) resolves a voice as a **directory**: `<name>/reference.wav` must exist. `model` field is ignored; the voice comes solely from `voice`/`voice_id`.
|
||||
- Live census: `curl -s http://127.0.0.1:7772/v1/audio/voices` (and `/health` returns the same list). This is the truth; the vault docs may be stale.
|
||||
- Two consumers carry voice names, in **different stores**:
|
||||
- **repo**: `scripts/frank-tts.env` `DEFAULT_VOICE=` — answers a voice-less request (the omp harness's normal shape). Deploy via git.
|
||||
- **app-side**: Open Notebook speaker profiles (SurrealDB) and OpenWebUI `audio.tts` voice — NOT repo files.
|
||||
|
||||
## Step 1 — repo: DEFAULT_VOICE
|
||||
Edit `scripts/frank-tts.env` `DEFAULT_VOICE=<new real voice>`. Trunk-based: this worktree can't `git checkout -b` (shared-workspace policy); commit on HEAD and `git push origin HEAD:refs/heads/<branch>`, open PR vs `main`. No bats test pins the DEFAULT_VOICE value (the tts bats use `narrator1` as a self-contained fixture that builds its own `reference.wav` — mechanism test, leave it).
|
||||
|
||||
## Step 2 — Open Notebook speaker profiles (REST, host-side, takes effect immediately)
|
||||
Voice ID is **free text passed to the TTS request untouched** (no enum/whitelist). The active fleet profile is `dank-duo` (episode profile `dank-daily` → speaker_config). PW is the host `OPEN_NOTEBOOK_PASSWORD`.
|
||||
|
||||
```bash
|
||||
ssh dankclaw bash -s <<'REMOTE'
|
||||
PW=$(grep -m1 '^OPEN_NOTEBOOK_PASSWORD=' /home/dank/Containers/open-notebook/env | cut -d= -f2- | tr -d '"')
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: Bearer %s"\n' "$PW" > "$cfg"
|
||||
base=http://127.0.0.1:5056/api/speaker-profiles
|
||||
# census: name -> speaker/voice_id
|
||||
curl -s -K "$cfg" "$base" | jq -c '.[]|{name,speakers:[.speakers[]|{name,voice_id}]}'
|
||||
# repoint by NAME (GET-by-name returns full record incl id; PUT whole transformed object to /{id})
|
||||
curl -s -K "$cfg" "$base/dank-duo" > /tmp/s.json
|
||||
id=$(jq -r .id /tmp/s.json)
|
||||
jq '(.speakers[].voice_id) |= ({"OLD1":"NEW1","OLD2":"NEW2"}[.] // .)' /tmp/s.json > /tmp/s2.json
|
||||
curl -s -K "$cfg" -X PUT -H 'Content-Type: application/json' -d @/tmp/s2.json "$base/$id" -w ' HTTP %{http_code}\n'
|
||||
curl -s -K "$cfg" "$base/dank-duo" | jq -c '[.speakers[]|{name,voice_id}]' # verify
|
||||
rm -f "$cfg" /tmp/s.json /tmp/s2.json
|
||||
REMOTE
|
||||
```
|
||||
- A duo has only 2 speakers — an extra voice is available-but-unassigned, not missing.
|
||||
- Upstream's seeded profiles (`business_panel`, `solo_expert`, `tech_experts`) use OpenAI names + `voice_model:null`; they aren't wired to frank-tts — leave them.
|
||||
|
||||
## Step 3 — OpenWebUI (check before touching)
|
||||
`curl -s http://127.0.0.1:<webui-port>/api/config | jq '.audio.tts'` (port from `podman ps`). If `engine`/`openai.*` are null, the active path is `browser-kokoro` (synthesizes in-browser) and the stored voice is **dormant, not broken** — flag, don't edit a live UI unasked.
|
||||
|
||||
## Step 4 — docs (point of use, one owner)
|
||||
Update the census + speaker-mapping lines in `serverAppConfigs-docs/Dankclaw/Frank TTS.md` and `Open Notebook.md` to the live packs. Leave DATED measurement logs (e.g. a `2026-10-19` `/health` probe) as history. Do NOT chase `scripts/voice-packs.env` / `Voice Packs.md` narrator builder defs — ask the operator whether those packs are being rebuilt or retired.
|
||||
|
||||
## Verify (behavior, not config text)
|
||||
Synthesize each new voice and expect 200 + distinct bytes:
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code} %{size_download}B\n' -X POST http://127.0.0.1:7772/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' -d '{"model":"voxcpm2","input":"test","voice":"<name>"}'
|
||||
```
|
||||
`tests/run.sh` EXIT=0 before commit. `DEFAULT_VOICE` only lands after merge + `./scripts/deploy.sh dankclaw --service frank-tts`; run that from the dev `main` checkout (`/home/blank/Projects/serverAppConfig`), NOT the agent worktree (deploy.sh refuses a non-`main` branch and pushes `main` forward — it must run on the dev box, not the target).
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
name: frank-tts-voice-locate
|
||||
description: "Answer fleet-voice questions (where voices are stored, add a voice) by jumping straight to the serverAppConfig vault pages and the live 7772 roster probe, instead of re-searching Gitea for a frank-tts repo that does not exist."
|
||||
---
|
||||
|
||||
# Frank-TTS voice facts: skip the search, go to the owners
|
||||
|
||||
Questions like "how do I add a voice?", "where are voices stored?", "why is narratorN missing?" are already answered elsewhere. The frank-tts code does NOT live in a Gitea repo named frank-tts (repo search finds nothing; it is `dank/serverAppConfigs`), and the engine repo is upstream qwentts.cpp pinned at `5916bf7c`.
|
||||
|
||||
## Resolution order (one grep, no repo search)
|
||||
|
||||
1. **Vault pages (canonical, all of it lives here)** in the local checkout `~/Projects/serverAppConfig`:
|
||||
- `serverAppConfigs-docs/Dankclaw/Voice Packs.md` — pack anatomy, the builder, the 10-second source invariant, "Adding a fourth voice" steps, engine-bump arithmetic.
|
||||
- `serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md` — registry law (memory-only; warm hook), routes, Vulkan device law, troubleshooting table.
|
||||
- `serverAppConfigs-docs/Dankclaw/Frank TTS.md` — the 7772 facade contract.
|
||||
- Code/seam: `dankclaw/voiceservice/`, `scripts/build-voice-pack.sh`, `scripts/voice-packs.env`, `tests/voice_pack_unit.bats`.
|
||||
2. **Live roster** (authoritative over any doc census): `curl -s http://192.168.0.143:7772/v1/audio/voices` (bare JSON array; `/health` also echoes `voices`). A missing name = its dir lacks `reference.wav` or the name is not lowercase — check `/home/dank/voices/<name>/` via `ssh dankclaw`.
|
||||
|
||||
## Facts that cost turns to re-derive
|
||||
|
||||
- Voice storage: `/home/dank/voices/<name>/` on Dankclaw (host state, never git; RO mount at `/voices`). Files: `reference.wav` (24 kHz mono, 18–25 s, existence gate), `reference.spk` (8192 B), `reference.rvq` (5456–6864 B), `manifest.json`.
|
||||
- Roster = facade directory scan; engine registry is in-memory, filled by `podman exec frank-tts-engine /app/warm-voices.sh` (boot `ExecStartPost`; re-register replaces, restart costs 6–8 s/voice).
|
||||
- Clients (omp-dankvoice, OpenWebUI) store NO voices — they GET `/v1/audio/voices`; only the chosen name persists client-side (`~/.omp/cache/dankvoice.json` `voice`).
|
||||
- Crop law: past first 10 s (hard invariant), 20–25 s preferred, floor 19.84 s for `.rvq`; sources pinned by sha256 in `scripts/voice-packs.env`; sample media live on workstations (`/home/blank/audiosample*`) and must be copied to `/home/dank/voice-sources` before a build.
|
||||
- Engine `GET /v1/audio/voices` returns `{}` at the pinned commit — always ask the 7772 facade, never 8880, for the list.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: frank-tts-voice-pack-audit
|
||||
description: Census live frank-tts voice packs against scripts/voice-packs.env to catch name-drift (renamed/retired packs) and untracked CLI-built packs; use after any voice add/rename when reconciling the builder recipe with what is actually on disk.
|
||||
---
|
||||
|
||||
# frank-tts voice-pack audit (builder ↔ live census)
|
||||
|
||||
Read-only census answering: *does `scripts/voice-packs.env` still describe the packs that actually exist, and is every live pack reproducible from the repo?* Run it after anyone adds or renames a voice. The runtime being healthy (all voices synthesize) does NOT mean the builder is honest — the gap is reproducibility, not serving.
|
||||
|
||||
## Why this exists
|
||||
Packs are host state under `/home/dank/voices` (never in git). A voice can be:
|
||||
- **built by the env** (a `VOICE_PACK_<NAME>_*` block exists), or
|
||||
- **built by CLI override** (`build-voice-pack.sh <name> <dir> --source --offset --duration --sha256` with NO env block) — then its only provenance is `/home/dank/voices/<name>/manifest.json`, and `--all` silently skips it.
|
||||
|
||||
Renaming a pack directory (e.g. `narrator1` → `wilwheaton`) preserves the crop bytes and its source sha256, so the env recipe stays *correct but keyed under a dead name*. `build-voice-pack.sh --all` then rebuilds the dead names and skips the live ones.
|
||||
|
||||
## Procedure (on Dankclaw)
|
||||
|
||||
1. **Live set + which builder made each** — the manifest `kind` field tells them apart:
|
||||
- `kind: "voice-pack"` → built by `scripts/build-voice-pack.sh` (env or CLI override).
|
||||
- `kind: "voice-reference-pack"` → the older trial `pipeline.sh`.
|
||||
```
|
||||
for d in /home/dank/voices/*/; do n=$(basename "$d"); [ -f "$d/reference.wav" ] \
|
||||
&& printf '%-16s %8sB %s\n' "$n" "$(stat -c%s "$d/reference.wav")" "$(stat -c %y "$d/reference.wav" | cut -d. -f1)"; done
|
||||
```
|
||||
2. **Per-voice provenance** — `manifest.json` is the sole record for CLI-built packs. Read `name/voice`, `kind`, `source.path`, `source.sha256`, `crop.offset_seconds`, `crop.duration_seconds`. Use `python3 -c 'import json,...'`, NOT a `jq '{a,b,c}'` object — the brace form gets brace-expanded by the remote zsh and errors.
|
||||
3. **Builder tracked list**:
|
||||
```
|
||||
grep -E '^VOICE_PACKS=|^VOICE_PACK_[A-Z0-9_]+_(SOURCE|OFFSET|DURATION|SHA256)=' /home/dank/serverAppConfigs/scripts/voice-packs.env
|
||||
```
|
||||
Diff the names in `VOICE_PACKS` against the live dirs. Three findings to report:
|
||||
- env keys that have **no live dir** (renamed/retired),
|
||||
- live dirs with **no env block** (CLI-built, unreproducible from repo),
|
||||
- matching source **sha256** between a dead env key and a live dir = that live pack is the renamed crop (safe to re-key the env, don't recompute).
|
||||
4. **Distinguish CLI-built from uncommitted env drift** — check the deployed env matches the repo:
|
||||
```
|
||||
cd /home/dank/serverAppConfigs && git diff --stat HEAD -- scripts/voice-packs.env # empty = no drift
|
||||
```
|
||||
Empty diff + a `kind: "voice-pack"` manifest with no env block ⇒ built via CLI overrides (correct, just untracked), NOT a rogue remote edit.
|
||||
|
||||
## Repoint the consumers (separate concern)
|
||||
If voices were renamed/retired, the runtime consumers still point at dead names — see the `frank-tts-repoint-voice-consumers` skill. Consumers that name a voice: `scripts/frank-tts.env` `DEFAULT_VOICE`, Open Notebook `dank-duo` speaker profile (`voice_id`, app-side REST), OpenWebUI `audio.tts.voice` (config DB). A voice-less request to frank-tts 400s naming `DEFAULT_VOICE`, so a dead default is a live break, not just cosmetic.
|
||||
|
||||
## Gotchas
|
||||
- **A published port is reachable at `host.containers.internal` too.** OpenWebUI's docs say use the LAN IP because that name "doesn't resolve on the bridge" — true only for *non-published* ports. frank-tts 7772 IS published, so `host.containers.internal:7772` answers 200 from the open-webui container. Verify before calling a URL the cause; the real break is usually the dead voice name.
|
||||
- **Source custody is split**: `audiosample*.mp3/m4b` live on the workstation (`/home/blank`), newer sources on Dankclaw (`/home/dank/voice-sources`). `--all` is only reproducible once each env SOURCE path points at where the file actually is.
|
||||
- Do NOT edit `voice-packs.env` to add CLI-built voices without confirming each source path + offset + sha against its manifest, then run `./tests/run.sh` (the pack-list invariant in `voice_pack_unit.bats` walks `VOICE_PACKS`).
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
name: franksign-demo-inspection-stack
|
||||
description: "Boot franksign's prod-shaped stack, seed all assignment lifecycle states through the real API, and open a headed inspection tour (platform + storybook) without squatting the e2e port."
|
||||
---
|
||||
|
||||
Visual-inspection recipe for the franksign repo (Dank boxes). Proven 2026-10-20; use when asked to "show/open the platform" or tour a new slice.
|
||||
|
||||
## Boot (do NOT squat :3100 — the e2e seam curls whoever owns that port)
|
||||
```bash
|
||||
node --env-file=.env apps/api/dist/cli.js # binds :3000 by default; use named service
|
||||
cd apps/web/storybook-static && python3 -m http.server 6007 # gate seam 12 already built it
|
||||
curl -s http://127.0.0.1:3000/health # {"status":"ok","db":"up"}
|
||||
```
|
||||
|
||||
## Seed lifecycle states through the real API (cookie jar, NOT bearer)
|
||||
better-auth sign-up returns a bearer `token`, but the org routes resolve sessions
|
||||
from **cookies only** (`Authorization: Bearer …` → get-session null → 401). Use
|
||||
`-c /tmp/jar` at sign-up and `-b /tmp/jar` after; state-changing calls need an
|
||||
`origin:` header. Keep auth headers out of argv via `curl -K <file>`.
|
||||
|
||||
1. `POST /api/auth/sign-up/email` `{"email","password"(>=8 chars),"name"}` → save cookies (`-c`)
|
||||
2. `POST /api/orgs` `{"name"}` → `org.id`
|
||||
3. `POST /api/orgs/:org/templates` `{"name","baseTemplateId":"sample"}` → `template.id`
|
||||
4. `POST /api/orgs/:org/templates/:tpl/assignments` `{"email"}` → `{token, signUrl}` (repeat per demo signer)
|
||||
5. signed state: `POST /api/releases/:token` `{"values":{"name":…,"date":…,"agreement":true},"signatureName":…}` — no auth needed (token seam)
|
||||
6. voided state: `POST /api/orgs/:org/assignments/:assignmentId/void` (cookie session)
|
||||
Remaining pending assignment → three inspectable pages: `/app/sign/:token` renders form / already-signed / cancelled.
|
||||
|
||||
## Byte-identity spot check (reconstruction contract, issue #18)
|
||||
Fetch `GET /api/releases/:token/pdf` twice with `sleep 1.5` **between** fetches
|
||||
and sha256-compare, plus the org door `GET /api/orgs/:org/assignments/:aid/pdf`
|
||||
(cookie jar). Same-second-only comparisons are blind to clock leaks
|
||||
(pdf-lib's `PDFDocument.load` restamps `/ModDate` unless `updateMetadata:false`).
|
||||
To see printed text: inflate the content streams and unescape `\(`, `\)` —
|
||||
pdf-lib writes hex-string `Tj` operands.
|
||||
|
||||
## Inspection browser (managed Chromium via browser eval)
|
||||
- `viewport` must be `{width, height}` — string fails
|
||||
- `tab.waitFor(ms)` does NOT exist (ms is parsed as selector) → `await Bun.sleep(900)`
|
||||
- open with `headed: true, persist: true`; window lands on the desktop — confirm with `niri msg windows | grep -i chrome`
|
||||
|
||||
## Teardown (hard gate hazard)
|
||||
A live demo API on :3000 **hangs the full `tests/run.sh` gate for ~10 minutes**
|
||||
(observed 2026-10-21: >600s kill with the service up; 14/14 in ~15s without
|
||||
it). Always `proc://franksign-demo-api/kill` BEFORE running the gate; verify
|
||||
`pgrep -f apps/api/dist/cli.js` is empty so the next `tests/run.sh` e2e seam
|
||||
owns :3100 honestly.
|
||||
|
||||
## Adjacent Gitea quirk (same fleet)
|
||||
Closing an issue/PR: `PATCH {"state":"closed"}` → 201; `POST` → 405. Merging needs form-encoded lowercase `do=merge` (see gitea-merge-form-encoded). Label assignment: `PATCH /issues/N` with `labels:[id]` silently clears labels — use `POST /issues/N/labels` `{"labels":[id]}`.
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: franksign-worktree-db-cred-trap
|
||||
description: "Diagnose franksign e2e seam 5 / health 503 db:down in a fresh worktree: setup-dev.sh minted a password that mismatches the shared franksign-db container."
|
||||
---
|
||||
|
||||
# franksign fresh-worktree DB credential trap
|
||||
|
||||
## Symptom
|
||||
`tests/run.sh` in a fresh franksign worktree fails seam 5 ("e2e server did not
|
||||
become healthy") while the server log says `franksign-api listening on :3100`.
|
||||
Manual probe shows the server IS up:
|
||||
|
||||
```bash
|
||||
PORT=3100 node --env-file=.env apps/api/dist/cli.js >/tmp/manual-e2e.log 2>&1 &
|
||||
sleep 2; curl -s http://127.0.0.1:3100/health
|
||||
# -> 503 {"status":"degraded","db":"down",...}
|
||||
```
|
||||
|
||||
Not a regression, not a :3100 port collision (check `ss -tlnp | grep 3100` to rule that out).
|
||||
|
||||
## Root cause
|
||||
`infra/setup-dev.sh` mints a fresh `POSTGRES_PASSWORD` per checkout
|
||||
(`if [ ! -f infra/postgres.env ]`), but `franksign-db` + `franksign-pgdata`
|
||||
are shared box state owned by the main checkout
|
||||
(`~/Projects/franksign`). The new worktree's `.env` DATABASE_URL then fails
|
||||
TCP auth against the initialized volume.
|
||||
|
||||
The setup script masks it: its readiness gate is
|
||||
`podman exec franksign-db pg_isready`, which uses the in-container local
|
||||
socket and passes regardless of password — so setup prints "postgres ready"
|
||||
while the checkout is dead on arrival.
|
||||
|
||||
## Fix (git-ignored files, no commit needed)
|
||||
Copy the shared credentials from the owning checkout:
|
||||
|
||||
```bash
|
||||
grep DATABASE_URL /home/blank/Projects/franksign/.env # source of truth
|
||||
# PUT that exact DATABASE_URL line into this worktree's .env
|
||||
# PUT the matching POSTGRES_PASSWORD into this worktree's infra/postgres.env
|
||||
```
|
||||
|
||||
Gotcha: the shared password may contain literal `$$`. Keep it byte-for-byte —
|
||||
systemd `EnvironmentFile` and `node --env-file` both read values literally,
|
||||
so `$$` in the file IS `$$` in the password. No escaping, no de-escaping.
|
||||
|
||||
## Verify
|
||||
```bash
|
||||
PORT=3100 node --env-file=.env apps/api/dist/cli.js & # then
|
||||
curl -fsS http://127.0.0.1:3100/health # -> {"status":"ok","db":"up"}
|
||||
tests/run.sh # 14/14
|
||||
```
|
||||
|
||||
## Permanent fix (proposed, not yet landed)
|
||||
`setup-dev.sh` should reuse `~/.config/containers/systemd/postgres.env`
|
||||
(symlink to the owning checkout) when present instead of minting new creds.
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
name: gguf-header-vram-fit-analysis
|
||||
description: "Decide if a GGUF model fits an LLM endpoint's GPU at a target context window: HTTP range-parse the header for KV geometry and custom quant types, derive the KiB/token rate law, price non-KV from measured boots (own repo + the publisher's benchmark reports), and gate on runtime lock-in (fork, build flags, drafter) before any download."
|
||||
---
|
||||
|
||||
Answer "can model M fit window W on endpoint E (plus vision?)" **before** downloading anything. The GGUF header carries the truth; measured boots are the price list.
|
||||
|
||||
## 1. Range-parse the GGUF header from the Hub (no download)
|
||||
|
||||
`GET` the `resolve/main/<file>.gguf` URL with `Range: bytes=0-N` and a `User-Agent: curl/8` header, parse GGUF in Python (`struct`):
|
||||
|
||||
- Layout: magic `GGUF` (read `buf[0:4]` directly — do NOT `read_string()` it), u32 version, u64 ntensors, u64 nkv, then KV pairs (v3: u64 string lengths; v2: u32), then tensor-info table (name, ndim, dims u64[], type u32, offset u64). Handle EVERY array element type or the parse desyncs mid-metadata.
|
||||
- **Metadata is >8 MiB** when vocab is ~250k (tokenizer arrays live in the KV section). Fetch 32–64 MiB.
|
||||
- **Per-tensor byte sizes without a downloader**: the data section is in table order — `size(i) = offset(i+1) - offset(i)`, last = `file_size - header_end - offset(last)`. Then `bits/elem = bytes*8/prod(dims)`. That names unknown type ids by their rate (2.125 bpw ⇒ ternary-in-2-bit-slots + fp16 scale/128; 16.0 ⇒ BF16; 32 ⇒ F32) and totals weights per type.
|
||||
|
||||
Extract: `general.architecture`, `block_count`, `full_attention_interval`, `attention.head_count_kv`, `key_length`/`value_length`, `context_length`, `embedding_length`, `ssm.*`, and **`<arch>.nextn_predict_layers`** — its absence (plus `block_count` = real layers, not +1) means **no in-file MTP head** (MTP is a property of the file, not the flags). Type histogram decides stock vs fork: a type id above stock `GGML_TYPE_COUNT` is fork lock-in — read the fork's `ggml/include/ggml.h` enum to name it.
|
||||
|
||||
## 2. KV rate law (the wall)
|
||||
|
||||
For hybrid (qwen35-style) models only the FULL-attention layers bill per token; linear/DeltaNet layers bill a fixed state per SLOT (≈50–60 MiB/slot on 27B-class hybrids — noise vs KV, but it is what lets N slots fit).
|
||||
|
||||
```
|
||||
elems/token/cache = (block_count / full_attention_interval) × head_count_kv × key_length
|
||||
B/token = elems × bytesK + elems × bytesV (K and V both)
|
||||
bytes/elem: f16=2.0, q8_0=1.0625, q4_0=0.5625, q4_1=0.578125
|
||||
```
|
||||
|
||||
Fleet-verified instances: qwen35-9B (8 full layers) 32 KiB/token f16; qwen35-27B (16 full) 64 KiB/token f16, 34 KiB q8_0, 18 KiB q4_0. A low-bit WEIGHT quant never moves this line — state it explicitly, it is the question people actually ask. `--flash-attn on` is required for quantized V; quantized KV needs a `GGML_CUDA_FA_ALL_QUANTS=ON` build.
|
||||
|
||||
## 3. Price non-KV from MEASURED boots, never a formula
|
||||
|
||||
Two anchors beat one:
|
||||
- **Own repo**: closest runbook boot (same arch, same card, ideally same mmproj): `non_kv = boot_used − rate × boot_pool`, then swap weights/mmproj/drafter/ubatch-rung deltas.
|
||||
- **Publisher**: their published benchmark/server reports (e.g. a `community-benchmarks/<card>.md` or `KV-CACHE.md`) carry peak `nvidia-smi` and a steady-state figure for a named argv — back out their overhead = steady − weights − KV(× their pool) − mmproj. Divergence between the two anchors IS your uncertainty band; report both ends.
|
||||
Card/vendor peak-VRAM at bench context also calibrates compute buffers at small ubatch; the fleet's ubatch-rung costs (≈0.3 GiB @1024, ≈0.75 GiB @2048 on a 27B) cover the rest.
|
||||
|
||||
Card ceiling from settled boots (`used + free`, e.g. 24,125 MiB on the 3090), not the spec sheet. An mmproj bills ~1.5× its file size. Keep ≥1 GiB headroom when ranking arms; tightest proven fleet fit is 489 MiB.
|
||||
|
||||
## 4. Gates beyond VRAM
|
||||
|
||||
- **Target window > `context_length`** ⇒ YaRN required (`--rope-scale = target/native`); quality at >1.2–1.5× on aggressive quants is unmeasured — say so.
|
||||
- **Pool divides across slots**: `n_ctx_slot = CTX_SIZE / PARALLEL`; the per-slot number is the harness contract, and the endpoint's role (e.g. compaction hub needing ≥170k) picks the geometry.
|
||||
- **Fork/custom-quant lock-in checklist** (answer all, cite ids):
|
||||
1. which fork/branch, and does the pinned release tag contain the needed fixes — `GET /compare/<tag>...prism` and read `ahead_by` + commit subjects (a tag 5 commits behind contains everything merged earlier);
|
||||
2. host CPU traps (e.g. AVX-512 load segfaults crash even with all layers GPU-offloaded);
|
||||
3. **does the release workflow pass the build flags the contract needs** — grep the release workflow's cmake args: prebuilts often omit `GGML_CUDA_FA_ALL_QUANTS`, and without it mismatched/unsupported K/V types silently run flash attention on the CPU;
|
||||
4. drafter availability (in-file `nextn`, a paired drafter, or nothing) — no drafter means the decode regression is unfixable;
|
||||
5. pack choice per architecture (the vendor's throughput table is generation-specific; a community report on the same GPU family outranks it);
|
||||
6. behavior/role risk: base model censorship posture, `tool_calls` defects, `reasoning_effort` template gate, prompt-cache `--cache-ram` vs host RAM actually available.
|
||||
- Vision: needs `--mmproj`; probe with ≥256 px solid color; only the OpenAI `image_url` content part reaches the model.
|
||||
|
||||
## 5. Output
|
||||
|
||||
Table of arms (pack × KV dtype × slot geometry) → KV MiB, total, free vs ceiling, FITS/FITS-tight/NO FIT, with the uncertainty band; then the gate list. State every `[INFERENCE]`/estimate. The boot ledger (verbosity 4), not the arithmetic, is the referee; benching requires stop+mask per fleet policy.
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
name: gguf-lora-adapter-fit-census
|
||||
description: "Read-only census before applying a GGUF LoRA/adapter to a llama.cpp (or fork) endpoint: Hub metadata, declared base provenance, runtime flag support, and the behavioural probe that proves the effect."
|
||||
---
|
||||
|
||||
# GGUF LoRA / adapter fit census (read-only, before you promise a file to a payload)
|
||||
|
||||
A 3 MB adapter is a payload change: it edits the compute graph at load. Do the census
|
||||
before editing an endpoint's trio, and prove the EFFECT behaviourally — no metadata,
|
||||
card table, or health gate can see it.
|
||||
|
||||
## 1. Census the Hub repo first (no auth for public repos)
|
||||
|
||||
```bash
|
||||
curl -sS "https://huggingface.co/api/models/<org>/<repo>?blobs=true" | python3 -c "
|
||||
import json,sys; d=json.load(sys.stdin)
|
||||
print(d.get('pipeline_tag'), d.get('library_name'))
|
||||
print([t for t in d.get('tags',[])][:20]) # base_model / base_model:adapter rows matter
|
||||
for s in d['siblings']: print(s['rfilename'], s.get('size'), (s.get('lfs') or {}).get('oid'))"
|
||||
curl -sS "https://huggingface.co/<org>/<repo>/raw/main/README.md" > /tmp/card.md # grep for --lora flags
|
||||
```
|
||||
|
||||
- `base_model` / `base_model_relation: adapter` name the pack the adapter was built for.
|
||||
- **A file with `lfs.oid = None` is a plain git object**: pin the sha256 of the Hub's bytes,
|
||||
not an LFS oid. Fetcher law still applies: `curl -fsSL --fail-with-body`, byte count,
|
||||
sha256, `--verify-only`, no resume.
|
||||
|
||||
## 2. Read the adapter's own GGUF KV section (settles provenance in one read)
|
||||
|
||||
Small python read of the KV pairs — no torch, no download beyond the adapter itself:
|
||||
magic `GGUF`, `u32 ver`, `u64 n_tensors`, `u64 n_kv`; key = `u64 len` + bytes; type `u32`
|
||||
(0 u8,1 i8,2 u16,3 i16,4 u32,5 i32,6 f32,7 bool,8 str,9 array,10 u64,11 i64,12 f64);
|
||||
array = `u32 elem_type` + `u64 count`.
|
||||
|
||||
Keys worth printing: `general.type` (`adapter`), `adapter.type` (`lora`),
|
||||
`adapter.lora_alpha`, and whatever provenance the publisher wrote — e.g. PrismML/abliteration
|
||||
packs carry `prism.abliterate.base_gguf`, `base_sha256`, `direction_row`, `target_layers`,
|
||||
`target_embedding`, `sites`, `basis`.
|
||||
|
||||
**Provenance is metadata, not a gate.** The loader matches **tensor names, not hashes**, so an
|
||||
adapter fit against a sibling quantization of the same model still applies — which is a
|
||||
decision to surface to the operator, not a failure to hide. Compare `base_sha256` against the
|
||||
sha the endpoint's own fetcher pins; if they differ, say so in the PR body, the env header and
|
||||
the endpoint page, and prove the effect behaviourally instead.
|
||||
|
||||
Two publisher facts that decide whether a fork can carry the file at all: the build it was
|
||||
verified on, and what it deliberately does NOT touch (an embedding rank is the usual crash —
|
||||
`token_embd` left alone is a graph constraint, not an omission).
|
||||
|
||||
## 3. Ask the runtime binary, not your memory
|
||||
|
||||
```bash
|
||||
<llama-server> --help | grep -iE "lora|adapter"
|
||||
```
|
||||
Expect `--lora FNAME`, `--lora-scaled FNAME:SCALE,...`, `--lora-init-without-apply`
|
||||
(+ `POST /lora-adapters` for runtime scale). Prefer `--lora-scaled <file>:<scale>` as ONE
|
||||
argv token so a scale can never reach the engine without its file, and follow the endpoint's
|
||||
existing opt-out convention (the `MMPROJ_FILE=` shape: `:`-less default so an explicitly empty
|
||||
value means "off, on purpose"). Gate the deployer on the file's existence with the adapter's
|
||||
OWN fetcher named in the error text.
|
||||
|
||||
## 4. Prove the effect: refusal/behaviour probe, before AND after
|
||||
|
||||
The probe is not a bench, so the no-bench law does not block it — and the **before** half must
|
||||
be captured while the old shape is still live.
|
||||
|
||||
- Same prompts, `temperature` low, both before and after; classify with a refusal-marker regex
|
||||
and print only the opening characters, never the generated body.
|
||||
- **Budget ≥ 2 048 tokens on a thinking model.** A 600-token request can be swallowed whole by
|
||||
the reasoning channel: `content` comes back empty with `finish_reason: length` and
|
||||
`reasoning_content` full — which reads exactly like a refusal that never happened.
|
||||
- A leading `⚠️ Disclaimer`/hedge after the adapter is style, not the circuit: judge the marker
|
||||
sentence, not the tone.
|
||||
|
||||
## 5. Post-deploy reads (the ledger is the instrument)
|
||||
|
||||
- Boot log shows `llama_adapter_lora_init_impl: loading lora adapter from '<file>'` and then
|
||||
**dumps the adapter's KV block** — the provenance mismatch is printed at every start, so the
|
||||
runbook should point at the journal instead of restating it.
|
||||
- Confirm the flag on argv: `tr '\0' ' ' < /proc/$(systemctl show <svc> -p MainPID --value)/cmdline`;
|
||||
confirm `systemctl show <svc> -p DropInPaths --value` is empty (a leftover bench drop-in outranks
|
||||
everything you just deployed).
|
||||
- Shape must be untouched by an adapter: `n_slots` / `n_ctx_slot` / KV lines unchanged, and
|
||||
`/metrics` still answering (a llama.cpp endpoint is invisible to Prometheus without `--metrics`).
|
||||
- Root units log to the journal; rootless podman storage is read WITHOUT sudo (`sudo podman
|
||||
inspect` sees a different storage namespace and reports "no such object").
|
||||
|
||||
## Yields
|
||||
|
||||
One sentence per question: does the file exist and at what size/sha; which pack it declares as
|
||||
its base and whether that matches the served payload; which flag the runtime actually offers;
|
||||
what the before/after probe classified. Anything else is a decision for the operator, and the
|
||||
rollback should be one env line on a file the endpoint already loads.
|
||||
@@ -0,0 +1,68 @@
|
||||
---
|
||||
name: git-repo-secret-audit
|
||||
description: "Audit a git repo for leaked secrets across history, unreachable objects, LFS payloads, git config and host configs — with triage of every hit."
|
||||
---
|
||||
|
||||
# Git repo secret audit (history + unreachable objects, not just the worktree)
|
||||
|
||||
Worktree greps prove nothing: the leak you care about is in a commit that was amended
|
||||
away, in a branch nobody checks out, in an LFS payload, or in `.git/config`.
|
||||
|
||||
## 0. Build the known-value set from the live secrets, never from literals
|
||||
Read the values from the secret store at runtime so no key enters the transcript:
|
||||
```python
|
||||
known = {}
|
||||
for p in (home/".config/secrets").iterdir():
|
||||
t = p.read_text().strip()
|
||||
vals = {t}
|
||||
m = re.search(r":([^:@/]+)@", t) # credential-store URL form
|
||||
if m: vals.add(m.group(1))
|
||||
known[p.name] = {v for v in vals if len(v) >= 12}
|
||||
```
|
||||
Report ONLY `value[:4] + len + sha256[:10]`. Never print a value, and never quote a
|
||||
secret-shaped literal in a doc/commit message — the pre-commit gate will (correctly) block it.
|
||||
|
||||
## 1. Shapes to match (superset of the Dank gate)
|
||||
`sk-[A-Za-z0-9]{18,}`, `sk-or-v1-`, `sk-ant-`, `hf_[A-Za-z0-9]{30,}`, `gh[pousr]_[A-Za-z0-9]{30,}`,
|
||||
`github_pat_`, `glpat-`, `xox[baprs]-`, `(AKIA|ASIA)[0-9A-Z]{16}`, `npm_`, `pypi-`, `AIza[0-9A-Za-z_-]{30,}`,
|
||||
`(sk|pk)_(live|test)_`, Telegram `\d{8,10}:AA`, JWT `eyJ…\.eyJ…`, `-----BEGIN [A-Z ]*PRIVATE KEY-----`,
|
||||
`AGE-SECRET-KEY-1`, `ssh-(ed25519|rsa)\s+AAAA`, `scheme://user:pass@host` basic-auth URLs,
|
||||
`Bearer <24+>`, and the noisy `(password|secret|api_key|token)\s*[:=]\s*<12+>` assignment shape.
|
||||
|
||||
## 2. Scope, in this order
|
||||
```bash
|
||||
git rev-list --all | wc -l # revs incl. every branch + tag
|
||||
git ls-tree -r --format='%(objectname) %(path)' <rev> # sha -> path, per rev
|
||||
git cat-file --batch # stream reachable blob content
|
||||
git cat-file --batch-all-objects --batch-check # EVERY object, incl. dangling
|
||||
git rev-list --objects --all # the reachable set; difference = dangling
|
||||
git log --all --oneline --find-object=<sha> # empty => never committed on any ref
|
||||
git status --porcelain -uall --ignored=matching # !! lines: on-disk ignored files
|
||||
git config --list --show-origin # local/global/system credentials
|
||||
git ls-files | grep -E '(auth\.json|trust\.json|\.env|\.pem|\.key|identity)' # quarantine
|
||||
```
|
||||
LFS: scan `.git/lfs/objects/**` as BYTES (`decode("latin-1")`) — a screenshot is a classic
|
||||
secret carrier and the tracked tree holds only pointers.
|
||||
|
||||
## 3. Entropy layer (catches issuer-specific keys you did not list)
|
||||
Tokens `[A-Za-z0-9+/_-]{32,}` with Shannon entropy >= 4.3 containing both digits and letters.
|
||||
Flag every one whose value is in `known` as a LIVE leak; list the rest for triage.
|
||||
|
||||
## 4. Triage — the shapes that always fire and are fine
|
||||
- `api_key="sk-no-key-required"` and other doc placeholders
|
||||
- prose: `Set password: user.set_password(...)`, `authenticate(user, password)`
|
||||
- 40-hex hits that are git object ids / packfile names (`pack-<40hex>.pack`)
|
||||
- memory/KB records that *describe* a secret operation
|
||||
A hit is a leak only when the value matches a live secret, decodes to a real credential, or
|
||||
sits in a field a program reads.
|
||||
|
||||
## 5. Adjacent host scan (the audit's real yield)
|
||||
Check file modes of the secret dir (`0600`, dir `0700`) — world-readable secret files are the
|
||||
most common real finding. Then grep the live values across shell rc files, `~/.gitconfig`,
|
||||
`~/.npmrc`, `~/.netrc`, `~/.curlrc`, `~/.docker/config.json`, `~/.cache/huggingface/token`,
|
||||
`~/.aws/credentials`, `~/.ssh/config` and report filenames only.
|
||||
|
||||
## 6. Report
|
||||
Per layer: scope numbers + verdict. Every hit: path:line, redacted value, verdict
|
||||
(leak / benign + why). Name the gate's coverage gap if the project's own hook misses shapes
|
||||
you matched. Fix modes/chmod on the spot; changes to a security hook are the operator's call.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
name: git-secret-gate-hook
|
||||
description: "Write a fail-closed pre-commit secret-gate hook (staged-content scan) with the grep/ERE traps that kill naive versions, plus its bats fixture test."
|
||||
---
|
||||
|
||||
# Git secret-gate pre-commit hook
|
||||
|
||||
Use when a repo promises a "pre-commit secret gate" but none exists, or when porting the gate to another dank repo. Ships with three traps already paid for.
|
||||
|
||||
## Shape
|
||||
|
||||
- Track the hook at `hooks/pre-commit` (chmod +x). Git never self-installs hooks — wire per clone:
|
||||
`git config core.hooksPath hooks` — run it in the MAIN checkout once; `core.hooksPath` lives in the
|
||||
**common** git dir config, so every linked worktree inherits it.
|
||||
- Do NOT put `hooks` in `package.json` `files` — an installed plugin under node_modules never runs git.
|
||||
- Scan **staged** content (`git show ":$f"`), not the working tree: that is what lands in history.
|
||||
- Denylist by path (README, docs/, tests/, `.env.example`) — files whose job is naming secrets or holding
|
||||
fixture dummies produce theatre, not safety.
|
||||
- Never echo the matched secret; print file + line number only.
|
||||
- Red-first test: fixture repo + `cp` the real hook in + `git config core.hooksPath hooks`, then drive
|
||||
REAL `git commit` attempts: credential-shaped file must exit non-zero AND `git log` must show no new
|
||||
commit (block, not warn); clean commit passes; hook must survive committing itself.
|
||||
|
||||
## The three traps (each cost a red test)
|
||||
|
||||
1. **`--` is a ONE-SHOT end-of-options marker.** `grep -nE -- 'p1' -- 'p2'` treats everything after the
|
||||
first `--` as FILE NAMES: `grep: --: No such file or directory`, rc=2. Multi-pattern = repeat `-e`:
|
||||
`grep -nE -e 'p1' -e 'p2'`.
|
||||
2. **`if hits=$(grep ...)` is fail-OPEN on rc>=2** (rc=2 is truthy, hits empty). Capture `rc=$?`
|
||||
explicitly: `rc>=2` must BLOCK with "pattern error, refusing to pass unchecked" — a gate whose regex
|
||||
silently dies is worse than none.
|
||||
3. **POSIX ERE (`grep -E`) gotchas:** no `(?:...)` groups (use plain `(...)`); a `[:-]` inside a bracket
|
||||
starts a collating element `[:` and dies with "repetition-operator operand invalid" — put `-` FIRST
|
||||
in every class (`[-A-Za-z0-9/+=_.:+]`). Also `-----BEGIN` literals: prefer printf-safe fixtures in tests.
|
||||
|
||||
## Pattern set (shapes, not entropy — say so in the header comment)
|
||||
|
||||
```
|
||||
-----BEGIN[ A-Z0-9]*(OPENSSH|RSA|DSA|EC|PGP)? ?PRIVATE KEY-----
|
||||
AKIA[0-9A-Z]{16}
|
||||
gh[pousr]_[A-Za-z0-9]{36,}|github_pat_[A-Za-z0-9_]{50,}
|
||||
xox[baprs]-[A-Za-z0-9-]{10,}
|
||||
https://hooks\.slack\.com/services/[A-Za-z0-9/]+
|
||||
(aws_secret_access_key|secret(_access)?_key|(api|access|auth)[_-]?key|[a-z_]*token|passwd|password|credential)[[:space:]"']?[[:space:]]*[:=][[:space:]]*["']?[-A-Za-z0-9/+=_.:+]{16,}
|
||||
(Bearer|token|authorization)[[:space:]]*[:=][[:space:]]*["']?[-A-Za-z0-9._+/]{20,}
|
||||
```
|
||||
|
||||
## Test fixture hygiene
|
||||
|
||||
Fake secrets must be inert (RFC 2606 domains, all-zero ids, `AKIA0000000000000EXAMPLE`) AND must not
|
||||
match the hook's own self-scan. Single quotes in ERE patterns: write `'"'"'`.
|
||||
|
||||
## Wiring gotcha (dank worktree policy)
|
||||
|
||||
Set `core.hooksPath` AFTER the merge that lands `hooks/` into main — with the config set but the path
|
||||
missing, git fails every commit in worktrees lacking `hooks/` ("cannot run hooks/pre-commit: No such
|
||||
file or directory"). Order: merge -> land -> `git config core.hooksPath hooks` from main checkout.
|
||||
|
||||
Reference implementation: `dank/dankvoice` `hooks/pre-commit` (PR #19), pinned by `tests/gate.bats`.
|
||||
@@ -0,0 +1,85 @@
|
||||
---
|
||||
name: gitea-api-pr-operations
|
||||
description: "\"Open, list, merge, and diagnose issues/PRs and manage labels on the Dankclaw Gitea API (192.168.0.143:3000) with the token stored in ~/.config/secrets/gitea — incl. the token-only write auth (Basic reads but 422s on POST/PATCH), the required merge payload, where review comments actually live on 1.27.3, and how to recover the session id for the mandatory body footer.\""
|
||||
---
|
||||
|
||||
# Gitea API operations (Dankclaw `dank/command-center`, `dank/serverAppConfigs`)
|
||||
|
||||
## Where
|
||||
- Token: `~/.config/secrets/gitea` (`http://user:TOKEN@192.168.0.143` single line — git-credential URL form, NOT `username=`/`password=` lines).
|
||||
Extract WITHOUT echoing:
|
||||
`tok=$(sed -nE 's#^http://[^:/@]+:([^@]+)@192\.168\.0\.143.*#\1#p' ~/.config/secrets/gitea | head -1)`
|
||||
- Build a 600-perm temp curl config and use `curl -K`:
|
||||
`cfg=$(mktemp) && chmod 600 "$cfg" && printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"`
|
||||
Delete `$cfg` after use. NEVER pass `-H "Authorization: token $tok"` (the literal
|
||||
header lands in transcript + logs). NEVER echo the token.
|
||||
- Base URL: `http://192.168.0.143:3000/api/v1/repos/dank/<repo>`.
|
||||
|
||||
## Auth: reads tolerate Basic, writes DO NOT (measured 2026-10-20)
|
||||
GETs succeed with an HTTP Basic header built from the same credential, but every
|
||||
POST/PATCH/DELETE with Basic returns `407/422 {"message":"token is required"}`.
|
||||
Always use the `Authorization: token <tok>` header for everything — never debug
|
||||
"permissions" when the header shape is the fault.
|
||||
|
||||
## Recipes (all verified live)
|
||||
- **Open PR** (token only needed for private repos; this one reads publicly but
|
||||
POST needs auth): `POST /pulls` JSON `{title, head, base:"main", body}`.
|
||||
- **PATCH PR title/body**: `PATCH /pulls/<n>` with JSON body `{title}` and/or
|
||||
`{body}`. Returns 201 with the full PR. Use `jq -n --arg` to build JSON; for
|
||||
multi-line bodies write the text to a temp file and `--arg b "$(cat b.txt)"`.
|
||||
(jq CLI: a bare `---` argument after the program is parsed as a flag → error;
|
||||
fold footers into the --arg string instead.)
|
||||
- **List open PRs**: `GET /pulls?state=open` → `jq -r '.[] | "#\(.number) \(.head.ref) \(.title)"'`.
|
||||
- **Read PR**: `GET /pulls/<n>` (no token; public repo) → `jq -c '{state,merged,mergeable,head:.head.ref}'`.
|
||||
`mergeable:false` is often Gitea's *not-yet-checked* state, not a conflict — prove a real
|
||||
conflict locally with `git merge-tree --write-tree HEAD origin/main` before believing it.
|
||||
- **MERGE**: `POST /pulls/<n>/merge` with `-H 'Content-Type: application/json'`
|
||||
and body **`{"Do":"merge"}`** — this build REQUIRES the `Do` field:
|
||||
`{}` or `{"DoNotMerge":false,...}` → 422; `{"Do":"merge"}` → 200 +
|
||||
`{"state":"closed","merged":true,"merge_commit_sha":"..."}`. Verify with a plain
|
||||
GET /pulls/<n> afterwards (merge endpoint's own stdout may not parse).
|
||||
Merge is the `--no-ff` merge commit on main; then `git -C ~/.omp fetch && git
|
||||
merge --ff-only origin/main` propagates the live deployment.
|
||||
- **Issues**: same shape at `/issues` (`POST {title, body}`), `/issues/<n>/comments`
|
||||
(`POST {body}`, `PATCH .../comments/<id>`) — both work with the temp-cfg auth.
|
||||
- **Labels**: `GET /labels` (list incl. numeric `id`); `POST /labels`
|
||||
`{"name":"<name>","color":"#rrggbb"}` to create; attach by ID with
|
||||
`POST /issues/<n>/labels {"labels":[<id>,...]}` (names are rejected);
|
||||
detach with `DELETE /issues/<n>/labels/<id>`. serverAppConfigs gained
|
||||
needs-triage / needs-info / ready-for-agent / ready-for-human / wontfix
|
||||
(2026-10-20) alongside the older `human-only`.
|
||||
|
||||
## Review comments: the API lies on 1.27.3 (learned 2026-10-18)
|
||||
- `GET /pulls/<n>/comments` (inline review comments) → **404 `not found` on every
|
||||
route** (token or not, `?pull_request_id=` or not) even when a review with
|
||||
comments exists.
|
||||
- `GET /pulls/<n>/reviews` works (shows `state`, `comments_count`) but bodies are empty.
|
||||
- **The comments are readable from the PR HTML**:
|
||||
`curl -sS http://192.168.0.143:3000/dank/command-center/pulls/<n> -o /tmp/pr.html`
|
||||
then extract `<div class="content comment-content">` blocks
|
||||
(python: `re.finditer(r'comment-content', src)` → widen ±800 chars, strip tags).
|
||||
Duplicates appear (suggested-change UI shows text twice); dedupe mentally.
|
||||
- Per-conversation resolve API also 404s — flag resolutions in an issue comment instead.
|
||||
|
||||
## Body-footer law (AGENTS.md)
|
||||
Every issue/PR body ends with `---` line then `cwd: <wd> · session: <id>`. Keep
|
||||
it inside the --arg string (not a `---` CLI arg).
|
||||
|
||||
`cwd` is THIS session's own working directory (note a build worktree separately if they
|
||||
differ). When `/status` is unreachable (headless/subagent context), recover the session id
|
||||
from the session store instead of inventing one:
|
||||
`ls -t ~/.omp/agent/sessions/<cwd-slug>/ | head -2` → `<timestamp>_<uuid>` (the same uuid as
|
||||
`<uuid>.jsonl`); the slug is the cwd with `/` replaced by `-`. Subagent files live under the
|
||||
**parent** session's directory, so the newest entry there is the parent id.
|
||||
|
||||
## Gotchas
|
||||
- Content-Type: POSTs with `-d @file.json` MUST carry
|
||||
`-H 'Content-Type: application/json'` on the command line. `-H` entries put in
|
||||
the curl config file are ignored for this (header ordering quirk).
|
||||
- `xdg-open <pr-url>` lands the PR in Frank's Firefox the moment it is opened —
|
||||
law, not courtesy.
|
||||
- `jq: cannot index "not found" with "user"` = endpoint 404'd into an error
|
||||
object; check `-w '%{http_code}'` before blaming jq.
|
||||
- The token in `.git/config`'s remote URL is the same secret; the credential
|
||||
file (`~/.config/secrets/gitea`) is the one git actually consults (global
|
||||
credential.helper) and the one to reuse for API calls.
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: gitea-credential-store-url-format
|
||||
description: "Auth to the Dankclaw Gitea API without the empty-header trap: ~/.config/secrets/gitea is a git-credential-store URL file, so awk '/^password=/' returns empty; resolve the token via git credential fill, and Gitea writes need Authorization: token (reads pass anonymously on a public repo)."
|
||||
---
|
||||
|
||||
# Gitea auth on Dankclaw: the URL-format credential store
|
||||
|
||||
`~/.config/secrets/gitea` is `git config credential.helper = store --file ~/.config/secrets/gitea`, i.e. a **git-credential-store URL file**:
|
||||
|
||||
```
|
||||
http://dank:<40hex-token>@192.168.0.143:3000
|
||||
```
|
||||
|
||||
It does NOT contain `username=`/`password=` lines. So:
|
||||
|
||||
```bash
|
||||
# WRONG — silently returns empty string
|
||||
p=$(awk -F= '/^password=/{print substr($0,10)}' ~/.config/secrets/gitea)
|
||||
```
|
||||
|
||||
An empty `$p` makes every curl send `Authorization: token ` (blank) → Gitea replies `{"message":"token is required"}` on **writes**, while **reads still return 200** (the repo is public-readable, so a green `GET /labels` or `/issues` proves nothing about auth). Symptom: PR-create returns `token is required` even though a `GET` just "worked".
|
||||
|
||||
## Resolve the real token via git
|
||||
|
||||
```bash
|
||||
p=$(printf 'protocol=http\nhost=192.168.0.143:3000\n\n' \
|
||||
| git credential fill 2>/dev/null \
|
||||
| awk -F= '/^password=/{print substr($0,10)}') # 40-hex token
|
||||
```
|
||||
|
||||
## Write auth = token scheme, via a 600-perm -K file
|
||||
|
||||
```bash
|
||||
cfg=$(mktemp); chmod 600 "$cfg"
|
||||
printf 'header = "Authorization: token %s"\n' "$p" > "$cfg"
|
||||
curl -s -K "$cfg" http://192.168.0.143:3000/api/v1/user # confirms identity
|
||||
curl -s -K "$cfg" -X POST -H 'Content-Type: application/json' --data @body.json "$API/pulls"
|
||||
```
|
||||
|
||||
- Never pass the token with `-H` — it lands in `/proc/<pid>/cmdline`, world-readable.
|
||||
- Confirm identity with `GET /user` before trusting a write path; a `token is required` there means the token never authenticated.
|
||||
- Merges are a separate shape (form-encoded `do=merge`, auth-only cfg, NO `Content-Type` header) — see the gitea-merge-form-encoded skill.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: gitea-issue-labels-and-pr-curl-traps
|
||||
description: "Dankclaw Gitea API curl traps: PR create needs Content-Type application/json (fake \"[Head]: Required\"), issue labels only via POST /issues/N/labels (PATCH labels silently dropped), token from URL-format secret file."
|
||||
---
|
||||
|
||||
Operate Dankclaw Gitea (`http://192.168.0.143:3000`, repo `dank/franksign` and siblings) with curl without the three traps that produce misleading errors.
|
||||
|
||||
## Token extraction (URL-format store, NOT key=value)
|
||||
|
||||
`~/.config/secrets/gitea` is one line: `http://user:TOKEN@192.168.0.143:3000`. `cat`-ing it yields the whole URL, not the token.
|
||||
|
||||
```bash
|
||||
TOK=$(sed -n 's|.*://[^:]*:\([^@]*\)@.*|\1|p' ~/.config/secrets/gitea | head -1)
|
||||
```
|
||||
|
||||
Never pass the token as a `-H` argument (world-readable in `/proc/<pid>/cmdline`). Write a 600-perm curl config:
|
||||
|
||||
```bash
|
||||
printf 'header = "Authorization: token %s"\n' "$TOK" > /tmp/g.cfg
|
||||
```
|
||||
|
||||
## Trap 1 — PR create answers `"[Head]: Required"` when the body is the problem
|
||||
|
||||
Two independent causes, both look like a missing field:
|
||||
|
||||
1. No `Content-Type: application/json` header → Gitea parses the raw JSON as form data, sees no `head`, returns 422 `[Head]: Required`.
|
||||
2. Putting `data = "@/tmp/body.json"` inside a `-K` curl config does **not** expand the `@`.
|
||||
|
||||
Fix: add `header = "Content-Type: application/json"` to the config and pass `--data @/tmp/body.json` on the command line:
|
||||
|
||||
```bash
|
||||
curl -s -K /tmp/g.cfg -X POST --data @/tmp/pr.json \
|
||||
"http://192.168.0.143:3000/api/v1/repos/dank/franksign/pulls"
|
||||
```
|
||||
|
||||
## Trap 2 — issue label assignment via PATCH /issues/N is silently dropped
|
||||
|
||||
`PATCH /api/v1/repos/{o}/{r}/issues/{n}` with `{"labels":[id]}` returns **200/201 with the issue JSON**, yet a follow-up GET shows `labels: []` (it can even clear existing labels). Labels are a subresource:
|
||||
|
||||
```bash
|
||||
curl -s -K /tmp/g.cfg -X POST --data '{"labels":[4]}' \
|
||||
"http://192.168.0.143:3000/api/v1/repos/dank/franksign/issues/18/labels"
|
||||
```
|
||||
|
||||
200 returns the assigned label array. Resolve label names→ids via `GET /repos/{o}/{r}/labels` (franksign canonical five: ids 2–6; `ready-for-agent` = 4).
|
||||
|
||||
## Verification law
|
||||
|
||||
Both traps fail *successfully*. After any label/PR write, GET the resource and check the field changed; never trust the status code alone.
|
||||
|
||||
## Cleanup
|
||||
|
||||
`rm -f` the temp config + body files (config holds the token).
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
name: gitea-merge-form-encoded
|
||||
description: "Merge a Dankclaw Gitea PR with form-encoded do=merge — and the two body failures: 422 [Do]: Required (JSON body) and jsontext invalid character 'd' (form body with Content-Type: application/json header)."
|
||||
---
|
||||
|
||||
# Gitea merge on Dankclaw: form-encoded, and nothing else
|
||||
|
||||
`POST /api/v1/repos/<owner>/<repo>/pulls/<n>/merge` with `-d 'do=merge&merge_style=no-ff'`.
|
||||
|
||||
## The two body failures
|
||||
|
||||
1. **JSON body** → `422 {"message":"[Do]: Required"}`. The endpoint reads FORM fields, not JSON.
|
||||
2. **Form body but a `Content-Type: application/json` header** (e.g. left over in the curl `-K` config file from an earlier PR-body PATCH on the same request setup) → `422 {"message":"[]: jsontext: invalid character 'd' at start of value"}` — Gitea tries to parse `do=merge...` as JSON.
|
||||
|
||||
## Working pattern
|
||||
|
||||
Keep TWO curl config files when a session both patches PR bodies (JSON) and merges (form):
|
||||
```
|
||||
# merge.cfg — auth ONLY, no Content-Type
|
||||
header = "Authorization: token $TOKEN"
|
||||
```
|
||||
`curl -s -K merge.cfg -X POST -d 'do=merge&merge_style=no-ff' http://192.168.0.143:3000/api/v1/repos/<o>/<r>/pulls/<n>/merge`
|
||||
|
||||
- Success = **HTTP 200 with an empty body**; verify by landing a checkout (`git ls-remote origin main`), never by re-reading the PR.
|
||||
- PATCH `/pulls/<n>` (title/body) is the opposite: JSON body WITH the Content-Type header.
|
||||
- Token from `~/.config/secrets/gitea` via a 600-perm `-K` file, never `-H` (`/proc/<pid>/cmdline` is world-readable).
|
||||
@@ -0,0 +1,86 @@
|
||||
---
|
||||
name: gitea-pr-api-cycle
|
||||
description: "\"Open/update/comment/merge PRs and issues on the Dankclaw Gitea API (192.168.0.143:3000) with the token-safe sequence (POST /pulls rejects Basic with 'token is required'), numeric label ids, the form-encoded merge field, the async 405 'try again later' merge trap, the empty-token-returns-404 trap, and post-merge live-checkout fast-forward plus the plugin-package-name upgrade gotcha.\""
|
||||
---
|
||||
|
||||
# Gitea PR/issue cycle — Dankclaw API
|
||||
|
||||
Friction-earned 2026-10-18 (PRs #15–#20, issue #18), re-earned 2026-10-19 on `dank/dankvoice` PRs #4–#6 (the merge payload claim was wrong; an empty token masqueraded as a missing repo), and re-earned 2026-10-20 on `dank/command-center` PRs #55…#57 (the 405 recalculation trap). Re-earned again 2026-10-20 on `dank/dankvoice` #25 (merge gate loop worked; suite verdict delivered as an issue comment). Re-earned 2026-10-21 on `dank/command-center` issue #58 / PR #59 (issue labels are numeric ids; the `<type>/<issue#>-<slug>` branch name has to be right *before* the first push). Same day on `dank/serverAppConfigs` #64: the Basic-vs-token split below cost two wasted POSTs.
|
||||
|
||||
## Where
|
||||
- API base: `http://192.168.0.143:3000/api/v1/repos/dank/<repo>/{pulls,issues}` — repo varies (`command-center`, `dankvoice`, `serverAppConfigs`); numbering is per-repo, so a footer citing `#36` means nothing without the repo.
|
||||
- Web UI pages under `/dank/<repo>/pulls/N` are session-auth private: a plain URL fetch answers **404** even when the API and `xdg-open` work. Merge state questions go through the API (`GET /pulls/N` → `state`, `merged`, `merge_commit_sha`) — never conclude from a failed page fetch.
|
||||
- **Basic vs token split**: the `username=`/`password=` fields of `~/.config/secrets/gitea` as `Authorization: Basic …` work for **reads and issue/PR comment writes**, but `POST /pulls` refuses them with `403 token is required`. Pull-request lifecycle writes need the **token** header. Use the token for everything PR-shaped; never conclude "API is broken" from a Basic 403 on `/pulls`.
|
||||
- Token: never inline, never in argv (`-H "Authorization: token …"` lands in `/proc/<pid>/cmdline`, world-readable). Extract into a 600-perm curl config and hand it to curl with `-K`:
|
||||
```bash
|
||||
tok=$(sed -nE 's#^https?://[^:/@]+:([^@]+)@192\.168\.0\.143.*#\1#p' ~/.config/secrets/gitea | head -1)
|
||||
echo "tok len ${#tok}" # 40 — assert before use
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"
|
||||
curl -sS -K "$cfg" http://192.168.0.143:3000/api/v1/repos/dank/<repo>/labels
|
||||
rm -f "$cfg"
|
||||
```
|
||||
- **Empty token answers `404 {"message":"not found"}`, NOT 401** — Gitea hides the private repo. A 404 on a path you have used before means auth, not a typo'd repo. The trap that produces the empty token: `awk -F'/' … '{print $NF}'` on the store line (`scheme://user:token@host/`) — the host is in `$3`, the match never fires, you get a silent empty variable. Always assert the length before POSTing.
|
||||
- Multi-line bodies: build JSON in python (`json.dumps({"title":…, "body":open(f).read(),…})`) or `jq -n --arg b "$(cat b.txt)"`. NEVER pass `---` as a bare jq argument (parsed as a flag). Escaped quotes inside `python3 -c '…'` break on `\"` — write the script to a file or use a heredoc.
|
||||
- Always capture status: `-o r.json -w '%{http_code}'`; a bare `-o` hides failures. A `201` response piped straight into `python3 -c` that prints `None None` means you parsed an **error** body (missing `number`) — print `message` too.
|
||||
|
||||
## Create
|
||||
`POST /pulls` with `{title, body, base:"main", head:<branch>}` — JSON is accepted here (unlike merge). Plain `<branch>` works; the `owner:branch` fork form is unnecessary on same-repo branches.
|
||||
- **409 `pull request already exists for these targets [issue_id: N, …]`** — a PR for that head/base exists already. Common cause: the push message printed `/pulls/new/<branch>` and someone (Frank, or an `xdg-open` you ran) clicked Create. `GET /pulls?state=all`, find it, then `PATCH /pulls/N` with your title+body instead of re-posting.
|
||||
- `GET /pulls?state=all` on this Gitea omits `head`/`base` per entry — read `/pulls/N` for those, or key off `number`/`user`.
|
||||
- Issues: same shape, `POST /issues` `{title, body}`; `PATCH /issues/N` fixes a body retroactively (works for PRs too, since a PR is an issue) — used to correct a false verification claim in a PR that had already been merged. Body update returns **201**, not 200; confirm by reading `body` back, not by status code.
|
||||
- **Issue labels are numeric ids, not names.** `POST /issues` with `"labels":["ready-for-human"]` answers `422 {"message":"[]: json: cannot unmarshal JSON string into Go int64 within \"/labels/0\""}`. Resolve first: `GET /labels` → pick `id` by `name` → post `"labels":[20]`. (PRs take no `labels` field on create; label afterwards via `/issues/N/labels` with `{"labels":[id]}`.)
|
||||
- Comments (the channel for late-arriving evidence, e.g. a test-suite verdict posted after PR creation): `POST /issues/N/comments` with JSON `{body}` — works for PRs too, returns **201**.
|
||||
- **Open the issue before the first commit**, so `Refs #N` fits the message, and get the branch name right before the first push. `git branch -m <new>` + `git push -u origin <new>` + `git push origin --delete <old>` is a clean rename with no PR yet; do it rather than ship a branch that violates `<type>/<issue#>-<slug>` (a `docs/…` branch carrying a `fix:` commit is the common slip, and `git grep` on the old name still shows it).
|
||||
|
||||
## Merge (the 422 trap) — corrected
|
||||
`POST /pulls/N/merge` needs **form-encoded** fields; the merge form field name is lowercase **`do`**:
|
||||
|
||||
```bash
|
||||
curl -sS -X POST -K "$cfg" \
|
||||
--data-urlencode 'do=merge' --data-urlencode 'merge_style=no-ff' \
|
||||
http://192.168.0.143:3000/api/v1/repos/dank/command-center/pulls/N/merge
|
||||
```
|
||||
|
||||
JSON `{"Do":"merge"}` (what this skill used to claim) and header variants all return `422 {"message":"[Do]: Required"}`. `merge_style` ∈ `no-ff|merge|rebase|squash`. Success returns **200 with an empty body**; verify by landing a checkout rather than re-reading the PR. Merge only with Frank's explicit per-PR OK.
|
||||
|
||||
### The async 405 when several PRs merge in a row
|
||||
Merge #55, then #56 and #57 back-to-back, and #56/#57 answered:
|
||||
|
||||
```
|
||||
405 {"message":"Please try again later","url":"http://dankclaw:3000/api/swagger"}
|
||||
```
|
||||
|
||||
That is not a failure and not rate limiting: merging one PR moves `main`, which invalidates every other PR's cached mergeability, and Gitea recalculates it asynchronously. A merge attempt during recalculation is refused. Remedy — bounded poll on the PR, merge when the flag is true:
|
||||
|
||||
```bash
|
||||
for attempt in 1 2 3 4 5 6 7 8 9 10; do
|
||||
m=$(curl -sS -K "$cfg" "$api/pulls/$n" | python3 -c \
|
||||
'import json,sys;d=json.load(sys.stdin);print(d.get("mergeable"),d.get("merged"))')
|
||||
case "$m" in *True*) <merge POST>; break ;; *) sleep 4 ;; esac
|
||||
done
|
||||
```
|
||||
|
||||
All three landed within ~5 s of retrying. Treat `mergeable` as a readiness gate, not a report. Match `*True*` on the pair defensively: `False True` means already merged — stop, don't merge again.
|
||||
|
||||
### Verifying a merge without touching the deployment
|
||||
`GET /pulls/N` → `state: closed`, `merged: true`, `merge_commit_sha`. Then prove content from the bare repo, not the working tree: `git fetch -q origin && git show origin/main:<path> | grep -n <key line>` and `git log --oneline --graph -4 origin/main` (two parents = the `--no-ff` Frank asks for). A number you deliberately destroyed can legitimately reappear once — quoted in `omp-docs/log.md` as the wrong value; `git grep -c <badvalue> origin/main` naming only the log page is the pass condition, not a leftover. `Closes #N` in the PR body auto-closes the issue; confirm with `GET /issues/N` → `state`.
|
||||
|
||||
## Stacked PRs (no rebase)
|
||||
When two open branches edit the same section of the same file, merge the sibling branch into the later one (`git merge docs/other-branch`) instead of rebasing — Frank rejects the history rewrite. Consequences to state in the PR body: merging the lower PR first reduces the upper diff to just its own work; merging the upper first makes the lower an already-contained no-op. Either order is clean; `git branch -d` on the lower branch refuses until its commits reach `main`, because `-d` checks against the current HEAD.
|
||||
|
||||
## Post-merge landing
|
||||
- Command-center: the checkout IS the deployment — `git -C ~/.omp fetch -q origin && git -C ~/.omp merge --ff-only origin/main`. This refuses if **any** incoming path is dirty in the worktree — see `land-merged-pr-with-blocked-tree-ops` for the shared-workspace case where `checkout` and `stash` are both policy-blocked.
|
||||
- serverAppConfigs: `deploy.sh` pushes the dev checkout's branch first — if the checkout's `main` is behind a merge you just made (via the API), the deploy dies with `error: failed to push some refs`. `git merge --ff-only origin/main` in the deploy checkout BEFORE running `deploy.sh` after any externally-merged PR.
|
||||
- **A PR that changes a path you edited live adds that path to the discard list.** Tonight: the live dotfile fix was dirty *and* incoming, so landing needed `git -C ~/.omp checkout -- <that file>` as well as the usual file. Discard is content-free only when you proved byte identity first (`cmp` the branch file against the live one before merging) — say so in the PR body, and note that if `live-sync.sh` commits the dirty file first the merge stays clean because both sides hold identical bytes.
|
||||
- Package repos (`~/Projects/<pkg>`): same `--ff-only`, then reinstall (`omp plugin upgrade <name>`) and `cmp` the installed `dist/index.js` against the repo `dist/index.js` to prove the live artifact is the merged one. Check `ls -ld node_modules` after landing anything that touched it.
|
||||
- **`omp plugin upgrade` takes the npm package name, not the short name.** `omp plugin upgrade dankvoice` fails with `Failed to upgrade dankvoice: dankvoice is not installed` while the plugin is plainly installed; the package is `omp-dankvoice`. Find the truth with `ls ~/.omp/plugins/node_modules` or `find ~/.omp -path '*<name>/dist/index.js' -not -path '*/Projects/*'` — the installed tree is unscoped (`~/.omp/plugins/node_modules/omp-dankvoice/dist/index.js`), not `@dank/dankvoice`.
|
||||
- **`cmp` exiting 1 BEFORE the upgrade is the expected state** (installed bundle is the pre-merge build, diff reported at some char/line). Do not treat it as a failed deploy and do not "fix" the build; run the upgrade, then `cmp` again for byte identity, and grep the new bundle for a string only the merged code contains (e.g. `grep -c "Heckle instead of summarizing"`) as a second, semantic confirmation.
|
||||
- Prune: `git worktree remove --force <path>` (force: untracked deps link), `git branch -D`, `git push origin --delete <branch>` (allowed: local Gitea). Before removing a worktree, check no live process has a cwd inside it (`ls -l /proc/*/cwd | grep -c "wt/<name>"`) — same guard `live-sync.sh` uses — and copy any file you moved into a gitignored `.AGENTTRASH/` there to its permanent home first, or the removal destroys the only copy.
|
||||
|
||||
## Attribution footer (AGENTS.md)
|
||||
Every issue/PR body ends with `---`, then `cwd: <session cwd> · session: <session id>`. Read the real values, never invent an id.
|
||||
- **The session id is the UUID suffix of the LIVE transcript `.jsonl`, not the session directory name.** Transcript dirs are keyed by cwd (e.g. `-.omp-wt-wt-20261019-020309-6d1fc1f`) and ONE cwd can hold several sessions (resume / continuation forks), so `ls -t ~/.omp/agent/sessions/` cannot tell you which is yours and `ls -t <dir>` is equally blind. Pick the file being appended to right now: `ls -l --time-style=+%H:%M:%S <dir>/*.jsonl` beside `date +%H:%M:%S` — the mtime that equals the current second is the live session. The id is the part after `_` in `<timestamp>_<uuid>.jsonl`.
|
||||
- A `history.db` query for `session_id` filtered by `cwd` usually agrees, but it is not authoritative for WHICH of several sessions in this cwd is live — use the jsonl mtime test.
|
||||
- `local/` does NOT disambiguate: a forked transcript dir carries copies of the same `local://` artifacts, so matching artifact names proves nothing about which session is live.
|
||||
- `cwd` is that session's own working directory (the path the harness states), **not** the build worktree you committed in — when they differ, name the build worktree inside the body and keep `cwd` the session's.
|
||||
- Resolve `local://<name>.md` with `read local://<name>.md` (its header prints the absolute path) instead of guessing the directory.
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: gitea-review-comment-html-scrape
|
||||
description: "Read inline review comments on the Dankclaw Gitea when pulls/N/comments 404s (Gitea 1.27.3): scrape comment-content blocks and diff anchors from the PR HTML page"
|
||||
---
|
||||
|
||||
# Gitea review-comment extraction when the API 404s
|
||||
|
||||
On the Dankclaw Gitea (192.168.0.143:3000, Gitea 1.27.3) the review-comment API is dead for inline comments:
|
||||
`/api/v1/repos/{owner}/{repo}/pulls/{N}/comments` returns 404 `not found` **even with a valid token**, and `pulls/comments?pull_id=` / `?pull_request_id=` variants 404 too. `/pulls/{N}/reviews` works but its bodies are empty for inline reviews (read `comments_count` for how many exist); `/issues/{N}/comments` shows only conversation comments, never inline ones.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Confirm the review exists and how many inline comments it holds:
|
||||
`curl -sS http://192.168.0.143:3000/api/v1/repos/dank/command-center/pulls/{N}/reviews | jq -c '.[] | {id, state, comments_count}'`
|
||||
2. Scrape the PR page instead (no token needed for a public repo):
|
||||
`curl -sS http://192.168.0.143:3000/dank/command-center/pulls/{N} -o /tmp/pr.html`
|
||||
3. Extract each comment's text — every inline comment renders a `comment-content` block; dedupe (each comment appears twice in the HTML):
|
||||
|
||||
```python
|
||||
import re, html
|
||||
src = open('/tmp/pr.html').read()
|
||||
seen = set()
|
||||
for m in re.finditer(r'comment-content', src):
|
||||
t = html.unescape(re.sub(r'<[^>]+>', ' ', src[m.start()-100:m.start()+800]))
|
||||
t = re.sub(r'\s+', ' ', t).strip()[:400]
|
||||
if t not in seen:
|
||||
seen.add(t); print('>>', t)
|
||||
```
|
||||
|
||||
The comment text appears twice back-to-back inside each block ("text text") — read it once. Blocks start after `Copy Link|Copy Source`; ignore those chrome words.
|
||||
|
||||
4. **Locate the file/line anchor** (comment-content blocks carry no `data-path`/`data-line` attrs on this build): search the raw HTML for the comment string, walk *backward* ~4–7 KB, strip tags; the nearby `agent/...` path, `@@ -a,b +c,d @@` hunk header, and surrounding diff context lines identify the anchored file and the quoted source line.
|
||||
|
||||
## Replying / housekeeping
|
||||
|
||||
- Per-conversation resolve API does not exist on this build (404s) — reply to the round as a normal issue comment (`POST /repos/{o}/{r}/issues/{N}/comments`, token via `curl -K` temp cfg, `header = "Authorization: token …"`, 600 perms, `jq -n --arg b "$(cat b.txt)" '{body:$b}'`) listing which note got which fix commit.
|
||||
- Edit a posted comment: `PATCH /repos/{o}/{r}/issues/{N}/comments/{id}` with `{"body": ...}` — use to fix your own typos rather than double-posting.
|
||||
- Token source on this box: `~/.config/secrets/gitea` (grep the remote-URL pattern); never echo it.
|
||||
|
||||
## Why this works
|
||||
|
||||
The PR page is server-rendered with full inline comment bodies; Gitea's "Copy Source" UI element guarantees the raw text is present pre-markdown-render.
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
name: gitea-token-from-http-credential-store
|
||||
description: "Extract the Dankclaw Gitea token from ~/.config/secrets/gitea (it is the URL password, not key=value lines) for API calls, and diagnose the silent empty-token trap that makes public-repo reads pass while writes fail."
|
||||
---
|
||||
|
||||
The Gitea credential store line on Frank's boxes is **http** (not https) with a percent-encoded port, and the **token is the URL password** — there are NO `username=`/`password=` lines:
|
||||
|
||||
```
|
||||
http://dank:<40-char-token>@192.168.0.143%3a3000
|
||||
```
|
||||
|
||||
## Extraction that actually works
|
||||
|
||||
```bash
|
||||
tok=$(sed -nE 's#^[a-z]+://[^:]+:([^@]+)@.*#\1#p' ~/.config/secrets/gitea | head -1)
|
||||
curl -s -K <600-perm cfg file> http://192.168.0.143:3000/api/v1/<path>
|
||||
```
|
||||
|
||||
## `git credential-store --file … fill` answers EMPTY — do not debug it
|
||||
|
||||
Tried 2026-10-21 while opening `dank/command-center` PR #61: the obvious extraction
|
||||
|
||||
```bash
|
||||
printf 'protocol=http\nhost=192.168.0.143\nport=3000\n\n' | git credential-store --file ~/.config/secrets/gitea fill
|
||||
```
|
||||
|
||||
prints **nothing**, with no error. It is not a permissions problem and not a wrong path: the stored line encodes the port inside the host field (`@192.168.0.143%3a3000`), so a `host=`+`port=` query can never match it. Add `path=` or drop `port=` and it still answers empty. Read the file with `sed` (recipe above) and move on — the chain `TOKEN=$(…) && [ -n "$TOKEN" ] && …` fails silently at the test, which reads like a curl failure if the `&&` hides it.
|
||||
|
||||
## The silent-empty-token trap (bit 2026-10-21, cost a debug detour)
|
||||
|
||||
An `awk -F= '/^username=/{...}'` parse of this file yields `tok=""` **silently** — and reads still appear to work when the repo is **public**: `dank/serverAppConfigs` and `dank/command-center` return real issue/label data unauthenticated, so every census "passes" until the first **write** (issue comment, label, PR create/merge) fails with `{"message":"token is always required"}`-style errors or 404s (private repos 404 on an empty token — that is an auth problem, NOT a missing repo).
|
||||
|
||||
- `echo "tok len ${#tok}"` (must be 40) before believing ANY API result — reads included.
|
||||
- `curl -s -K <600-perm file>` carrying `header = "Authorization: token $tok"` — never `-H` on argv (`/proc/<pid>/cmdline` is world-readable; this leaked an HF token once).
|
||||
- Prove auth once per session: `GET /api/v1/user` → `"login":"dank"`.
|
||||
- The temp `-K` config holds a live token: `rm -f` it in the same session that made the calls, and never `cat` it.
|
||||
|
||||
## Other notes
|
||||
|
||||
- `git credential fill` fails headless here (`could not read Username … terminal prompts disabled`); read the store file directly instead.
|
||||
- Never print the token: mask when inspecting the file —
|
||||
`sed -E 's#(://[^:]+:)[^@]+@#\1REDACTED@#' ~/.config/secrets/gitea`
|
||||
- Issue/PR numbers are **per repo**: `dank/dankvoice` #8–#11 (refactor, hardening, AbortError fix, menu defects) are unrelated to `dank/command-center` #8–#11 (SDLC, trunk-law). Read the body from the repo the work targets before referencing an issue in a commit message.
|
||||
@@ -0,0 +1,52 @@
|
||||
---
|
||||
name: gitea-token-out-of-remote-url
|
||||
description: "Move a Gitea/HTTP git token out of remote.origin.url into a credential store file, and verify auth works with prompts disabled."
|
||||
---
|
||||
|
||||
# Move a git token out of the remote URL
|
||||
|
||||
A remote URL like `http://user:<token>@host:port/org/repo.git` puts the credential in
|
||||
`.git/config` (copied by every clone), prints it in `git remote -v`, and git-lfs echoes the
|
||||
whole URL in its `Locking support detected on remote` warning on every push.
|
||||
|
||||
## 1. Capture the token from the existing remote (never paste it into a message)
|
||||
```bash
|
||||
url=$(git remote get-url origin)
|
||||
user=$(printf '%s' "$url" | sed -E 's#^http://([^:]+):[^@]+@.*$#\1#')
|
||||
tok=$(printf '%s' "$url" | sed -E 's#^http://[^:]+:([^@]+)@.*$#\1#')
|
||||
```
|
||||
|
||||
## 2. Write the credential file (URL form, port included)
|
||||
```bash
|
||||
umask 077
|
||||
printf 'http://%s:%s@HOST:PORT\n' "$user" "$tok" > ~/.config/secrets/gitea
|
||||
```
|
||||
Git 2.56 `credential-store` does NOT parse the `protocol=`/`host=`/`password=` block shape
|
||||
that `git credential fill` accepts on stdin: a block-format file gives a SILENT empty get,
|
||||
then git falls through to askpass (`could not read Username`). One URL per line, port included.
|
||||
|
||||
## 3. Point git at it, strip the URL, silence the LFS echo
|
||||
```bash
|
||||
git config --global credential.helper 'store --file ~/.config/secrets/gitea'
|
||||
git remote set-url origin http://HOST:PORT/org/repo.git
|
||||
git config lfs.http://HOST:PORT/org/repo.git/info/lfs.locksverify true
|
||||
```
|
||||
|
||||
## 4. Verify with prompts disabled (a hang is a failed credential)
|
||||
```bash
|
||||
printf 'protocol=http\nhost=HOST:PORT\n\n' | git credential-store --file ~/.config/secrets/gitea get
|
||||
GIT_TERMINAL_PROMPT=0 git ls-remote --heads origin main # read auth
|
||||
GIT_TERMINAL_PROMPT=0 git push --dry-run origin main # write auth
|
||||
git remote get-url origin # must contain no '@'
|
||||
grep -l "$tok" ~/.gitconfig */.git/config ~/.zshrc # must find nothing
|
||||
```
|
||||
A successful push over the bare URL is the end-to-end proof: the credential came from the file.
|
||||
|
||||
## 5. Sweep the leak
|
||||
Same shape on every clone on the host — loop over `find ~ -maxdepth 4 -path '*/.git/config'`,
|
||||
read `remote.origin.url`, scrub any URL containing `@`. One credential file serves them all
|
||||
via the global helper.
|
||||
|
||||
## Rotate
|
||||
Rewrite only the credential file, then delete the old token in the Gitea UI
|
||||
(Settings → Applications). No remote URL, rc file, or repository changes.
|
||||
@@ -0,0 +1,23 @@
|
||||
---
|
||||
name: gitflow-back-merge-plumbing
|
||||
description: "(HISTORICAL for dank/command-center — switched to trunk-based 2026-10-18, no develop branch.) Back-merge main into develop with pure git plumbing when branch checkout is policy-blocked; avoids the polluted-base evil-merge trap."
|
||||
---
|
||||
|
||||
# Git-flow back-merge via plumbing (checkout blocked)
|
||||
|
||||
## Applicability
|
||||
The command-center repo retired `develop` on 2026-10-18 (trunk-based; see its AGENTS.md § Trunk-based SDLC). Use this only in repos still running git-flow where a `main -> develop` back-merge must land and branch checkout is forbidden. Never merge while HEAD is a session branch — the merge silently lands on it.
|
||||
|
||||
## Procedure
|
||||
1. `git fetch origin` and verify with `git ls-remote --heads`.
|
||||
2. Find the CLEAN fork point: `mb=$(git merge-base origin/develop origin/main)`. NEVER build the merge commit with a develop-side that already contains main (e.g. a previous broken merge): merge-base then equals main, main-side looks "unchanged", and stale develop content wins everywhere — a clean rc=0 evil merge. Sanity-trigger: if a merge you expect to conflict resolves rc=0, suspect the base.
|
||||
3. Real merge, no worktree: `out=$(git merge-tree --write-tree <clean-dev-tip> origin/main); rc=$?; tree=$(echo "$out"|head -1)`. rc=1 = conflicts; list them from the "Conflicted file info" section (lines starting with a 6-digit mode), NOT from `ls-files -u` in some other worktree.
|
||||
4. Resolve in index only: `export GIT_INDEX_FILE=/tmp/idx.$$; git read-tree $tree; git update-index --add --cacheinfo 100644,$(git rev-parse origin/main:<path>),<path>` per conflicted path (pick a side deliberately — say which in the message).
|
||||
5. `final=$(git write-tree)`; prove no markers: `git grep -l '^<<<<<<<' "$final"` must be empty (run it correctly — an empty count from a malformed invocation hides markers).
|
||||
6. `commit=$(git commit-tree "$final" -p <clean-dev-tip> -p origin/main -m "...")`; `git push --force-with-lease=develop:<current-dev-tip> origin "$commit:develop"` (lease prevents clobbering concurrent pushes; document superseded commits). A back-merge onto an untouched develop is additive — plain push, no lease needed.
|
||||
7. Verify ALL of: `git merge-base --is-ancestor origin/main origin/develop`; grep a distinctive merged value in `origin/develop:<file>` (content, not just ancestry); `git diff --stat origin/main origin/develop` matches expectations.
|
||||
|
||||
## Traps
|
||||
- `merge-tree` writes conflict markers INTO the tree for conflicted paths — always overwrite every conflicted path before commit-tree.
|
||||
- A wrong tree pushed as a merge commit makes every later merge against it resolve "cleanly" to stale content; fix by rebuilding from the clean fork point and force-with-lease, citing the broken commits in the message.
|
||||
- Gitea PR merge API needs `Content-Type: application/json` AND `{"Do":"merge"}` in the BODY (query param alone 422s); a 405 "Please try again later" right after the base branch moved is transient mergeability recomputation — wait a few seconds and retry.
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
name: gitignore-trailing-slash-symlink-contamination
|
||||
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get committed; detect with git ls-tree mode 120000, fix with git rm --cached."
|
||||
---
|
||||
|
||||
A `.gitignore` entry written with a **trailing slash matches directories only**. A symlink of the same name is not a directory, so it slips past the rule, and `git add -A` in a `git worktree` checkout silently **tracks** it. Cost measured 2026-10-19: a machine-specific symlink (`node_modules -> /home/blank/Projects/dankvoice/node_modules`) landed in `main` with a merged PR.
|
||||
|
||||
Why worktrees are the trigger: a worktree has no `node_modules` (or other installed-deps dir) of its own, so the cheap way to make build/test work there is a symlink into the primary checkout. With `node_modules/` ignored, nothing complains.
|
||||
|
||||
## Detect
|
||||
|
||||
```bash
|
||||
git ls-tree HEAD -- node_modules # mode 120000 == tracked symlink
|
||||
git ls-files -s | awk '$1==120000' # every tracked symlink in the index
|
||||
```
|
||||
|
||||
## Fix (branch, never on main directly)
|
||||
|
||||
```bash
|
||||
git rm -q --cached node_modules # untrack; leaves the file on disk
|
||||
```
|
||||
|
||||
plus relax the ignore rule to the slash-free form, with the reason inline so nobody re-adds the slash:
|
||||
|
||||
```
|
||||
# Trailing slash would only match directories: a node_modules symlink (the
|
||||
# shape `git worktree` checkouts end up with) slips past it and gets tracked.
|
||||
node_modules
|
||||
```
|
||||
|
||||
Then `git check-ignore -v node_modules` must print the rule, and the suite runs after (the symlink still resolves, it is just ignored).
|
||||
|
||||
## Prevent
|
||||
|
||||
In a fresh worktree, create the deps symlink **after** confirming the ignore rule has no trailing slash:
|
||||
|
||||
```bash
|
||||
git check-ignore -v node_modules || sed -i 's|^node_modules/$|node_modules|' .gitignore
|
||||
```
|
||||
|
||||
Same class for any ignored-by-convention dir that a worktree might provide as a link: `node_modules`, `vendor`, `.venv`, `target`, `dist` when built artifacts are ignored.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: gluetun-dependent-boot-race-recovery
|
||||
description: "Diagnose the Dankclaw pattern where gluetun is active but its netns-sharing dependents (searxng, marreta, marreta-browser) are inactive (dead) with \"Dependency failed\" after a reboot — cascade-recover them and tie it to e2e 502/SSO failures."
|
||||
---
|
||||
|
||||
# Gluetun boot-race: netns dependents dead at boot, service is active
|
||||
|
||||
## Signature
|
||||
After a dankclaw reboot, `gluetun.service` is ACTIVE, yet e2e fails on
|
||||
`search.franksdank.cloud` (502 on the authenticated pass), the Marreta tile
|
||||
shows a blank interstitial, and the homepage crawl reports vhost resets.
|
||||
`systemctl --user status <dep> --no-pager` shows:
|
||||
|
||||
```
|
||||
Active: inactive (dead)
|
||||
<host> systemd[2224]: Dependency failed for <dep>.service
|
||||
<host> systemd[2224]: <dep>.service: Job ... failed with result 'dependency'
|
||||
```
|
||||
|
||||
## Cause
|
||||
Quadlet `Network=gluetun.container` emits `Requires=`/`After=` on
|
||||
`gluetun.service`. When gluetun's start job fails once transiently at boot
|
||||
and `Restart=` re-arms it seconds later, the dependents' start jobs are
|
||||
already consumed and are NEVER re-armed — they sit `inactive (dead)`
|
||||
indefinitely while gluetun runs. `Restart=always` on the dependent does not
|
||||
help: the service never ran, so the restart policy never armed.
|
||||
|
||||
## Diagnosis census (rootless, run as `dank`, NO sudo)
|
||||
```
|
||||
export XDG_RUNTIME_DIR=/run/user/$(id -u)
|
||||
uptime # reboot is the trigger
|
||||
systemctl --user is-active gluetun searxng searxng-valkey marreta marreta-browser
|
||||
systemctl --user status <dep> --no-pager | head # "Dependency failed" line
|
||||
journalctl --user --since "<boot>" | grep -iE "<dep>|dependency"
|
||||
```
|
||||
Dependents of the gluetun netns on dankclaw: `searxng`, `marreta`,
|
||||
`marreta-browser` (valkey is plain-networked and survives).
|
||||
|
||||
## Recovery (order matters: browser before marreta)
|
||||
```
|
||||
systemctl --user start marreta-browser marreta
|
||||
systemctl --user start searxng
|
||||
```
|
||||
Verify: `podman ps` shows them; `curl http://127.0.0.1:8081/healthz` (searxng)
|
||||
and gluetun-published 8082 (marreta) answer; then rerun the Playwright gate
|
||||
(`tests/e2e/run.sh`) — the 12-crawl / 13-sso / 16-marreta specs are the trip
|
||||
wires for this fault.
|
||||
|
||||
## Distinguish from the dead-pause-process failure
|
||||
The 2026-09-16 rootless-pause-process death breaks ALL podman commands and
|
||||
every quadlet loops; this fault leaves podman fully functional and only the
|
||||
gluetun-dependency chain is dead. The reboot repairs the former; manual
|
||||
cascade-start repairs this one.
|
||||
|
||||
## Open gap (dankclaw, as of 2026-10-15)
|
||||
The race recurred (observed 2026-10-15 boot 01:41). Quadlet law for these
|
||||
units has no cure inside the `.container` files themselves; options are a
|
||||
systemd user-level drop-in on gluetun
|
||||
(`ExecStartPost=/usr/bin/systemctl --user restart searxng marreta`) or
|
||||
converting the dependents' dependency to `BindsTo` + `StartLimit` retry
|
||||
semantics. Fleet precedent for boot-order pain: searxng/gluetun ordering,
|
||||
observability reboot gaps (AGENTS.md).
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
name: grafana-provisioned-dashboard-verify
|
||||
description: "Author and verify repo-provisioned Grafana dashboards on Dankclaw live: staging to the host provider path, the Grafana-13 unified-storage false-negative on grafana.db, the authenticated /api/search check, per-panel PromQL/LogQL validation before shipping, and the Loki $__auto shell-expansion trap in test harnesses."
|
||||
---
|
||||
|
||||
Verify or ship Grafana dashboards that are provisioned from the repo (`shared/grafana/provisioning/dashboards/*.json`, auto-loaded by the `fleet` file provider). Use when adding/staging a dashboard, or when asked to confirm provisioned dashboards actually load.
|
||||
|
||||
## Paths (Dankclaw)
|
||||
- Repo source: `shared/grafana/provisioning/dashboards/` (git is the owner).
|
||||
- Host provider dir: `/opt/grafana/provisioning/dashboards` (root-owned), bind-mounted `:ro` into the container at `/etc/grafana/provisioning/dashboards` (from `dankclaw/observability/quadlet/grafana.container`: `Volume=/opt/grafana/provisioning:/etc/grafana/provisioning:ro`).
|
||||
- Provider reload interval = 30s, so a staged file appears within ~35s.
|
||||
|
||||
## Stage on the host (canonical install is `deploy.sh dankclaw --service grafana` after merge)
|
||||
```
|
||||
scp -q <files>.json dankclaw:/tmp/
|
||||
ssh dankclaw 'for f in <names>; do sudo install -m644 -o root -g root /tmp/$f.json /opt/grafana/provisioning/dashboards/$f.json; done'
|
||||
```
|
||||
|
||||
## Verification — what DOES and DOESN'T work
|
||||
|
||||
**DO NOT trust the sqlite `dashboard` table.** Grafana 13 uses **unified storage**: file-provisioned dashboards do NOT live in the legacy `dashboard` table. Reading `grafana.db` (`sudo python3 - <<PY ... select ... from dashboard`) returns **total dashboards: 0** even when everything is provisioned correctly — a false negative. There is also no `sqlite3` binary in the grafana image or on the host. Grafana logs **no** line on a successful provider load (only errors), so a clean `podman logs grafana` proves only "no error", not "loaded".
|
||||
|
||||
**The authoritative check is `/api/search` through an authenticated browser session.** No admin password is configured and anonymous is off — do NOT reset the Grafana admin to do a read check. Drive the RP SSO login with the Playwright `qa-e2e` storage state, then GET the API:
|
||||
```js
|
||||
const ctx = await b.newContext({ storageState: '.auth/qa-e2e.json', userAgent: "<chrome UA>" });
|
||||
const p = await ctx.newPage();
|
||||
await p.goto('https://grafana.franksdank.cloud/', { waitUntil: 'domcontentloaded' });
|
||||
for (let i=0;i<30;i++){ const t=await p.locator('body').innerText(); if(!t.includes('not a bot')) break; await p.waitForTimeout(3000);} // clear Anubis PoW
|
||||
await p.goto('https://grafana.franksdank.cloud/login/generic_oauth', { waitUntil: 'domcontentloaded' });
|
||||
await p.waitForTimeout(6000);
|
||||
const r = await ctx.request.get('https://grafana.franksdank.cloud/api/search?type=dash-db&limit=50'); // 200 + [{uid,title}...] = provisioned
|
||||
```
|
||||
Run it from the checkout that has `tests/e2e/node_modules` + `.auth` + `.env.local` (a fresh worktree lacks them; symlink `node_modules` and copy `.auth` + `.env.local`). Throwaway `_*.cjs`, then move to `.AGENTTRASH/`.
|
||||
|
||||
## Validate every panel query BEFORE shipping (catch empty / mis-scaled panels)
|
||||
Query Prometheus (`http://127.0.0.1:9090/api/v1/query`) and Loki (`http://127.0.0.1:3100/loki/api/v1/query`) on Dankclaw loopback and confirm non-empty. Real traps found here:
|
||||
- **Unit mislabel**: `sum by(job,mode)(rate(node_cpu_seconds_total{mode!="idle"}[5m]))` returns **cores per mode** (up to #cores) — unit `short`, NOT `percentunit`. For a per-host busy fraction use `1 - avg by(job)(rate(node_cpu_seconds_total{mode="idle"}[5m]))`.
|
||||
- **Missing metrics**: `scrape_duration_seconds_bucket` and `prometheus_*`-prefixed self metrics may be absent — confirm from `/api/v1/label/__name__/values?match[]=<metric>` before building a panel; `prometheus_target_interval_length_seconds` (a summary with a `quantile` label) exists.
|
||||
- **Alertmanager** runs on :9093 but is **NOT a Prometheus scrape target** → its metrics are unavailable; don't build panels on it (they read empty).
|
||||
- **Loki streams**: journal arrives as `{job="systemd-journal"}`; `container_name` is a **parsed JSON field, not a stream label** → group at query time: `sum by (container_name)(count_over_time({job="systemd-journal"} | json [$__auto]))`.
|
||||
- **Shell trap in test harnesses**: `$__auto` inside a double-quoted ssh/curl string is eaten by the local shell → empty query → Loki returns empty (looks like a broken panel, is a harness bug). For live validation substitute a concrete `[5m]` window, and drive Loki queries with an on-host Python `urllib` client to avoid quoting hell entirely.
|
||||
|
||||
## Datasource uids
|
||||
`prometheus` (uid `prometheus`), `loki` (uid `loki`); reference as `"datasource": {"type":"prometheus","uid":"prometheus"}` in panel JSON.
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
name: grep-secret-gate-hook-traps
|
||||
description: "Write a zero-dependency grep-based pre-commit secret gate that fails closed: the GNU grep -- one-shot trap, POSIX ERE limits, bracket collating elements, and rc=2-as-success in command substitution."
|
||||
---
|
||||
|
||||
# Fail-closed grep secret gate (pre-commit hook)
|
||||
|
||||
Use when writing/auditing a bash pre-commit hook (or any CI tripwire) that scans
|
||||
staged content with `grep -E` for credential shapes — no gitleaks available.
|
||||
|
||||
## The four traps (all personally earned; each silently PASSES bad content)
|
||||
|
||||
1. **`--` is a ONE-SHOT end-of-options marker.** `grep -nE -- 'p1' -- 'p2' 'p3'`
|
||||
takes the later `--` and patterns as FILE names → rc=2, errors on stderr.
|
||||
Correct multi-pattern form: repeat `-e` for every pattern; never `--`.
|
||||
2. **`if hits=$(... | grep ...)` treats rc=2 as success.** Any non-zero-but-not-1
|
||||
(broken pattern, unreadable file) enters the `then` branch with EMPTY output.
|
||||
Never branch on the substitution's truthiness. Capture rc explicitly:
|
||||
`hits=$(...); rc=$?` then: `rc>=2` → BLOCK (fail closed — a dead regex is
|
||||
worse than none), `rc==1` → clean, `rc==0` → hits.
|
||||
3. **POSIX ERE (`grep -E`) has no `(?:...)` groups.** Use plain `(...)`.
|
||||
4. **`[:-]` inside a bracket expression starts a collating element** →
|
||||
`grep: repetition-operator operand invalid` (rc=2, see trap 2). Put `-` FIRST
|
||||
in every class: `[-A-Za-z0-9/+=_.:+]`.
|
||||
|
||||
## Gate structure that works
|
||||
|
||||
- Scan **staged content**, not the working tree: `git show ":$f"` over
|
||||
`git diff --cached --name-only --diff-filter=ACM`.
|
||||
- Denylist doc/fixture paths (`README`, `docs?/`, `tests?/`, `*.example`) — else
|
||||
the hook blocks its own patterns and every test fixture.
|
||||
- Report `BLOCKED <file>:<line>` only — never echo the matched secret.
|
||||
- Wire with tracked `hooks/pre-commit` + `git config core.hooksPath hooks`
|
||||
(lives in the COMMON git dir config → one set in the main checkout covers
|
||||
every worktree). Fresh clones are unprotected until it's set; document it.
|
||||
- Worktrees lacking `hooks/` while `core.hooksPath` is set: git dies on commit
|
||||
(`cannot run ... No such file or directory`) — merge main in, don't unset.
|
||||
- Do NOT ship the hook in an npm plugin artifact (`files`): node_modules never
|
||||
runs git; pin its ABSENCE in the pack test instead.
|
||||
|
||||
## Test it red-first through the real seam (bats)
|
||||
|
||||
`git init` a fixture repo, `cp` the real hook in, `git config core.hooksPath hooks`,
|
||||
then drive REAL `git commit` and assert:
|
||||
- credential-shaped `.env` (RFC-2606 domains / all-zero ids as inert fixtures) →
|
||||
non-zero AND `git log` count unchanged (blocks, not warns);
|
||||
- PEM header line → blocked;
|
||||
- clean prose → passes;
|
||||
- the hook committing ITSELF → passes (trap 1/3/4 usually die here);
|
||||
- exit-code masking: `bats ... | tail` returns tail's rc — never `&& commit`
|
||||
behind a piped suite, or you commit on red.
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
name: halogen-ctx-pool-resize
|
||||
description: "Resize a halogen flash-server endpoint's context/KV pool (frank-dragoon pattern): engine knobs, seam band, RAM ledger, re-key sweep, deploy + probe."
|
||||
---
|
||||
|
||||
Two knobs: HALOGEN_CTX (hard per-request ceiling; longer prompt = 400 naming the limit, never truncation) and HALOGEN_KV_POOL_POSITIONS (ONE pool shared behind HALOGEN_KV_SLOTS). Ceiling raises buy a longer conversation; pool raises buy concurrency.
|
||||
|
||||
Laws: native window 262,144, beyond it needs HALOGEN_ROPE_YARN (engine 400 text) so never extrapolate for sizing; seam band ctx <= pool <= slots*ctx (pool == slots*ctx => two maximum-length requests coexist, pool == ctx => one fills it); harness contextWindow = ceiling - 45056 (clamp max_tokens = min(maxTokens, window-est-4096), compaction at 85 %, chars/4 under-counts ~3 %) and pool-only changes do NOT move it; ~28 KiB/position measured (200,704 -> 5.5 GiB, 262,144 -> 7.2, 524,288 -> 14.6), quote the ledger reservation not the preflight formula (block scratch + ~115 MiB/slot); the 51.2 GB n-gram table is mmap page cache (Pin::None) so RAM the pool takes is RAM that table loses, re-measure the warm prompt-cache ratio (136x baseline) after a raise; engine ledger vs 121.2 GiB = weights 62.9 + KV + working (3.8 + 8.4*MAX_TOK/16384) + prompt cache + HALOGEN_HOST_RESERVE_GIB=20 + co-tenancy ~10-15.
|
||||
|
||||
Procedure: read scripts/frank-dragoon.env and the boot ledger in /home/dank/logs/frank_dragoon.log (not the journal; a printed clamp is the referee); candidate boot via a transient FRANK_DRAGOON_ENV_FILE drop-in sourcing the repo env (never a scratch port), then trash it and prove DropInPaths empty or the repo env silently does not govern; edit env assignment + SHAPE lore and the unit Description; update the seam good_env fixture, -e pass-through asserts and pool-law comment without loosening the band; re-key every quote (deploy-remote.sh service comment, AGENTS.md topology row/engine laws/memory ledger/runbook index, docs index and LLM/index prose and mermaid node text, endpoint runbook shape paragraph plus a dated section, forward pointers in other vault pages, ~/.omp/agent/models.yml comment and contextWindow only if the ceiling moved, live OpenWebUI row in webui.db after backup); ./tests/run.sh; commit explicit paths never -A (subagents share the tree); ./scripts/deploy.sh dankclaw --service frank-dragoon (bare target = fleet drain); verify the new boot ledger and append the ledger table to the runbook plus a log.md worklog entry.
|
||||
|
||||
Probes: over-ceiling request -> 400 naming the ceiling in <1 s (preflight rejects before prefill); ~120k admission at max_tokens 45056 accepted, wall time = cold prefill ~1,500 t/s; two maximum-length admissions fired together must BOTH be admitted once pool >= 2*ctx; never report the first request after a boot.
|
||||
|
||||
Traps: systemctl mask fails on relinked units, disable forbidden; reservation is prompt + max_tokens so an over-budget request fails preflight even with a free pool; HALOGEN_FLASH_PIN_TRUNK=0 rejected by the v2 loader; rollback = revert the commit and redeploy.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: halogen-npu-feasibility-census
|
||||
description: "Census whether halogen-flash-server's NPU small-model path can run and swap models on a Strix Halo host: image gate, amdxdna/XRT presence, IOMMU and fabric-clock blockers, enablement deltas, boot-set-only model loading (no live reload), and the silent-fallback-to-Flash trap for unarmed chat-class NPU models."
|
||||
---
|
||||
|
||||
Use when asked whether halogen/halogen-flash-server can use the XDNA2 NPU (embeddings, rerank, moderation/guard, decider, `qwen3.5-2b` generation, `flux2-klein-4b` images, `qwen3-asr-*`) on a Ryzen AI Max / Strix Halo host, before proposing to enable it, or when asked whether it can **hot-swap / live-reload** NPU models like FastFlowLM does.
|
||||
|
||||
## What the feature is (not a speed knob)
|
||||
|
||||
The Flash trunk stays on the iGPU. `HALOGEN_NPU_MODELS=<names>` starts a second engine in the SAME container and on the SAME API port; the request's `model` field picks the executor. Unset -> server identical to before. Each NPU model is a precompiled AIE-ML program (`devices/devices.hnpm`, per-partition `devices/u<N>.elf`, `devices/consts.hnpb`, `<id>.hnpw` weights, `tokenizer/tokenizer.json`), loaded via the HOST's XRT -> kernel `amdxdna` -> `/dev/accel/accel0`. One pass at a time; `HALOGEN_NPU_QUEUE` 64, overflow queues server-side, then `503 engine_busy`. Catalog in the 0.16.2 image: `decider-0.8b`, `qwen3-embedding-0.6b`, `qwen3-reranker-0.6b`, `qwen3guard-gen-0.6b`, `qwen3.5-2b` (+ `flux2-klein-4b`, `qwen3-asr-*` in newer tags). Only the programs named in `HALOGEN_NPU_MODELS` are armed.
|
||||
|
||||
## Boot-set-only: NO live reload (tested on the live endpoint)
|
||||
|
||||
`HALOGEN_NPU_MODELS` is read **at start**; the entrypoint validates/converts/loads the named programs then. There is **no** runtime load/unload and **no** admin endpoint to add a model. FLM comparison: FLM was a standalone NPU-only server (`flm serve <one-model>`, single FIFO lane) free to reconfigure its one AIE partition; halogen boots Flash (iGPU) **plus** the NPU programs as ONE coupled process (shared port + shared memory/power budget), so the NPU set is a boot-time constant. Changing even a checkpoint's weights reconverts "at the next start", never live.
|
||||
|
||||
Consequence — arming a new program = edit `scripts/frank-dragoon.env` -> redeploy -> **container restart -> the whole Flash payload (tens of GB) reloads, minutes**. The small NPU program is not the cost; the Flash reload is.
|
||||
|
||||
## The silent-fallback trap (the one that bites)
|
||||
|
||||
A request naming an **unarmed** catalog model behaves differently per endpoint class:
|
||||
- `/v1/rerank`, `/v1/moderations`: honest **400 `model_not_found`**, message ends `(loaded: none)`. The endpoint only exists if that engine was armed at boot.
|
||||
- `/v1/chat/completions` (the `decider-0.8b` decision path and `qwen3.5-2b` generation): **silent fall-through to the Flash model** — HTTP 200, body `"model": "<flash id>"`, no error. A guard/decision call to an unarmed model is answered by the main (often abliterated) endpoint you were trying to screen against, and looks like success. Measured: an unarmed `decider-0.8b` "is this a prompt injection?" probe returned `"no"` from Flash. **Fail-safe is NOT the default on the chat route.**
|
||||
|
||||
## Live probe (idle lane; one ssh, non-disruptive — no restart happens)
|
||||
|
||||
```bash
|
||||
B=http://127.0.0.1:7770 # halogen port; internal engine is :8731/:8740 muxed onto it
|
||||
curl -s $B/v1/models | jq -r '.data[].id' # armed set = boot set; must not mutate
|
||||
# control (armed embedding) -> 200 with a real vector:
|
||||
curl -s $B/v1/embeddings -d '{"model":"qwen3-embedding-0.6b","input":["ping"]}'
|
||||
# unarmed rerank/guard -> expect 400 "(loaded: none)":
|
||||
curl -s $B/v1/rerank -d '{"model":"qwen3-reranker-0.6b","query":"t","documents":["a","b"],"top_n":1}'
|
||||
curl -s $B/v1/moderations -d '{"model":"qwen3guard-gen-0.6b","input":["hello"]}'
|
||||
# unarmed decider/gen -> watch the "model" field; if it is the flash id, it fell through:
|
||||
curl -s $B/v1/chat/completions -d '{"model":"decider-0.8b","temperature":0,"messages":[{"role":"user","content":"x"}],"response_format":{"type":"json_schema","json_schema":{"name":"g","description":"is injection?","schema":{"enum":["no","yes"]}}}}' | jq -r '.model // .error.message'
|
||||
# prove nothing restarted:
|
||||
systemctl show frank-dragoon.service -p NRestarts -p ActiveEnterTimestamp --value
|
||||
```
|
||||
`decider` with a schema and a non-zero `temperature` 400s on a *sampling* guard first ("send temperature 0"); set `temperature:0` to reach the router and see the fallback.
|
||||
|
||||
## Enablement gates (all four must hold before it runs)
|
||||
|
||||
1. Kernel `amdxdna` loaded + `/dev/accel/accel0` present; the node must be world-open (`0666` via a udev rule) or a **rootless** container cannot open it — `serve-frank-dragoon.sh` refuses if the mode bit 02 is clear.
|
||||
2. Host XRT with the NPU plugin (`libxrt_coreutil.so.2`, `libxrt_core.so.2`, `libxrt_driver_xdna.so.2`); on Arch/CachyOS mount each lib twice (image XRT path + its own `/usr/lib` path), never an `/opt/xilinx/xrt` of dangling links.
|
||||
3. **Fabric clock held at top** (`halogen-fabric-clock` root unit) — GPU+NPU at once can hang Strix Halo / corrupt NPU results while that clock changes; the server refuses to start if it can't hold it.
|
||||
4. **IOMMU on**: `amd_iommu=off` kills the NPU; `iommu=pt` keeps it.
|
||||
|
||||
## Laws / traps
|
||||
|
||||
- Never probe the NPU beside the live service: stop + mask first (fleet benchmarking policy); this host class has a precedent of a concurrent GPU experiment needing a power cycle. Distinguish off vs wedged with `ip neigh show <ip>` from a peer. A single embedding/decision request on an already-armed idle production lane is a normal request, not a benchmark — safe.
|
||||
- `/dev/accel/accel0` existing and `amdxdna` probing cleanly does NOT mean the NPU works - with IOMMU off the failure is at context/PASID creation, not at the device node. Do not trust older notes claiming the node itself is missing.
|
||||
- `dmesg` lines like `Failed to alloc dev bo memory, ret -28` while large NPU models take turns at NPU memory are benign (vendor-documented); the server recovers.
|
||||
- NPU model RSS is host-side, bypassing the container cgroup (GTT/shared pool), like all Strix Halo GPU memory — container stats under-report the true footprint.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
name: harden-precommit-secret-gate
|
||||
description: "Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer."
|
||||
---
|
||||
|
||||
Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer. Use when a secret gate is wrong (misses a leak) or too right (rejects an inert example and bricks the heartbeat).
|
||||
|
||||
## The two failure directions
|
||||
|
||||
A gate can be **too weak** (misses a real key) or **too strong** (rejects its own documentation and takes down an automated committer). Both are outages; the second is silent.
|
||||
|
||||
## Audit recipe for an existing gate
|
||||
|
||||
1. Read the hook and list what it actually inspects. The classic scope hole: `git diff --cached -- '*.env' '*.pem'` — an extension whitelist. A credential in `agent/config.yml`, a quadlet unit, or a SKILL.md is then invisible to the gate. The rule that survives: scan **the entire staged diff**, every path, and let only binary content out.
|
||||
2. Test fail-closed behaviour: can the hook be made to print nothing and still exit 0? Common escapes — running outside a git repo, unreadable staged diff, a hook that pipes through `grep` and inherits `grep`'s exit code from the wrong stage, or `set -e` swallowing the verdict. Verdict must be: unknown ⇒ **block**.
|
||||
3. Quarantine list, never tracked: `.env`, `auth.json`, `trust.json`, `*.pem`, `*.key`, `identity*` — and the `.gitignore` policy must quarantine env files **at any depth under any name** (`**/.env*`, not `.env`), or `config/prod.env` slips past.
|
||||
4. Escape hatch must be explicit, per-commit, and loud: `ALLOW_SECRETS=1 git commit …`, and the hook says so when it honours it.
|
||||
5. Gate the gate: a test that commits a fixture repo through the hook and asserts it blocks. `tests/secret_gate.bats` here covers: shape in an unexpected extension, bcrypt hash, JWT, placeholder/indirection allowed, prose allowed, binary ignored, unreadable diff refused, deletion of a credential never blocked, `ALLOW_SECRETS` override.
|
||||
|
||||
## Self-trip: the gate rejects its own example (live 2026-10-21)
|
||||
|
||||
Symptom: a systemd timer that commits machine-written files fails **every hour**, and the journal shows the gate's own refusal:
|
||||
|
||||
```
|
||||
SECRET GATE: staged diff contains 1 secret-shaped string(s):
|
||||
agent/managed-skills/git-secret-gate-hook/SKILL.md
|
||||
Fake secrets must be inert (RFC 2606 domains, all-zero ids, `AKIA0000000000000EXAMPLE`) AND must not
|
||||
```
|
||||
|
||||
The offender is the AWS **documentation** example key — entirely inert — and `AKIA[0-9A-Z]{16}` cannot tell it from a leak. So the gate's own advice about writing an inert fake is uncommittable by the gate. Auto-learned skills describe credential shapes for a living, so this recurs by construction, and an automated committer cannot use `ALLOW_SECRETS=1`: that would blind the gate for the *other* 64 staged files.
|
||||
|
||||
Diagnose, in this order:
|
||||
|
||||
```bash
|
||||
journalctl --user -u live-sync.service -b --no-pager | tail -25 # which file, which line
|
||||
git -C ~/.omp status --short | head # how much is queued behind it
|
||||
# then scan the WHOLE staged queue, not just the reported file:
|
||||
git diff --cached -U0 | awk '/^diff --git/{f=$3;sub("^a/","",f)} /^\+/ && !/^\+\+\+/ {
|
||||
l=substr($0,2)
|
||||
if (l ~ /sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|glpat-[A-Za-z0-9_-]{16,}|AKIA[0-9A-Z]{16}|xox[baprs]-[A-Za-z0-9-]{10,}|eyJ[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{10,}\.|-----BEGIN [A-Z ]*PRIVATE KEY-----/)
|
||||
print f": "substr(l,1,90) }'
|
||||
```
|
||||
|
||||
The second count is the decision input: one hit ⇒ fix the line; many ⇒ the gate needs an exemption channel before anything lands.
|
||||
|
||||
Fixes, in order of preference:
|
||||
|
||||
1. **Describe the shape instead of printing a matching literal** — `AKIA` followed by zeroes and `EXAMPLE`, spelled out in prose, cannot match. One line, the gate stays strict, the next timer run drains the queue.
|
||||
2. **Per-line allow marker** honoured by the hook (e.g. a trailing `secret-gate: allow` on the flagged line) — durable, keeps documentation honest, still fail-closed everywhere else. Needs a test both ways: a marked line passes, an unmarked one still blocks. Note the risk honestly: an attacker-adjacent string can self-authorise, so the marker belongs in review-able prose tiers, not in a diff that also touches secrets config.
|
||||
3. Never widen the regex to skip markdown, never run the automation with the override env var set.
|
||||
|
||||
Inert-fake rule for anything that must *show* a credential shape: RFC 2606 domains, all-zero ids, the word `EXAMPLE` — and if the shape itself trips the gate, describe it in prose rather than print it.
|
||||
|
||||
## Prose must not be collaterally blocked
|
||||
|
||||
Markdown explaining credential handling is the bulk of what a knowledge repo commits. `xox` inside a sentence about Slack token prefixes, or a `sk-` prefix discussed abstractly, must pass; the shapes are anchored long-suffix patterns for exactly that reason. When a gate fires on prose, that is a bug in the gate — but if the prose contains a *literal* matching token, the fix is the prose (option 1 above), not the pattern.
|
||||
|
||||
## Verifying a gate change
|
||||
|
||||
Write the fixture repo in a temp dir, commit through the hook, assert block/allow for each case. Do not test the regex by grepping the hook's source for the pattern — a copy of the pattern in a comment is not a gate that runs.
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
name: harness-model-config-wire-capture
|
||||
description: "Verify what the omp harness actually sends to a local inference endpoint (reasoning_effort, sampler keys, thinking-off) by pointing a scratch HOME at a logging proxy, instead of inferring it from models.yml comments."
|
||||
---
|
||||
|
||||
A `models.yml` comment claiming "the wire sends X" is a claim about the harness, not the engine, and the harness rewrites between the two: compat maps pass unmapped values through, effort ladders clamp before the map applies, and a "disable reasoning" role flag can be implemented as *send the lowest effort* rather than a switch. Read only the wire.
|
||||
|
||||
## Recipe (minutes, no production state touched)
|
||||
|
||||
1. **Logging proxy** — python `ThreadingHTTPServer` on `127.0.0.1:7799` forwarding body+path to the real engine and appending the request body's interesting keys (`model, reasoning_effort, max_tokens, temperature, top_p, top_k, chat_template_kwargs`) as one JSON line per request. Run it as a named bash service with `ready.port` so the port is proven before use.
|
||||
2. **Scratch HOME** — `mkdir -p /tmp/omph/.omp/agent`, copy the *candidate* `models.yml` + `config.yml` there, rewrite that provider's `baseUrl` to `http://127.0.0.1:7799/v1`. Never edit `~/.omp/agent/` — in `dank/command-center` the checkout is the deployment.
|
||||
3. **One-shot runs** — `HOME=/tmp/omph omp --model '<provider>/<id>:<effort>' --no-tools --no-session --no-skills --no-rules --no-extensions --no-pty --no-lsp -p 'Reply with exactly: OK'`, once per effort (`off minimal low medium high xhigh max`). Add `--print-thoughts` when the question is "did thinking actually happen".
|
||||
4. Read the log, and assert the value **after** the change. A config edit that never engaged is the failure mode this exists to catch.
|
||||
|
||||
## Traps found doing this on frank-dragoon (2026-10-21)
|
||||
|
||||
- `--model X:off` does **not** mean `reasoning_effort: off` on the wire. Reasoning is switched through `compat.reasoningDisableMode` (enum includes `lowest-effort`, `none-effort`, `qwen-enable-thinking-false`, `qwen-template-false`, `chat-template-thinking-false`); the default derives from `thinkingFormat`, and `lowest-effort` quietly keeps thinking alive (`:off` arrived as `"minimal"`, with a canned 26-token thinking block on every "off" request). `none-effort` sent `"none"` → `reasoning_tokens: 0`.
|
||||
- Effort levels are clamped to the harness's per-model ladder **before** `reasoningEffortMap` applies, so a map row for a level the ladder lacks is inert (`:max` arrived as `xhigh` even with `max: max`). Capture before promising a level works.
|
||||
- Map lookup is `compat.reasoningEffortMap?.[effort] ?? thinking?.effortMap?.[effort] ?? effort` — removing a row is a pass-through, never a 400. Claims about rejection must be proven against the engine.
|
||||
- Engine-side claims: ask for a deliberately bad value and read its 400 — the engine prints its own accepted set (`reasoning_effort 'maximum' unsupported; use minimal|low|medium|high|xhigh|max|none`).
|
||||
|
||||
## Engine-side probes that transfer to any OpenAI-compatible endpoint
|
||||
|
||||
- **Accepted parameters**: `{model, messages, max_tokens:1, <param>:<value>}` → status + `error.message`. Acceptance is not honouring: report it as acceptance.
|
||||
- **Level honoured?** same prompt at `max_tokens 2048` per level, compare `usage.completion_tokens_details.reasoning_tokens`. Equal values across levels = the parameter is ignored; `reasoning_closed_at` reveals a per-level thinking ceiling.
|
||||
- **Thinking off**: `chat_template_kwargs {"enable_thinking": false}` and top-level `enable_thinking: false` → expect `reasoning_tokens: 0`.
|
||||
- **Vision**: build a solid-colour PNG in-script (`zlib`+`struct`, no PIL), 512×512, ask for the colour in one word **with thinking disabled** — otherwise the answer is cut off by the thinking block and the probe is inconclusive. A 1×1 image only proves token accounting.
|
||||
|
||||
## Teardown
|
||||
|
||||
Kill the proxy service, remove the scratch HOME, the proxy script and its log. Never leave copies of `config.yml`/`models.yml` in `/tmp`: they carry provider endpoints and sometimes tokens.
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
name: hf-checkpoint-profile
|
||||
description: "Profile a Hugging Face safetensors checkpoint (sizes, dtypes, tensor-name space, gated/license) with compact HTTP-range reads — safe for subagents on small context slots."
|
||||
---
|
||||
|
||||
# Profile an HF safetensors checkpoint cheaply
|
||||
|
||||
Goal: inventory a repo (files/bytes, dtype histogram, tensor-name patterns, per-group GiB, config keys, license/access) without pulling gigabytes and without overflowing a subagent's context slot.
|
||||
|
||||
## Endpoints (all public, no token needed even for gated models)
|
||||
- `https://huggingface.co/api/models/<id>?blobs=true` → `siblings` (name+size), `gated`, `private`, `downloads`, `safetensors` metadata, `tags`.
|
||||
- `https://huggingface.co/api/models/<id>` for tags/base model; `https://huggingface.co/api/models?author=<x>&limit=100` to find sibling repos (canonical id check: cards often link both `X-1.5-Y` and `X-Y`).
|
||||
- `https://huggingface.co/api/models/<id>/resolve/main/<file>` for `config.json`, `model.safetensors.index.json`, `chat_template.jinja`, `LICENSE`, `generation_config.json`; `/raw/main/<file>` for text.
|
||||
|
||||
## dtype + per-group bytes without downloading shards
|
||||
Parse the safetensors header over HTTP Range: first 8 bytes = little-endian u64 header length, then that many bytes of JSON (`{name: {dtype, shape, data_offsets}}`). Fetch every shard header with a thread pool (131 shards ≈ seconds), then bytes = prod(shape) × dtype size. Collapse names with `re.sub(r"\.\d+\.", ".N.", k)` (and `\.shard_\d+` → `.shard_N`) to get the pattern space; group-sum by prefix to see where the mass sits (experts / ngram-PLE / MTP / vision / trunk).
|
||||
|
||||
Compare the key set against the base model's index to prove "full-weight finetune, identical names" (a zero-diff key set means per-tensor recipes written for the base apply verbatim, and it reveals fused vs per-expert layout, PLE/ngram shards, MTP head, vision tower, absence of `quantization_config`).
|
||||
|
||||
## Guardrails
|
||||
- Run python scripts from a clean directory: a stray `/tmp/inspect.py` shadows stdlib `inspect` and kills `concurrent.futures` imports.
|
||||
- Never put an HF token on a command line (`/proc/<pid>/cmdline` is world-readable); use `curl -K <mode-600 file>`.
|
||||
- `curl -f` / `--fail-with-body` and verify byte counts — a 401/403 writes the error body as the "artifact" and exits 0.
|
||||
|
||||
## Subagent context budget
|
||||
Subagents on a 131k-token slot die from `exceed_context_size_error` when told to read whole doc pages. Give grep-only instructions (`| head -40`, `-o`, `-c`), demand a line cap ("report in ≤250 lines, no verbatim dumps"), and prefer `git clone --depth 5` + local grep to web-page reads. A peer scout that overflows returns exit 1 with the 400 in the harness log and NO structured output — assume nothing was learned and redo the fact inline.
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: hf-large-artifact-publication
|
||||
description: "Publish a large quantized artifact + model card (and an optional anonymized corpus dataset) to HF Hub from dankserver: staging set, HF-style card spine, upload gates, transient-unit launcher, sentinel watching, card-metadata and scrub laws."
|
||||
---
|
||||
|
||||
# HF large-artifact publication (dankserver)
|
||||
|
||||
Publish a big model artifact (100+ GiB) + card, and optionally an anonymized corpus dataset, from dankserver under `frank8465`.
|
||||
|
||||
## Staging
|
||||
- Dev box `~/Projects/hf-swift15-upload/` (author card + scripts here), `scp -r` synced to `/home/dank/hf-swift15-upload/` on dankserver. Keep staging + logs + `.rc` sentinels after publication as receipts.
|
||||
- Payload set: artifact, `README.md` card, `scripts/` (recipe/drivers/probes), `artifacts/` (gate JSONs, per-position `.rows`, manifests, run logs).
|
||||
- Secret-scan the whole payload before arming: credential regexes (hf_/ghp_/pat, private keys, Bearer values, AKIA/sk-), LAN IPs, hostnames, domains, emails. Verdict each hit — variable *references* and upstream test fixtures are not leaks; check real values in context.
|
||||
|
||||
## Card law (HF validator)
|
||||
- Custom licenses MUST be `license: other` + `license_name:` + `license_link:` (mirror the base model's card trio — fetch `https://huggingface.co/api/models/<base>` and read `cardData`). A raw non-SPDX `license:` value raises `ValueError: Invalid metadata in README.md` inside `upload_file` — AFTER all local gates have run.
|
||||
- Card spine: what-the-format-is + engine pin digest, recipe table, quality gates (KL/top-1/PPL + per-position data shipped), proof hardware, exact `podman run` line, license chain, manifest table with the artifact's sha256 receipt, "In plain English:" block per section.
|
||||
- Quote the SHIPPED filename (build name), not a prettified one — a hub-side rename of a 100 GiB LFS file costs a full re-upload.
|
||||
|
||||
## Upload mechanics
|
||||
- venv `/home/dank/venvs/radcorpus` (huggingface_hub 1.28 + hf_xet); `HF_XET_HIGH_PERFORMANCE=1`; token from `~/.cache/huggingface/token` (see defuse-shell-rc-secrets pattern: `~/.config/secrets/<name>` 600 + rc loader, mirrored to the cache file). NEVER token in argv/chat.
|
||||
- Script gates, in order: sha256 of live artifact vs recorded digest (`gate 1 : MATCH`), `create_repo(..., private=False, exist_ok=True)`, small files first, artifact last, post-upload census — **compare remote sizes to local and FAIL the RC on any mismatch** (this caught a stale card mid-campaign).
|
||||
- hub 1.28 API traps: `upload_file`/`create_commit` kwargs are keyword-only; no `rename_file`, `hf_hub_url`, `HfApi.commit_api` (use `CommitOperationAdd`/`CommitOperationDelete` from `huggingface_hub` top-level if needed); `whoami` hides `canWrite` for legacy tokens — probe write scope by `create_repo(exist_ok=True)` on the real target.
|
||||
- zsh-over-ssh: never `echo ===` (triggers `=cmd` expansion); quote it.
|
||||
|
||||
## Launcher + watchers
|
||||
`setsid nohup` over ssh dies at session teardown. Use a transient user unit:
|
||||
```
|
||||
systemd-run --user --collect --unit=hf-upload -p SendSIGKILL=no \
|
||||
-p StandardOutput=append:/home/dank/logs/hf-upload.log \
|
||||
-p StandardError=append:/home/dank/logs/hf-upload.log \
|
||||
sh -c "cd <staging> && <venv>/bin/python -u <uploader>.py --go; echo \$? > /home/dank/logs/hf-upload.rc"
|
||||
```
|
||||
Sentinel: background ssh loop polling the `.rc` file (60 s period, bounded), tails the log on appearance. Progress bars are `\r`-fed — `tr '\r' '\n' | tail` to read them.
|
||||
|
||||
## Corpus dataset variant
|
||||
- License audit first: read `urls.txt`/manifest, fetch each source card (`license` in cardData). Share-alike sources (CC BY-SA) → publish derived corpus CC BY-SA with attribution; model weights unaffected.
|
||||
- Fleet-harvested slices = agent-session transcripts: scrub with deterministic substring-class rules (NOT word-boundary: catches `dankclaw_default`, `FRANKBOOK_SVCS`, `franksdank/alloy` image namespace): LAN IPs→10.x, hostnames→server-a/b/client-c, domain→example.com (catch `\.`-escaped forms + bare brand), `/home/<user>`, emails→user@example.com. Residual probes run on OUTPUT; whitelist self-replacements + audited fixture noise.
|
||||
- Publish original hashes, disclose non-byte-identity, cross-link model card ↔ dataset card, verify via `raw/main/README.md` fetches.
|
||||
- Dataset repo needs dataset-write scope (fine-grained token; access tokens 403). `repo_type="dataset"` everywhere; 45 MB rides in ~60 s.
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
name: homepage-smoke-verify-loop
|
||||
description: "Smoke-verify Dankclaw homepage changes locally before commit/deploy: isolated static tree + fake status + browser census, then deploy gate."
|
||||
---
|
||||
|
||||
# Homepage smoke-verify loop (serverAppConfig)
|
||||
|
||||
Verify `shared/homepage/` changes against a real browser BEFORE commit/deploy.
|
||||
|
||||
## Procedure
|
||||
1. `node --check shared/homepage/app.js` and `python3 -c "import json; json.load(open('shared/homepage/services.json'))"`.
|
||||
2. Isolated tree: `rm -rf /tmp/hp-smokeN && cp -r shared/homepage /tmp/hp-smokeN`.
|
||||
3. Fabricate runtime files in the tree:
|
||||
- `status.json`: `{"checked":"x","services":[{"name":<every services.json name>,"state":"up"}]}` (set one model `down` to exercise the red-pulse path).
|
||||
- `deployed.js`: `window.DEPLOYED_AT="smoke"`.
|
||||
4. Serve: `(cd /tmp/hp-smokeN && nohup python3 -m http.server 8099 >/dev/null 2>&1 & echo $! > /tmp/.hp-pid)`.
|
||||
- If reusing port 8099: `pkill -f "http.server 8099"` FIRST — a second server on a busy port fails silently and the OLD tree keeps serving.
|
||||
5. Browser census via eval tab on `http://127.0.0.1:8099/?v=<epoch>` — ALWAYS cache-bust with `?v=`; python http.server gets heuristically disk-cached by the browser. Assert: broken images (`.app-art,.mi-art` naturalWidth==0), computed styles (`getComputedStyle` for palette/fonts/clipPath), presence of new classes, `document.querySelectorAll('.docs section').map(s=>s.id)` for doc order, regex text census for banned words.
|
||||
6. Screenshot full page + `selector` shots; inspect visually.
|
||||
7. `./tests/run.sh` (expect only the known frank-primary MAX_SEQS failure), commit, `./scripts/deploy.sh dankclaw --service homepage` (runs Playwright e2e gate), then repeat the census against `https://www.franksdank.cloud/?v=<epoch>`.
|
||||
8. Cleanup: kill server, move `/tmp/hp-smokeN` to `~/.AGENTTRASH/YYYYMMDD-homepage-smoke/` (never `rm`).
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
name: homepage-theme-smoke
|
||||
description: "Locally smoke-test the Dankclaw homepage theme (style.css/app.js/docs.html) in the managed headless browser, with the stale-server traps that fake the result."
|
||||
---
|
||||
|
||||
Goal: prove a homepage theme change on a throwaway tree before commit/deploy.
|
||||
|
||||
Protocol:
|
||||
1. Stage a fresh copy: `cp -r shared/homepage/. /tmp/hp-smoke-<rand>/`, drop `.AGENTTRASH` dirs, and write a fake `status.json` (`{"generated_at":"x","services":[{"name":...,"state":"up"}]}`) — app.js fetches services.json + status.json and renders NOTHING without both.
|
||||
2. Serve: `cd <tree> && nohup python3 -m http.server 8099 --bind 127.0.0.1 &`.
|
||||
3. STALE-LISTENER TRAP (fakes every census): a survivor server from an older wave keeps answering on 8099 — `ss -ltn | grep 8099` proves a listener exists, not WHICH tree it serves. After every (re)start verify identity: `for p in $(pgrep -f "http.server 8099"); do readlink /proc/$p/cwd; done` must print the NEW tree; then `curl -s localhost:8099/<file> | grep -c <new-marker>`. `kill` the stale pid — never trust a `||`-guarded restart. Also: `pkill -f "http.server 8099"` from a shell whose own command line contains that string kills the shell; kill by pid.
|
||||
4. Browser census: navigate with `?v=<epoch>` cache-bust (browser disk-caches http.server responses); `waitForSelector('.mi-card', ...)` before evaluating — rendering is async (fetch). Geometry checks: rect-gap to 1px for bottom-pinned chips; screenshot for visual proof. `screenshot({path})` may not write — use the returned omp-sshots artifact.
|
||||
5. In headless pages `new Audio()` is detached: prove voice playback via `tab.requests()` (206 on the /sfx file) + the `mi-speaking` class, not a DOM `<audio>` query.
|
||||
6. Live-domain check after deploy: the homepage is at `/`, the docs page ONLY at `/docs/` — `/docs.html` falls through the SPA fallback and serves the homepage shell. Anubis adds a challenge: `waitForSelector` with 30-45 s timeout.
|
||||
7. Deploy loop: `./tests/run.sh` → commit/push → `./scripts/deploy.sh dankclaw --service homepage` (runs the Playwright e2e gate) → live census with fresh `?v=`.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user