Frank
a34322aeb0
The managed-skill library was regenerated: 94 narrow per-incident skills
replaced by 25 broader ones (radiance-*, llamacpp-*, homepage-*, fleet/
endpoint ops). Git renders this as mass delete + mass add because whole
directories changed name; old content stays in history at 12c5e88.
Surface check: the 25 on disk are exactly the skills the harness advertises
this session.
1.7 KiB
name, description
| name | description |
|---|---|
| frank-primary-decode-tuning | Tune frank-primary (radiance TP=2) decode throughput: benchmark harness usage, flag levers, known optima and boot-failure walls. |
frank-primary decode tuning
Harness: bash autoresearch.sh (dev box) scp's scripts/frank-primary.env + scripts/autoresearch-frank-primary.sh to dankserver, stops prod (systemctl stop frank-primary, NEVER mask — relinked unit), boots scratch container arbench on :7770 (fresh prefix-cache tier at $STATE_DIR/kvcache/arbench for cold prefill), polls health 720s, sends ONE greedy completion (fixed 140k-char repo-corpus prompt -> 43,627 tokens, 256-token decode), parses timings into METRIC lines, tears down, restores prod via EXIT trap. ~4 min/run; decode noise ~0.3%.
Metric: METRIC decode_tps = timings.predicted_per_second at 43.6k depth.
Optima (measured 2026-10-06, keep these)
FRANK_PRIMARY_SPEC_TOKENS=2(unimodal: 1->31.89, 2->32.40, 3->31.72, 4->24.57 t/s; acceptance collapses past 3)FRANK_PRIMARY_MAX_SEQS=3withMAX_MODEL_LEN=200000(600k KV; +4.2% total; 4 slots regresses)FRANK_PRIMARY_EXPERT_VS_CACHE=0.82bracket-confirmed (0.78 fails boot, 0.84 -8.9%, 0.87 collapse)TP_WIRE=wht6,MAX_BATCHED_TOKENS=2048— REQUIRED at seqs=3: bf16 wire and MNBT 4096 both fail boot (VRAM)
Non-levers (decode-neutral, don't spend runs)
GPU_HEADROOM_MIB (48 = neutral), HOST_POOL_MIB 49152 (neutral), MAX_MODEL_LEN any change (regresses: bigger KV steals resident weights), prefix-cache tiers (prefill-side).
Rules
One flag per iteration; keep only > +1% decode over best; a boot failure is crash (trap restores prod). Constraint floor: MAX_SEQS*MAX_MODEL_LEN >= 400000.