live: heartbeat sync 2026-10-18T16:47:59-07:00 (Refs #11)

2026-10-18 16:47:59 -07:00
1 parent a550db6027
commit 979e9ef711
@@ -0,0 +1,39 @@
+## LLM Endpoint Inventory
+
+### frank-secondary (Dankclaw, port 7771)
+
+**Current**: Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K (llama.cpp, RTX 3090, 200,704-token slots, vision via BF16 mmproj).
+
+**Target**: Underdog-Saluki-27B-1.0 (llama.cpp, RTX 3090, 400,000-token slots, vision via BF16 mmproj).
+
+Both run on the same hardware with the same runtime stack. The swap is model-only; container image and systemd unit remain unchanged. Vision support persists via the publisher's BF16 mmproj pass-through (`--mmproj`).
+
+**Context window law**: 400,000 tokens is the per-slot hard ceiling (`--ctx-size 401408 --parallel 2`). The harness clamps to `min(45056, contextWindow − est − 4096)` to keep room for the MTP head. With 400K context, a 45K token request leaves 355K for MTP head + estimation buffer.
+
+### frank-primary (Dankserver, port 7770)
+
+**Radiance 1.0.13**, ukisai/Swift1.5-Qwen3.8-Flash-Next BF16, 49,152 experts GPTQ w4 + 24 bf16-protected experts, 200K max-model-len, 47.7 GiB n-gram pinned to host RAM, 2 R9700s TP=2 (`--tp-wire wht6`), MTP spec decode depth 2, `--max-num-seqs 3`, `--max-model-len 200000`.
+
+### frank-large (Dankclaw, port 7770)
+
+**Halogen-flash-server 0.16.2**, Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated, 262,144 context per-slot, 2 slots × 262,144 ctx, MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0`), vision ON (v2 tower loads beside checkpoint), 524,288 positions KV pool.
+
+## Runtime Stack
+
+- **frank-primary**: Radiance 1.0.13 (StillDeadcode, digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`), plain `podman run`, root systemd unit, 2 R9700s TP=2.
+- **frank-large**: Halogen-flash-server 0.16.2 (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`), digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit, Strix Halo iGPU.
+- **frank-secondary**: llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build, root systemd unit, RTX 3090.
+
+All LLM services are root systemd units with `Restart=always`. Container exit codes drive service restarts (`ExecStop=podman stop -t 60 <name>`). Boot persistence requires quadlet units in `~/.config/containers/systemd/<name>.container`, enabled in user manager, with linger on for the deploy user.
+
+## Model Config Resolution
+
+Official model ids are StarCraft unit names: `frank-archon` → `192.168.0.235:7770` (`FRANK_PRIMARY_SERVED_NAME`), `frank-dragoon` → `192.168.0.143:7770` (`HALOGEN_MODEL_ID`), `frank-zealot` → `192.168.0.143:7771` (`ALIAS`).
+
+**Per-slot vs pool**: llama.cpp reports per-slot window (`--ctx-size 401408 --parallel 2`), not the pool. A 2×200K pool means 200K per slot, not 400K total.
+
+**Reasoning field**: llama.cpp and SGLang emit `reasoning_content`; vLLM emits `reasoning`. `models.yml` sets `reasoningContentField` per endpoint to match.
+
+**Vision**: All three endpoints accept `input: [text, image]`. Radiance runs the checkpoint's 27-layer Qwen-VL tower (patching, pad-token expansion, 3-part rotary, `--mm-max-patches` available). Halogen v2 loads the vision tower beside the checkpoint. Llama.cpp passes the model's BF16 mmproj via `--mmproj`.
+
+**Audio**: No audio input anywhere in the fleet (left with FLM NPU endpoint 2026-09-11, returned 2026-09-17, left again 2026-10-01).
@@ -0,0 +1,41 @@
+## Model Capabilities
+
+### Context Window
+
+**400,000 tokens** is the fleet-wide per-slot ceiling. The harness clamps max_tokens to `min(45056, contextWindow − est − 4096)` to preserve MTP head and estimation buffer. With 400K context, a 45K token request leaves 355K for MTP head + estimation.
+
+### Multimodal
+
+All three LLM endpoints accept `input: [text, image]`. Vision is native: Radiance and Halogen v2 load their vision towers beside the checkpoint; llama.cpp passes the BF16 mmproj via `--mmproj`.
+
+### Audio
+
+No audio input in the fleet. The FLM NPU endpoint had audio 2026-09-17 → 2026-10-01, but neither MiMo nor Qwen3.5 tower carries it.
+
+### MTP (Multi-Token Prediction)
+
+- **frank-primary** (Radiance): MTP spec decode depth 2, `--max-num-seqs 3`.
+- **frank-large** (Halogen): MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0`), 2 slots × 262,144 ctx, harness window 217,088 = ceiling − 45k estimate headroom.
+- **frank-secondary** (llama.cpp): MTP OFF (no `nextn` tensors exist in Qwen3.5-9B Q6_K GGUF).
+
+## Benchmarking Policy
+
+**STOP the production service BEFORE benchmarking**. A live service starves the GPU and corrupts results.
+
+**Procedure**:
+1. Stop and mask (self-healing would restart it):
+   - Dankclaw `frank-large`: `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
+   - Dankclaw `frank-secondary`: `sudo systemctl stop frank-secondary && sudo systemctl mask frank-secondary`
+   - Dankserver `frank-primary`: `sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
+2. Bench.
+3. Unmask and start; verify `curl http://192.168.0.<host>:7770/health`.
+
+**CRITICAL**: `systemctl disable` deletes the unit file from `/etc/systemd/system/`; `mask` symlinks to `/dev/null` and is safe/reversible. Re-deploy after benching instead of unmasking.
+
+**Gufo bench**: Cannot rank speculative-decode settings; will invert the ranking. Use for shape, not for A/B. Generate **at least 128 tokens** — at 8 tokens, `inter_token` is first-token noise and run-to-run spread swamps real differences. For A/B, replay **real** text through the live server and read the `Server-Timing` header (`ttft;dur=402.485` in the name half, split on `=` keys the dict `ttft;dur`).
+
+**GPU experiment law**: A bare 2-rank RCCL/torch.distributed probe against a live service can wedge the GPU driver (host went hard down ~20 minutes, power cycle required). If an experiment needs GPUs, stop and mask first, or use a host that serves nobody. Distinguish: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, merely wedged kernel usually still answers ARP.
+
+**Live measurement**: llama.cpp servers report honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`).
+
+**Scratch ports**: Not used on LLM hosts — a test process cannot share the single GPU anyway; validate against the service itself.
@@ -0,0 +1,10 @@
+{
+	"v": 1,
+	"lastConsolidatedAt": 1791601698371,
+	"lastResult": {
+		"at": 1791600472354,
+		"sessions": 1,
+		"deltas": 2,
+		"model": "frank-zealot"
+	}
+}
@@ -0,0 +1,32 @@
+## Documentation Voice
+
+Use terse, normative statements. Avoid "the user wants", "currently", "now", "fixed", "regressed", "recently". State what the system **does**, not what happened.
+
+## Bullet Budget
+
+At most 120 lines per file. Nearly empty or empty files are honest.
+
+## Single-Home Rule
+
+- `architecture.md`: structural/runtime decisions (component boundaries, chosen abstractions, protocol/storage direction, explicit "X over Y" with reason).
+- `product.md`: behavior, UX, defaults, scope, naming/terminology, workflow decisions.
+- `style.md`: visual/aesthetic language, presentation/typography conventions, prose/voice rules.
+
+## Concreteness Test
+
+Every bullet must change a fresh agent's behavior. "Provide configurable, reference-compatible interactions" is delete-tier. "Status bar: powerline segments, solid right edge, inward-facing left triangle, no bold, elapsed time ≤3 chars" is keep-tier.
+
+## Exclusions
+
+- Project docs digest (already documented).
+- Global user taste not a project decision.
+- Current-task state (bugs in flight, progress, plans, TODOs, "for now").
+- File paths, line numbers, function/type names, commit/issue ids (product component vocabulary allowed).
+
+## Friction Markers
+
+A terse marker like "(recurrently regressed)" MAY close a bullet where it strengthens the instruction; keep rare.
+
+## Grouping
+
+Group bullets under short `##` domain sections. One line per bullet where possible.