docs: the vault and AGENTS.md describe two Dankclaw endpoints
halogen-flash-server is ACTIVE again with its 2026-09-16 measured ledger (budget 33.5 GiB against a need of 27.2 — no clamp — and 14.7 GiB left for everything else), ik_llama.cpp is retired with its measurements kept as the record of why, and the new frank-small runbook covers the -mtp file, the 64 KiB/token KV arithmetic, the MTP contract, and the reasoning_effort trap that cost me every subagent on the endpoint. AGENTS.md: topology, Active Services, runbook index, and three new llama.cpp lessons (MTP is a property of the file; --jinja is what `tools` needs; a model's template can reject the harness's thinking level). Three pre-existing defects found while editing it: a truncated sentence mid-file, a harness path that does not exist on either server, and the benchmarking stop/mask recipe still calling frank-large a root unit. Truthfulness notes worth keeping: the co-tenant stall NOTE prints again, so the claim that it "no longer prints" is gone and the three observed block counts are recorded instead of a guessed threshold; and the Observability page says out loud why frank-large has no scrape job rather than leaving a silent absence.
This commit is contained in:
1 parent
40850bbc8f
commit
49e8dfff66
8 files changed
+565
-153
No files matched your search
@@ -541,3 +541,83 @@ timestamp: 2026-09-14
|
||||
Active Services section + bench-policy unit type. Committed 795b0b3;
|
||||
tests 300 ok with the single pre-existing not-ok-73 (comfyui selector
|
||||
bug, unrelated).
|
||||
|
||||
## 2026-09-16 — Dankclaw re-split: halogen returns as frank-large, llama.cpp takes the 3090 as frank-small
|
||||
- `/dankclaw/frank-large/quadlet/frank-large.container` — halogen-flash-server
|
||||
restored as the active runtime (the 09-15 ik swap never removed the file) and
|
||||
retuned to the shape Frank asked for: `HALOGEN_CTX` 262144 → **200192**,
|
||||
`HALOGEN_KV_POOL_POSITIONS` 401408 → **400384** (exactly 2 × 200192),
|
||||
`HALOGEN_KV_SLOTS` 3 → **2**, `HALOGEN_MAX_TOK` 32768 → **24576** and
|
||||
`HALOGEN_HOST_RESERVE_GIB` 15 → **20** (arena −4.2 GiB, for the co-tenant),
|
||||
`HALOGEN_VISION_TOWER` **unset** (the tower moved to frank-small). Live ledger:
|
||||
budget `121 − 67.7 − 20 = 33.5 GiB` against a need of `27.2`, so **no clamp**;
|
||||
68.0 GiB weights + 11.0 KV + 16.5 working = 95.4 GiB held, 14.7 GiB left for
|
||||
everything else (475 contiguous 2 MiB blocks). The co-tenant stall NOTE is
|
||||
back, and the boot logged 4,350 compaction stalls (285 failed) — tight and
|
||||
fragmented, not out of memory.
|
||||
- `/scripts/frank-small.env`, `frank-small.service`, `serve-frank-small.sh` (new)
|
||||
— mainline llama.cpp `37b53fd` on the RTX 3090 serving
|
||||
`ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF` at IQ3_S: the `-mtp` build (the head is
|
||||
`blk.64.nextn.*` in the SAME file, so `--spec-type draft-mtp` runs the draft
|
||||
context on the main model's weights — no second copy, no `-md`) plus the BF16
|
||||
projector, 131072 ctx / 1 slot, f16 KV, `--jinja` and `--metrics` pinned, port
|
||||
7771, root systemd unit. 22,362 MiB of the card's 24,576 in use. Verified:
|
||||
`/health`, chat with `reasoning_content`, a `tool_calls` round trip, slot
|
||||
timing `draft acceptance 0.66–0.94`, and an image answered from the projector.
|
||||
- `/scripts/deploy-remote.sh` — `deploy_llm_large` flipped back to halogen
|
||||
(retire the ik ROOT unit into
|
||||
`~/.AGENTTRASH/20260916-halogen-restore/host-units`, pull + re-tag the image,
|
||||
**`wait_port_free 7770`**, install the Quadlet, start) and `deploy_llm_small`
|
||||
added (root unit, model-file gate, health on 7771). The port gate exists
|
||||
because the FIRST cutover attempt failed exactly there: pasta reported
|
||||
`Address already in use` about a second after the ik server exited, the unit
|
||||
died with code 126, and only `Restart=on-failure` recovered it. A redeploy now
|
||||
stops this service's own quadlet before waiting, or the gate would deadlock
|
||||
against itself.
|
||||
- `/scripts/deploy-plan.py`, `/scripts/deploy.sh`, `/scripts/test-changed.sh` —
|
||||
frank-small added to `SERVICE_ORDER` and the Dankclaw inventory (the stale
|
||||
`dankclaw:frank-npu` entry dropped with it); the changed-test mapping for
|
||||
`dankclaw/frank-large/*` pointed at `tests/frank_large_quadlet.bats`, retired
|
||||
on 09-15, so the wrapper had been failing on that path ever since — now
|
||||
`tests/frank_large_unit.bats`.
|
||||
- Retired to `scripts/.AGENTTRASH/`: the ik trio (`frank-large.env`,
|
||||
`frank-large.service`, `serve-frank-large.sh`), `chat-template.jinja` (only the
|
||||
ik runtime read it), and four benchmark harnesses bound to retired runtimes.
|
||||
- `/tests/frank_large_unit.bats` rewritten for the Quadlet (the `pool == 2 × ctx`
|
||||
invariant, no vision tower, the port gate) and `/tests/frank_small_unit.bats`
|
||||
added (root-unit shape, MTP-from-the-file contract, KV budget vs the card, port
|
||||
separation). One assertion failed on its first run because the negative grep
|
||||
matched a COMMENT in the serve script — the test now strips comments before
|
||||
grepping the effective flags.
|
||||
- `shared/homepage/services.json`, `shared/prometheus/prometheus.yml`,
|
||||
`rules/llm-fleet.yml`, `grafana/.../llm-fleet.json` — a frank-small card at
|
||||
`127.0.0.1:7771` and a `frank-small` scrape job at `192.168.0.143:7771`; the
|
||||
`frank-large` job **removed**, because the halogen engine has no `/metrics` at
|
||||
all (its routes are `/health`, `/cache` and `/v1/*`), so a target there would
|
||||
have been red by construction; the dashboard's endpoint picker had been sourced
|
||||
from a vLLM-only metric, so a llama.cpp job could never be selected. Live: 11
|
||||
targets, 7 up, 4 down (comfyui, gitea, services — all structural), and all
|
||||
three LLM homepage cards up.
|
||||
- Harness (`~/.omp/agent/models.yml` + `config.yml`, dev box) — third provider
|
||||
`llamacpp (dankclaw-3090)` for `:7771`; `frank-large` ctx 262144 → 200192 and
|
||||
`input: [text]`; roles `smol`, `tiny`, `task`, `vision` → frank-small (the
|
||||
image path and the small/fast roles), `slow`/`plan`/`designer`/`advisor` →
|
||||
frank-large. Two failures found by driving it, both fixed: (1) frank-small's
|
||||
template accepts only `xhigh|medium|low` for `reasoning_effort` and 500s on
|
||||
anything else, and subagents run at `high` — every subagent on that endpoint
|
||||
died until `compat.reasoningEffortMap` folded the levels onto that set; (2) the
|
||||
ik runtime that had been serving `frank-large` rejected every request carrying
|
||||
`tools` (`500 tools param requires --jinja flag`), which is what killed a
|
||||
monitoring subagent earlier in the day. Both endpoints now answer tool calls,
|
||||
and a subagent completes a tool-using turn.
|
||||
- Docs: `frank-small-qwen38-gsq-rco.md` (new runbook),
|
||||
`halogen-flash-server.md` re-activated with the 09-16 ledger,
|
||||
`frank-large-ik-uncensored.md` marked retired, LLM index, deployment flow,
|
||||
Observability and AGENTS.md topology/sections/lessons. Three AGENTS.md defects
|
||||
found on the way: a truncated sentence mid-file ("There is **no audio"), a
|
||||
harness path that does not exist (`/home/dank/.omp/agent/models.yml` — the
|
||||
harness runs on the dev box), and the benchmarking stop/mask recipe still
|
||||
describing frank-large as a root unit.
|
||||
- Deploy: `./scripts/deploy.sh dankclaw --service frank-large --service
|
||||
frank-small` — per-service health gates, then the e2e gate PASSED (31 passed,
|
||||
1 flaky-retried). Suite: 323 seams ok, 0 failures.
|
||||
Reference in new issue
Block a user