llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot
The 2026-10-09 wire aliases become the service names everywhere: systemd units, serve scripts, env files and FRANK_* variables, deploy-plan/deploy-remote service keys, container names, Prometheus job names + alert rules, Grafana dashboard queries, bats seams, runbook filenames, homepage/AGENTS lore, and historical prose. Rollback tags renamed in lockstep. Residue in the tree: zero. Retired frank-medium/small/npu names are history and stay. Known casualty: scripts/patches/*.patch context lines were rewritten; those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
Frank committed
1 parent
5ced474eb9
commit
55f87fcdce
195 files changed
+2085
-2064
No files matched your search
@@ -30,7 +30,7 @@ timestamp: 2026-09-29
|
||||
|
||||
This is the ad hoc `qwen-fp8` container on Dankserver (rootless podman,
|
||||
image `docker.io/tcclaviger/vllm:latest`, engine
|
||||
`v0.29.0.dev0+g2bdbbc8080`), NOT the `frank-primary` systemd service —
|
||||
`v0.29.0.dev0+g2bdbbc8080`), NOT the `frank-archon` systemd service —
|
||||
the R9V production unit is untouched, so no stop-and-mask was required for
|
||||
any of this. All numbers are decode tok/s measured from the server's own
|
||||
`usage.completion_tokens / (wall - ttft)` — SSE chunk counting is invalid
|
||||
@@ -58,7 +58,7 @@ Published to the host LAN via the serve script's `PORTMAP` hook
|
||||
(`PORTMAP=7773:7770`): health and completions answer on
|
||||
`http://192.168.0.235:7773` (verified host-side 2026-09-29; 256k config,
|
||||
decode 36.49/34.55/35.98 through the published path). Host port 7773 was
|
||||
chosen over 7770 because `frank-primary` (7770) is stopped but ENABLED —
|
||||
chosen over 7770 because `frank-archon` (7770) is stopped but ENABLED —
|
||||
any future start would collide with a published 7770.
|
||||
|
||||
Device notes carried from the bring-up: `--device /dev/dri` must be the
|
||||
@@ -249,8 +249,8 @@ re-copies them; a recreate wipes container-local files).
|
||||
Two same-day cutovers moved this page's findings into production:
|
||||
|
||||
1. **Qwen FP8 + MTP4 replaced the failed GLM/DFlash2 experiment** (the GLM
|
||||
window is documented in [[frank-primary-glm53-flash-w4a16]]; git tag
|
||||
`frank-primary-fp8-mtp-20261001` is the last commit of that shape). The
|
||||
window is documented in [[frank-archon-glm53-flash-w4a16]]; git tag
|
||||
`frank-archon-fp8-mtp-20261001` is the last commit of that shape). The
|
||||
production trio took MAXLEN 200000 x 3, fp8 KV, MTP4, prefetch off —
|
||||
measured 37.4-40.6 tok/s at ~54% acceptance.
|
||||
2. **The overnight redo campaign re-measured EVERY lever at matched
|
||||
@@ -278,20 +278,20 @@ is only sampled at 3 depths; a 64k-192k-depth probe is the flagged gap.
|
||||
### The deployed shape (2026-10-03 shape G: 3 slots x 200704, fp8, precise memory, MNBT 2048, arena at the flag ceiling)
|
||||
|
||||
```
|
||||
FRANK_PRIMARY_MAXLEN=200704 200704-token window
|
||||
FRANK_PRIMARY_SEQS=3 --max-num-seqs 3 (SEQS=2 CANNOT boot - below)
|
||||
FRANK_PRIMARY_KV_BYTES=3166500000 PER RANK (fp8, 7,892 B/token -> 400,700 tokens)
|
||||
FRANK_PRIMARY_KV_DTYPE=fp8 --kv-cache-dtype fp8 (quantized)
|
||||
FRANK_PRIMARY_MNBT=2048 --max-num-batched-tokens 2048 (arena-free, see shape F)
|
||||
FRANK_PRIMARY_UTIL=0.982 budget = total x util, the law (1.0 unbootable, see below)
|
||||
FRANK_PRIMARY_CUDAGRAPH_BASE_MB=0 / _PER_GRAPH_MB=12 -e VLLM_CUDAGRAPH_MEM_*
|
||||
FRANK_PRIMARY_PRECISE_MEMORY=1 --expert-precise-memory (overhead 7.60 -> 5.25 GiB)
|
||||
FRANK_PRIMARY_EXPERT_OFFLOAD_MEM=120 --expert-offload-mem (host-region cap, explicit)
|
||||
FRANK_PRIMARY_GPU_MAX_HW_QUEUES=2 -e GPU_MAX_HW_QUEUES=2 (HSA queues/GPU)
|
||||
FRANK_PRIMARY_CACHE_DIR=/home/dank/.cache/frank-primary mounted at /root/.cache
|
||||
FRANK_ARCHON_MAXLEN=200704 200704-token window
|
||||
FRANK_ARCHON_SEQS=3 --max-num-seqs 3 (SEQS=2 CANNOT boot - below)
|
||||
FRANK_ARCHON_KV_BYTES=3166500000 PER RANK (fp8, 7,892 B/token -> 400,700 tokens)
|
||||
FRANK_ARCHON_KV_DTYPE=fp8 --kv-cache-dtype fp8 (quantized)
|
||||
FRANK_ARCHON_MNBT=2048 --max-num-batched-tokens 2048 (arena-free, see shape F)
|
||||
FRANK_ARCHON_UTIL=0.982 budget = total x util, the law (1.0 unbootable, see below)
|
||||
FRANK_ARCHON_CUDAGRAPH_BASE_MB=0 / _PER_GRAPH_MB=12 -e VLLM_CUDAGRAPH_MEM_*
|
||||
FRANK_ARCHON_PRECISE_MEMORY=1 --expert-precise-memory (overhead 7.60 -> 5.25 GiB)
|
||||
FRANK_ARCHON_EXPERT_OFFLOAD_MEM=120 --expert-offload-mem (host-region cap, explicit)
|
||||
FRANK_ARCHON_GPU_MAX_HW_QUEUES=2 -e GPU_MAX_HW_QUEUES=2 (HSA queues/GPU)
|
||||
FRANK_ARCHON_CACHE_DIR=/home/dank/.cache/frank-archon mounted at /root/.cache
|
||||
R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64 (container env)
|
||||
NO --speculative-config, NO --expert-prefetch
|
||||
FRANK_PRIMARY_NO_PREFIX_CACHING=1 --no-enable-prefix-caching (2026-10-05, the QFN align-mode GDN state fault — below)
|
||||
FRANK_ARCHON_NO_PREFIX_CACHING=1 --no-enable-prefix-caching (2026-10-05, the QFN align-mode GDN state fault — below)
|
||||
```
|
||||
|
||||
### 2026-10-05 — prefix caching OFF: the fork's QFN align-mode GDN state fault
|
||||
@@ -299,7 +299,7 @@ FRANK_PRIMARY_NO_PREFIX_CACHING=1 --no-enable-prefix-caching (2026-10-05, the Q
|
||||
**Symptom (Frank, 2026-10-04):** output turned repetitive/corrupted at
|
||||
~40k tokens into a long conversation; short sessions stayed clean.
|
||||
|
||||
**Evidence (live `podman logs frank-primary-qwen`, the 2026-10-04 18:19
|
||||
**Evidence (live `podman logs frank-archon-qwen`, the 2026-10-04 18:19
|
||||
boot):** one 0.6 s burst at 18:46:07 PDT — 5.6 h into service, one
|
||||
running request, and the engine's own 18:46:08 log line reading
|
||||
"Prefix cache hit rate: 48.1%". 48 `gdn_hip` faults, then the
|
||||
@@ -343,7 +343,7 @@ load; 4800 MT/s is the JEDEC DDR5 floor on 5600 MT/s-rated DIMMs (not an
|
||||
overclock); and a two-head, single-slot NaN state is a state-path fault,
|
||||
not scattered finite bit corruption.
|
||||
|
||||
**Fix.** `FRANK_PRIMARY_NO_PREFIX_CACHING=1` → `--no-enable-prefix-caching`
|
||||
**Fix.** `FRANK_ARCHON_NO_PREFIX_CACHING=1` → `--no-enable-prefix-caching`
|
||||
(the fork-named mitigation; the flag renders in both directions from the
|
||||
knob so a fork default change cannot silently flip the contract). Cost:
|
||||
no prefix reuse — every turn re-prefills the whole history instead of at
|
||||
@@ -388,7 +388,7 @@ conversation cannot distinguish fixed from unfixed; (2) a second
|
||||
long-context mechanism with no guard coverage — fp8 KV degradation at
|
||||
depth is the next suspect (a bf16-KV boot costs ~2× the pool per token
|
||||
and would settle it). Prefix caching is **back ON**
|
||||
(`FRANK_PRIMARY_NO_PREFIX_CACHING` commented out — the cost bought no
|
||||
(`FRANK_ARCHON_NO_PREFIX_CACHING` commented out — the cost bought no
|
||||
fix). The knob and the serve branch stay for a future fork build with a
|
||||
fixed GDN path. Next diagnostic: reproduce on a FRESH conversation; if
|
||||
it still breaks at ~40k with zero faults, A/B the KV dtype.
|
||||
@@ -472,7 +472,7 @@ running** — the bound tracks whatever the co-tenants hold at that moment. The
|
||||
demand is 105.48 GiB (52.74 GiB/rank x 2, 23,040 swap-target experts), and a
|
||||
boot where the bound drops near it silently converts next-hottest experts to
|
||||
VRAM-only (the `pinned_extra` warning is the only trace).
|
||||
`FRANK_PRIMARY_EXPERT_OFFLOAD_MEM=120` pins it: measured demand + 14% headroom,
|
||||
`FRANK_ARCHON_EXPERT_OFFLOAD_MEM=120` pins it: measured demand + 14% headroom,
|
||||
on a host with 246.1 GiB total. Semantics verified in the shipped image
|
||||
(`offloader/expert_plan.py`): the cap binds as the bound, an exceedance only
|
||||
warns, and the plan rewrites the field to the cap it met (+1 MiB/rank).
|
||||
@@ -484,7 +484,7 @@ was itself fitted at nbt 4096, which means 2048 is covered by an activation
|
||||
figure measured at a **larger** batch. SEQS 3 boots at both 1024 and 2048
|
||||
(measured 2026-10-02 — the `query_start_loc` `ConstraintViolationError` is a
|
||||
SEQS property). The costs are a one-time recompile (the compile **range** is
|
||||
part of the cache key, so `FRANK_PRIMARY_CACHE_DIR` pays once) and a wider
|
||||
part of the cache key, so `FRANK_ARCHON_CACHE_DIR` pays once) and a wider
|
||||
prefill chunk, which the 09-29 sweep found decode-interchangeable across
|
||||
1024-4096 while 512 doubled TTFT at 4064 ctx (2.46 s vs 1.36 s).
|
||||
|
||||
@@ -656,7 +656,7 @@ read them via `journalctl -o json`, not `-o cat` (and `podman logs`, which is
|
||||
only available while the container lives).
|
||||
|
||||
Rollback to the 2026-09-29 fp8+MTP4 shape: git tag
|
||||
`frank-primary-fp8-mtp-20261001` (env trio + seam re-pinned there).
|
||||
`frank-archon-fp8-mtp-20261001` (env trio + seam re-pinned there).
|
||||
|
||||
## Parallelism options on this box (2026-10-01 survey)
|
||||
|
||||
@@ -696,9 +696,9 @@ more cards.
|
||||
run in the engine's namespace and the files must be copied in first:
|
||||
|
||||
```
|
||||
podman cp /home/dank/mtp_measure.py frank-primary-qwen:/
|
||||
podman cp /home/dank/bench_iter.py frank-primary-qwen:/
|
||||
podman exec frank-primary-qwen python3 /mtp_measure.py
|
||||
podman cp /home/dank/mtp_measure.py frank-archon-qwen:/
|
||||
podman cp /home/dank/bench_iter.py frank-archon-qwen:/
|
||||
podman exec frank-archon-qwen python3 /mtp_measure.py
|
||||
```
|
||||
|
||||
Without the copy the exec dies with `can't open file '/mtp_measure.py': [Errno 2]
|
||||
@@ -717,7 +717,7 @@ reproducible across boots.
|
||||
- [[dankserver-frontier-moe-feasibility]] — the sibling feasibility page
|
||||
(which frontier MoE fits this box; the same PCIe-expert-stream wall at
|
||||
V4.1 scale)
|
||||
- [[frank-primary-r9v-radiance-qwen38-q8]] — RETIRED 2026-09-30 (the R9V
|
||||
- [[frank-archon-r9v-radiance-qwen38-q8]] — RETIRED 2026-09-30 (the R9V
|
||||
fork era; production has been the tcclaviger fork since)
|
||||
- [[x4-pair-experiment]] — the same residency law measured on a narrower
|
||||
link: streaming over PCIe loses, computing in RAM does not care
|
||||
|
||||
Reference in new issue
Block a user