frank-primary: cutover measured — docs reflect the live service
AGENTS.md: row + section carry the measured 177,109 MiB container, floor 177000, live cutover 2026-10-06. Runbook: the three convert laws the live runs proved (scale-name, file-shadow, symlink-through), the measured boot (125.26 GiB page-in 38.1 s, PLE pin 47.68 GiB/43.9 s, ~2 min to health), the MTP/vision/tool/reasoning contract results, and the SSD tier fix.
This commit is contained in:
@@ -38,7 +38,8 @@ same served alias (`frank-primary`), same container identity
|
||||
all 131 shards sha256-match the Hub LFS oids (2026-10-05, log
|
||||
`/home/dank/logs/verify-qwen38-official-fp8.log`).
|
||||
- **The container**: `/home/dank/models/rad/qwen4exp-fp8-native.rad`
|
||||
(~161 GiB; floor `FRANK_PRIMARY_RAD_MIN_MIB=158000` in the env gates a
|
||||
(177,109 MiB measured = 173.0 GiB; floor
|
||||
`FRANK_PRIMARY_RAD_MIN_MIB=177000` in the env gates a
|
||||
truncated convert). Built by `scripts/build-frank-primary-rad.sh`:
|
||||
1. stage `/home/dank/models/stage-qwen4exp-fp8` — `config.json` with
|
||||
**`quantization_config` dropped** + relative symlinks into the raw
|
||||
@@ -62,7 +63,26 @@ same served alias (`frank-primary`), same container identity
|
||||
exact round-trip (nearest-code distance 2⁻³ vs the bf16 rounding
|
||||
perturbation 2⁻⁹): the container gets the checkpoint's own PLE codes
|
||||
plus the one-element scale plane (`blk.1.ple_ngram.scale`, view {1});
|
||||
4. `rad-info` summary + size floor.
|
||||
4. THREE LAWS the live runs proved (each cost a full verify+declare
|
||||
cycle; all fixed in the two scripts, 2026-10-05/06):
|
||||
- **Scale-name law**: `scale_name()` appends `_scale_inv` to the
|
||||
TENSOR NAME verbatim — a stacked tensor named
|
||||
`…experts.gate_up_proj` (no `.weight` suffix) has its scale plane
|
||||
at `…experts.gate_up_proj_scale_inv`, NOT `…weight_scale_inv`.
|
||||
- **File-shadow law**: the reader opens every index-named file in
|
||||
SORTED order and emplaces EVERY tensor each file physically
|
||||
holds, first-wins (`checkpoint.cpp add_shard`). The raw
|
||||
model-000NN shards carry tensors NAMED exactly what the merged
|
||||
PLE shard is named, so the merged file MUST sort first
|
||||
(`00-ple.merged.safetensors`) or a raw fp8 shard wins the name
|
||||
and the plan reads one shard as the codes.
|
||||
- **Symlink-through law**: staging symlinks the raw index into the
|
||||
stage; `open(path,"w")` writes THROUGH it and corrupts the raw
|
||||
checkpoint (raw verify then fails 143/144). The stack script
|
||||
writes index/config temp+`os.replace` — an atomic swap replaces
|
||||
the symlink itself; the build script COPIES the index and
|
||||
symlinks only the shards.
|
||||
5. `rad-info` summary + size floor.
|
||||
- **Why the staging dir is mandatory** (the descriptor law, spec §2.4):
|
||||
the plugin is selected by the exact `(arch_id, quant descriptor)` pair
|
||||
with no fallback. A raw official checkpoint carries
|
||||
@@ -122,8 +142,10 @@ KV from the `0.82`-side remainder of claimed VRAM at 96 MiB headroom.
|
||||
while serving, 503 only while stopping — **the listener binds AFTER the
|
||||
full load**, so a bare `curl -fsS http://127.0.0.1:7770/health` gate is
|
||||
honest (the deployer loops on it; the FreeToken-era `"status":"ok"` body
|
||||
grep is gone). Boot budget 1800 s (publisher `start_period` 15 m); the
|
||||
big terms are the ~161 GiB page-in and the host-pool pin.
|
||||
grep is gone). Boot budget 1800 s (publisher `start_period` 15 m);
|
||||
measured 2026-10-06: health 200 ~2-3 min after start (~120 s warm) —
|
||||
the big terms are the 125.26 GiB page-in (38.1 s at 3.5 GB/s) and the
|
||||
host-pool pin (PLE 47.68 GiB in 43.9 s).
|
||||
- **Sampling** (oai.cpp): `temperature, top_p, top_k, min_p, typical_p,
|
||||
presence_penalty, frequency_penalty, repetition_penalty, repeat_last_n,
|
||||
chat_template_kwargs, reasoning_effort, enable_thinking,
|
||||
@@ -174,6 +196,39 @@ Post-cutover checklist (each is observable, not assumed):
|
||||
`image_url` content part.
|
||||
6. Prometheus target `frank-primary` green (not never-green).
|
||||
|
||||
## Cutover measured (2026-10-06, all values observed live)
|
||||
|
||||
- Container: 1,857,127,997,440 B = 177,109 MiB = 173.0 GiB. Plan-only
|
||||
priced 172.76 GiB — the DATA planes; the written container adds a
|
||||
metadata tail. Floor gates must be set from the real file (the first
|
||||
deploy refused on 178000 > 177109; floor is 177000).
|
||||
- Boot: 125.26 GiB read in 38.1 s (3.5 GB/s), PLE pinned
|
||||
"47.68 GiB in 43.9 s" (the 2 B scale plane with it), each rank's
|
||||
mover holds 25.46 GiB static+experts (19.15 GiB resident experts,
|
||||
79 %), 16,820 pinned pool slots, health 200 ~2-3 min after start
|
||||
(~120 s warm). No 7-minute vLLM-style boot.
|
||||
- KV ledger: kv_attn 99,956 blocks = 2.48 GiB (399,824 tokens),
|
||||
kv_qsa_bkey 317.24 MiB, kv_gdn_state 4 slots × 54 MiB, PLE conv
|
||||
states < 1 MiB; linear checkpoints 56.37 MiB × 8 slots (3 of 8
|
||||
card-backed at boot, 5 withdrawn under the 208 MiB floor).
|
||||
- MTP on real traffic (temp 0.7, 220-token ask): draft_n 225 /
|
||||
draft_n_accepted 119 = 52.9 % acceptance, decode 23.8 t/s. ONE
|
||||
sample — the shape's real number needs live-traffic measurement.
|
||||
- Contracts: `reasoning_content` field (matches models.yml), usage
|
||||
metering separates reasoning_tokens; vision — a 512² solid-red PNG
|
||||
through `image_url` answers `Red` (279 prompt tokens; NOTE the
|
||||
32-token budget is eaten by thinking — give vision probes ≥256
|
||||
max_tokens); native `tool_calls` with finish_reason tool_calls.
|
||||
- Boot defects fixed the same day: the SSD prefix tier opened only
|
||||
after the serve script mkdir -p'd the full path (the engine's
|
||||
create is NOT recursive: "cannot create /data/kvcache/..."); the
|
||||
converter's scale-name law, the PLE file-shadow and the
|
||||
symlink-through-raw-index bug are in the payload section above.
|
||||
- Prometheus: `/metrics` answers 200 with vLLM-compatible families
|
||||
(verified from Dankclaw); the `frank-primary` job needs a
|
||||
prometheus-service deploy to go green (config pushed, reload
|
||||
deferred behind a live parallel writer's dirty tree).
|
||||
|
||||
## Tuning levers (in order of likely use)
|
||||
|
||||
- **`FRANK_PRIMARY_MAX_SEQS`**: raise 2 → 3/4 only when the ledger's KV
|
||||
|
||||
Reference in New Issue
Block a user