docs: frank-primary runs FreeToken (PR587 ROCm) — runbook, index, AGENTS, log
New runbook frank-primary-freetoken-pr587 (build laws, shape, measured cutover numbers, TP=2 non-boot signature, rollback); LLM/index.md intro + mermaid + active/retired entries; AGENTS.md fleet row + section + bench and model-config mentions + runbook index; log.md entry. Seam pins the measured KV pool 299968 with the multiple-of-64 guard. NOTE: intended PGP signature skipped — key 0x76EA601A11B178D7 has no secret key in this machine's keyring (fresh ~/.gnupg); the PGP identity lives elsewhere.
This commit is contained in:
1 parent
6ba19f6f48
commit
e2308dff71
5 files changed
+295
-139
No files matched your search
@@ -99,13 +99,17 @@ DOCKERFILE="$REPO_ROOT/scripts/docker/frank-primary-freetoken.Dockerfile"
|
||||
@test "the FreeToken engine shape is in the env file" {
|
||||
# KV pool: TOKENS, not bytes. 24 KiB/token (12 full-attention layers x
|
||||
# 2 KV heads x 256 head_dim x K+V x f16) — the 36 GDN layers use fixed
|
||||
# per-seq state, not per-token KV. 300,000 tokens = 7.2 GiB and clears
|
||||
# per-seq state, not per-token KV. 299,968 tokens = 7.0 GiB and clears
|
||||
# the harness worst case (140,492-token prompt + 36,864 max_tokens =
|
||||
# 177,356); one NATIVE 262,144-token conversation + full budget also
|
||||
# fits. The arithmetic guard keeps future resizes honest.
|
||||
grep -q '^FRANK_PRIMARY_FT_NUM_TOKENS=300000$' "$ENV_FILE"
|
||||
# fits. The value must stay a MULTIPLE OF 64: the auto attention
|
||||
# backend is qsa_sparse, which overrides page size to 64 (measured
|
||||
# first boot — 300,000 was refused). The arithmetic guard keeps future
|
||||
# resizes honest.
|
||||
grep -q '^FRANK_PRIMARY_FT_NUM_TOKENS=299968$' "$ENV_FILE"
|
||||
nt=$(awk -F= '/^FRANK_PRIMARY_FT_NUM_TOKENS=/{print $2}' "$ENV_FILE")
|
||||
[ "$nt" -ge 177356 ]
|
||||
[ $((nt % 64)) -eq 0 ]
|
||||
# First-boot conservative shape: 2 decode slots, 10% expert LRU
|
||||
# (~12 GiB of the ~120 GiB fp8 expert set), output ceiling = the
|
||||
# harness maxTokens, single R9700 at index 0.
|
||||
|
||||
Reference in new issue
Block a user