docs(LLM): 3090 worked example — archon w4dn8 methodology ports to GGUF-native quants; 24 GiB size law for 35B-A3B

This commit is contained in:
Frank
2026-10-08 23:44:13 -07:00
parent 757a90057f
commit f99b698d25
@@ -151,6 +151,34 @@ Caveats before any swap: the serving contract (OpenAI server,
200,704 ×2 slots) must be met by the challenger runtime, and the
24 GB card runs ONE server — benching follows the stop-and-mask law.
#### Worked example — the frank-archon w4dn8 recipe ported to the 3090
(audit 2026-10-08, target `Qwen3.6-35B-A3B`)
**The methodology ports; the encoding must be translated.** llama.cpp
decodes no GPTQ — the recipe re-expresses on GGUF-native terms:
imatrix on our corpora (replaces the radiance grams; runs the 65 GiB
BF16 on host CPU), IQ4-class experts, hot-expert protection as
`--tensor-type "=f16"` patterns over the derive-the-hot-experts law
(statistic: imatrix importances), Q6_K trunk, KLD/perplexity gate at
the deployed slot shape. Production stays host-side; the 3090 is only
the decode target.
KV is cheap on this arch: 10 full layers × 2 KV heads × 256 =
**20 KiB/token f16** (3.8 GiB per 200k slot); DeltaNet state is
per-slot, tens of MiB. Size verdict (weights ≈ 32.2 B experts + 3 B
trunk):
- Archon-faithful (w4 experts + bf16 protected + i8 trunk) ≈ 22 GiB →
27 GiB with KV+buffers → **over 24; that shape is 2×3090-class.**
- Fits 1×200,704 text-only: IQ4_XS experts (17.4) + Q6_K trunk + q8_0
KV (1.9; `GGML_CUDA_FA_ALL_QUANTS=ON` build) ≈ 21.5 ✓
- Compaction-hub shape (2×200,704 + mmproj): experts at IQ3_M (14.4) ≈
24.0 — the exact Swift IQ3_XXS 2-slot edge, MTP precedent exists.
- Literal GPTQ-w4 encoding decodes only via vLLM/SGLang Marlin (Playbook
B); compressed-tensors expresses the protected set per-module.
Shelf check: HauhauCS publishes this model's K_P ladder (IQ2_M 10.86,
IQ4_XS 17.44, Q4_K_M 19.71, Q4_K_P 21.82, Q5_K_P 26.1, Q6_K_P 28.5,
Q8_K_P 40.6 GiB + mmproj 0.84) — Q4_K_P IS the protected-tensor blend.
The self-made delta: calibration on OUR corpora, our hot-expert law,
and a KLD gate scored at the serving shape.
### R9700 — frank-primary (radiance)
The full self-quant pipeline already exists end to end: fetch BF16 →
record KLD reference (1.0.13) → capture calibration grams (1.2.0 tap,