serverAppConfigs/serverAppConfigs-docs/LLM/Quantization Production.md

---
type: "howto"
title: "Quantization Production"
description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file."
tags:
  - llm
  - quantization
timestamp: 2026-10-19
---

# Quantization Production — making our own quants, per card and runtime

**Research page (2026-10-08).** Question: how do WE produce quantizations
for the RTX 3090, the R9700 (gfx1201), and Strix Halo (gfx1151); what is
native to each runtime; and the state of the named prior-art families
(REAP, Heretic, abliteration, HauhauCS K_P "Perfect"). Marked facts:
**[repo-verified]** = evidenced by a file in this repo; **[verified]** =
checked first-hand against the cited primary source today; the rest are
survey claims to re-verify before acting.

## Runtime-native quant matrix

| Card | Runtime | Container | Native weight encodings | Production tooling we own |
|---|---|---|---|---|
| RTX 3090 (sm_86) | llama.cpp CUDA | GGUF | K-quants (Q4_K_M…Q8_0), I-quants (IQ1_S…IQ4_XS), ML, f16/bf16; any tensor type per-tensor | `llama-quantize` + `ggml-imatrix` (built by `scripts/build-llama-rdna4.sh` targets) **[repo-verified]** |
| R9700 (gfx1201) | radiance (StillDeadcode) | `.rad` | per-tensor encodings from a recipe: GPTQ w4 (`w4nl64a8h`), bf16, int8, fp8_e4m3, u2, MXFP4/MXFP6 (vLLM-radiance era) | full pipeline: `rad-convert` + calibration tap + `derive-protected-experts.py` + `kld-gate-swift15.sh` **[repo-verified]** — CPU host-backend, the card is irrelevant to production |
| R9700 (gfx1201) | vLLM (tcclaviger/R4D) | safetensors | native block-FP8, MXFP4/DSFP4, GPTQ/AWQ, compressed-tensors | GPTQModel/AutoRound are hardware-agnostic torch (CPU-scale); AMD decode kernels are the gate **[survey]** |
| Strix Halo (gfx1151) | halogen-flash-server (peonist, **closed**) | `.hgn` | vendor-defined (w4b era, v2 checkpoint+ngram+vision files) | **NONE public** — no `.hgn` quantizer exists; every `.hgn` on the fleet was made by peonist/Quat3rnion **[repo-verified gap]** |
| Strix Halo (gfx1151) | llama.cpp / gufo / Kairic / ROCmFPX forks | GGUF | K/I quants (gufo UD-* = imatrix mixed profiles), custom types: `Q4_0_ROCMFP4_STRIX`, `Q8_0_ROCMFPX` (type 103), Kairic IU4 | we built the RUNTIMES (`scripts/build-kairic-edge-gfx1151.sh`, ROCmFPX pin `36e9acd`), never the custom-type quantizers — those files came from agentionai/kingjones **[repo-verified]** |

- **The lock is asymmetric: llama.cpp on the 3090 is a CHOICE.** radiance
  demands `.rad`, halogen demands `.hgn` (the v2 checkpoint loads in no
  other engine); sm_86 decodes GGUF, GPTQ/AWQ-Marlin, INT8, weight-only
  FP8, and EXL3 natively — see the 3090 menu under Per-card playbooks.

Laws that steer everything here:
- **Production is hardware-agnostic; decode is not.** GPTQ/imatrix/ablation
  surgery runs in torch/CPU anywhere. The card only picks which ENCODED
  FORM it can DECODE — hence the one-table-one-encoding law, the GGUF
  custom-type histogram law ([[GGUF Compatibility]]), and the fact that
  our w4 convert ran 5 h 50 m on the host backend while the GPUs served
  nobody (`scripts/convert-swift15-w4dn8.sh`) **[repo-verified]**.
- **A custom tensor type locks the file to its fork** (ROCmFPX type 103
  precedent). K_P quants deliberately avoid that trap (below).

## The named families

### REAP — Router-weighted Expert Activation Pruning (Cerebras, 2025)
- arXiv 2510.13999; `github.com/CerebrasResearch/reap` **[verified]**.
- One-shot MoE surgery: score every expert by router gate-values ×
  activation energy, prune the low-utility tail, renumber+export a
  SMALLER MoE checkpoint (safetensors; GGUF by conversion). Published:
  GLM-4.5-Air −25 % (82B→~A12B class), GLM-4.7-REAP-268B-A32B-FP8;
  near-lossless at 25–50 % compression on generative benchmarks, and
  pruning BEATS expert-merging there **[verified from paper/README]**.
- Fleet relevance: it is the **inverse of our protected-experts law**.
  `derive-protected-experts.py` ranks experts by calibration-gram energy
  share and keeps the hottest 24/layer at bf16; REAP's saliency is the
  same statistic and deletes the coldest. REAP the Flash-Next MoE (all
  49,152 experts w4, 512/layer) by 25–50 % → resident set ~58–87 GiB →
  the expert pool fits with room for KV/cache, killing the disk tier.
  A REAPed checkpoint re-enters our normal pipeline: stage → calibrate →
  rad-convert → KLD gate. Hardware-agnostic; runs on the host.

### Heretic (`p-e-w/heretic`) — automated abliteration + layer pruning
- `github.com/p-e-w/heretic`, AGPLv3 **[verified]**. NOT a quantizer.
  Fully-automatic censorship removal: directional ablation (refusal
  direction from PCA of harmful/harmless residuals), per-layer ablation
  weights, optional LAYER DROPPING (model shrinks), auto-tuned by
  co-minimizing refusals and KL divergence from the original. Runs on
  our 3090 (torch, ~24–48 GB working set for 9–27B classes; long batch
  probes) **[verified README; runtime cost survey]**.
- Output is a normal HF checkpoint (fp16/bf16) → feeds GGUF, safetensors
  (vLLM), and `.rad` conversion. Order law: **surgery → calibrate →
  quantize**; calibrate GPTQ/imatrix on the SURGICAL output, and keep
  refusal-shaped tokens out of the calibration corpus.
- Fleet note: our abliterated bases (Quat3rnion v2, HauhauCS
  Aggressive/Balanced, ukisai Swift1.5) are this family's output; the
  Quat3rnion repo lore "the abliteration workflow appends, compaction
  drops dead padding" is exactly Heretic-style layer-editing leaving
  dead blocks (89.8 → 66.7 GiB r2) **[repo-verified lore]**.

### Ablation / abliteration (directional ablation)
- Arditi et al. 2024, arXiv 2406.11717: refusal is mediated by a single
  direction; orthogonalizing weights against it removes refusals with a
  rank-1 edit. Open implementations: `p-e-w/heretic` (automated, above),
  Artio/abliteration-toolkits, `EleutherAI/repeng`-style rep-overlap
  variants **[verified (tool existence: heretic; rest survey)]**.
- Hardware-agnostic fp-editing. On a MoE the edit touches attention/norm
  weights, so it composes with any expert quantization.
- Fleet status: frank-dragoon runs an abliterated base **by Frank's choice**
  (AGENTS: censored-base refusals were the accepted trade of the 27B era);
  frank-archon's ukisai Swift1.5 is an abliterated finetune of Qwen3.8.

### HauhauCS QX_K_P "Perfect" quants
- Our frank-zealot publisher. Method, from the publisher's own cards
  **[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced"),
  (2) quantize with **imatrix calibration on the ABLITERATED weights**,
  (3) K_P = a **per-model, per-tensor mixed-precision profile** —
  "model-specific analysis to selectively preserve quality where it
  matters most", e.g. Q8_K_P = a census of Q8_K + F32 + F16 tensors
  (FriendliAI's safetensor census of a Q8_K_P shows exactly that
  399/353/99 tensor split). Result: +1–2 quality rungs for +5–15 %
  size. **It is a standard GGUF** — llama-quantize with per-tensor type
  overrides (`--tensor-type`), imatrix on; no fork, no custom types
  (LM Studio just fails to NAME the mixed file, shows "?").
- Reproducible with stock tooling: `ggml-imatrix` on the target model +
  `llama-quantize --tensor-type <pattern>=<type>` per the model's
  sensitivity census. Both binaries already build from
  `scripts/build-llama-rdna4.sh` **[repo-verified]**.
- Our own ladder precedent: the fleet's UD-Q8_K_XL (unsloth "Dynamic" =
  same family: imatrix + importance-tiered per-tensor types) and
  agentionai FAST imatrix ROCmFP4-FAST (we ran the imatrix step
  ourselves on the v2 joined-PLE file — the one self-made quant in repo
  history) **[repo-verified]**.

## Per-card playbooks

### RTX 3090 — the unlocked card (frank-zealot runs llama.cpp BY CHOICE)
The 3090 is the only fleet device with no format lock: radiance needs
`.rad`, halogen needs `.hgn`, and the v2 checkpoint loads in NO other
engine. sm_86 + CUDA is the industry's default target, so the runtime
menu is open and every self-produced artifact format has a native
decoder here. This makes the 3090 the natural **quant R&D bench**:
produce on the host, DECODE AND SCORE on the 3090 against the fleet's
own KLD/perplexity gates.

Runtime menu on sm_86 (verified against each project's support matrix):

| Runtime | Native quant on Ampere | Notes |
|---|---|---|
| llama.cpp CUDA (deployed) | K/I/ML quants, f16 | imatrix + `--tensor-type` K_P ladder |
| vLLM CUDA | GPTQ INT4 + AWQ via **Marlin** (sm_80/86 native), INT8 W8A8 SmoothQuant, **FP8 as W8A16 weight-only via Marlin** (FP8 W8A8 compute is Ada/Hopper-only) | full OpenAI server, batching |
| SGLang | GPTQ/AWQ (vLLM kernel lineage); Ampere FP8-via-Marlin tracked upstream | CUDA-first engine |
| ExLlamaV3 | **EXL3** trellis (QTIP) 2–8 bpw; runs on Ampere incl. the 3090 | the quantizer (`convert.py`) runs on this same card |
| TensorRT-LLM | INT4 AWQ/GPTQ (FP16 acts), INT8 SmoothQuant; FP8 and INT4-AWQ-with-FP8-acts need SM≥89, NVFP4 is Blackwell | engine-build workflow |

Hardware ceilings that no software unlocks on this card: no FP8 matmul
hardware (sm_86 < 89), no FP4 (Blackwell). Ampere's fast lanes are
**INT4 (Marlin/AWQ/GPTQ), INT8 (SmoothQuant), weight-only FP8, f16**.

Playbook A — GGUF ladder (status quo, llama.cpp kept):
1. Surgery on host/3090: abliterate/prune (Heretic) — torch.
2. `ggml-imatrix` (CUDA build) over a calibration corpus (our
   `/home/dank/corpora` style set; ≥1M tokens; the fleet's KLD corpus
   is a reasonable seed).
3. `llama-quantize` F16 → target rung; K_P-style `--tensor-type`
   overrides for attention/KV/first-layer upgrades; `--use-repack` for
   IQ rungs.
4. Validate: perplexity + KLD vs BF16 + the K-quant rank checks.

Playbook B — safetensors ladder (the unlocked option): GPTQModel /
AutoRound / AWQ run CPU-side, emit GPTQ/AWQ/FP8 checkpoints that vLLM
or SGLang serves on the 3090 with Marlin kernels — same KLD-gate
discipline as the radiance pipeline. A REAPed + abliterated Flash-Next
cut to GPTQ-w4 for the 3090 is a bench experiment that needs no new
hardware; EXL3 (`convert.py` on the 3090, exllama serve) is the
quality-per-bit extreme of the same menu.
Caveats before any swap: the serving contract (OpenAI server,
`reasoning_content`, vision mmproj, and the compaction-hub window the endpoint
declares in its Current shape — [[frank-zealot]]) must be met by the challenger
runtime, and the 24 GB card runs ONE server — benching follows the stop-and-mask
law.

#### Worked example — the frank-archon w4dn8 recipe ported to the 3090
(audit 2026-10-08, target `Qwen3.6-35B-A3B`; sized at that date's hub shape —
the window history is [[frank-zealot]]'s Eras table)
**The methodology ports; the encoding must be translated.** llama.cpp
decodes no GPTQ — the recipe re-expresses on GGUF-native terms:
imatrix on our corpora (replaces the radiance grams; runs the 65 GiB
BF16 on host CPU), IQ4-class experts, hot-expert protection as
`--tensor-type "=f16"` patterns over the derive-the-hot-experts law
(statistic: imatrix importances), Q6_K trunk, KLD/perplexity gate at
the deployed slot shape. Production stays host-side; the 3090 is only
the decode target.
KV is cheap on this arch: 10 full layers × 2 KV heads × 256 =
**20 KiB/token f16** (3.8 GiB per 200k slot); DeltaNet state is
per-slot, tens of MiB. Size verdict (weights ≈ 32.2 B experts + 3 B
trunk):
- Archon-faithful (w4 experts + bf16 protected + i8 trunk) ≈ 22 GiB →
  27 GiB with KV+buffers → **over 24; that shape is 2×3090-class.**
- Fits 1×200,704 text-only: IQ4_XS experts (17.4) + Q6_K trunk + q8_0
  KV (1.9; `GGML_CUDA_FA_ALL_QUANTS=ON` build) ≈ 21.5 ✓
- Compaction-hub shape (2×200,704 + mmproj): experts at IQ3_M (14.4) ≈
  24.0 — the exact Swift IQ3_XXS 2-slot edge, MTP precedent exists.
- Literal GPTQ-w4 encoding decodes only via vLLM/SGLang Marlin (Playbook
  B); compressed-tensors expresses the protected set per-module.
Shelf check: HauhauCS publishes this model's K_P ladder (IQ2_M 10.86,
IQ4_XS 17.44, Q4_K_M 19.71, Q4_K_P 21.82, Q5_K_P 26.1, Q6_K_P 28.5,
Q8_K_P 40.6 GiB + mmproj 0.84) — Q4_K_P IS the protected-tensor blend.
The self-made delta: calibration on OUR corpora, our hot-expert law,
and a KLD gate scored at the serving shape.

### R9700 — frank-archon (radiance)
The full self-quant pipeline already exists end to end: fetch BF16 →
record KLD reference (1.0.13) → capture calibration grams (1.2.0 tap,
`RADIANCE_CALIB_DIR/TOKENS`) → derive protected experts (energy-share) →
`convert-swift15-w4dn8.sh` (1.0.13, host backend) → KLD gate (MNBT 2048
law, pool 40960, mean KL ≤ 0.03, top-1 ≥ 95 %). New model = new recipe
+ new `.so` arch plugin; every step is scripted. Prior art to fold in:
REAP the expert set BEFORE step 3; ablate BEFORE calibration.
Native encodings beyond w4: i8 trunk, fp8 HC/PLE/KV, bf16 protected,
u2 draft — all recipe-expressible (see the w4dn8 runbook).

#### Recipe portability: the w4dn8 shape onto other MoEs (2026-10-08 audit)
The pin 1.0.13 container ships SIX arch plugins
(`/opt/radiance/share/radiance/architectures/`): `qwen4exp_fp8`,
**`qwen35moe_fp8`**, `qwen35_fp8`, `qwen35_bf16`, `llama_fp8`,
`llama_dense_fp8` (census via `podman run --rm --entrypoint sh <image> -c`
— the image has no `ls`/`strings`; grep the `.so` files directly, and
the image lives in ROOTLESS storage as `docker.io/stilldeadcode/radiance`).
`grep -c` receipts: `qwen35moe_fp8.so` carries the `RADIANCE_CALIB` tap
(2) and gram export (2) plus MTP (4) and NO ngram (0) — i.e. the full
calibration→GPTQ path exists for the qwen3.5-MoE family; GPTQ itself
lives in `rad-convert`, not the plugin (0 hits in both `.so`).

Worked example — `Qwen/Qwen3.6-35B-A3B` (config: `qwen3_5_moe_text`,
40 layers, `full_attention_interval=4` DeltaNet hybrid, **256 experts
top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
248320, multimodal wrapper with vision tower):
- Expert set ~32 B params → w4 ≈ 16 GiB; trunk (attn/DeltaNet/embed/
  lm_head ~3 B) at i8 ≈ 3 GiB ⇒ **~20 GiB total — fully VRAM-resident
  on ONE R9700** (32 GiB card): no TP=2, no host pool, no disk tier,
  no `--ngram-placement`, no 47.7 GiB PLE copy, no HC term. The recipe
  loses three lines (PLE/HC/protected-count scales with 256 experts).
- Convert cost scales with the file: Flash-Next's 335 GiB took 5 h 50 m
  host-backend ⇒ ~70 GiB BF16 ≈ 1–1.5 h. KLD reference boot is trivial
  (BF16 ~65 GiB in host pool beside the pool KV needs).
- Unresolved before any real campaign: does `qwen35moe` decode the
  `Qwen3_5MoeForConditionalGeneration` vision wrapper (assume text-only
  until proven), and does its MTP cast to the fp8/u2 encodings like
  qwen4exp. The MNBT-2048 disk-tier law is MOOT for a resident model.

### Strix Halo — frank-dragoon (halogen) and friends
- `.hgn`: **closed production door.** No public quantizer exists
  (verified absence across peonist-ai repos); the format's lock is the
  engine's lock. We consume vendor artifacts; the repo-side tooling is
  compaction/verification only.
- Self-quant work on this card therefore runs through **GGUF runtimes**:
  the same imatrix/llama-quantize flow as the 3090 (HIP/Vulkan decode of
  K/I quants, gufo UD-* precedent, `--use-repack` off on ROCm), or the
  custom-type forks (ROCmFP4 STRIX, Kairic IU4) whose PRODUCERS we have
  never been — the quantizers for types 101/103 ship in those forks'
  trees; building one is a fork build, not a flag.
- Native-FP4 note: RDNA4/Strix get hardware FP4 matmuls (hipBLASLt
  MXFP4/DSFP4), which is the engine-level path (radiance/vLLM), not a
  GGUF decode.

## Sources
- REAP: arXiv 2510.13999, github.com/CerebrasResearch/reap,
  cerebras/GLM-4.5-Air-REAP-82B-A12B-FP8, cerebras/GLM-4.7-REAP-268B-A32B-FP8.
- Heretic: github.com/p-e-w/heretic (AGPLv3).
- Abliteration: arXiv 2406.11717 (Arditi et al.).
- K_P: HauhauCS model cards (e.g. `HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive`
  README "What are K_P quants?"), FriendliAI Q8_K_P tensor census,
  fleet payload tree (BF16/Q4_K_M/Q6_K/Q8_0 + mmproj ladder in
  `HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive`).
- EXL3: github.com/turboderp-org/exllamav3 (QTIP lineage; CUDA; Ampere
  decode incl. 3090 documented in v1.0 performance write-ups).
- Ampere capability edges: vLLM FP8 docs ("only Hopper and Ada …
  W8A8; Ampere supported for W8A16 weight-only utilizing Marlin"),
  vLLM Supported-Hardware matrix (Marlin GPTQ/AWQ on SM 8.0/8.6),
  TensorRT-LLM Support Matrix (FP8 and INT4-AWQ-with-FP8-acts need
  SM≥89; NVFP4 Blackwell), SGLang issue #12887 (Ampere FP8-via-Marlin).
- Repo: `scripts/convert-swift15-w4dn8.sh`,
  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
  `scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
  `scripts/build-kairic-edge-gfx1151.sh`; endpoints [[frank-archon]],
  [[frank-dragoon]], [[frank-zealot]]; method [[qwen3.8-strix-optimizations]]
  and [[GGUF Compatibility]].