--- type: reference title: GGUF Compatibility and Runtime Lock-in description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can." tags: [gguf, llama.cpp, forks, quantization, speculative] timestamp: 2026-10-05T00:00:00-07:00 --- # GGUF Compatibility and Runtime Lock-in **The law: a GGUF's tensor-type histogram and ftype pick the runtime — read the header before promising a file to any loader.** Stock llama.cpp, Ollama and LM Studio refuse custom types at load (`unknown tensor type`); a fork-private type means exactly one engine can serve the file. Measured on this fleet repeatedly: `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is ftype 115 with 311 tensors of type 103 (`GGML_TYPE_Q8_0_ROCMFPX`) — only charlie12345/ROCmFPX reads it. The full type tables live in and (ggml_type 100–108, ftype families 100–106 / 110–113 / 118). ## 1. Read the header, do not guess Parse the GGUF KV section and histogram the tensor types plus `general.architecture.file_type` — one python read over the first few MiB, no model load. Counting types out of the KV section settles a runtime argument in minutes; repo names and model cards do not. ## 2. Compare routing variants by tensor mix, not by name Two files of the same quant can differ in how many tensors stay at plain Q8_0 (measured: 187 vs 2 output-head tensors in an `_AGENT` variant of the same size). That difference is what moves spec-decode acceptance (0.953 vs 0.911) — the "same size" file is not the same file. ## 3. Pin the build by commit SHA A fork that can read the file today can still crash mid-load: the 2026-08-25 ROCmFPX builds read the ftype-115 file and then SIGSEGV in `ggml_compute_forward_set_rows`. Record the SHA in the build script (`scripts/build-rocmfpx.sh` pins `36e9acd`) and check it out explicitly — never a branch. ## 4. Header diff method (candidate vs known-good) Answer "will engine X load this GGUF?" by diffing the candidate's header against a file the engine **already loads** — never by filenames or model cards; filenames lie about quantization and about MTP/vision heads. 1. **Get the bytes**: both files, from byte 0, at least 64 MiB. Tokenizer KVs (151k token strings + merges) routinely eat 5–8 MiB, so a small range desynchronises the parser. True size from `curl -sIL` → `x-linked-size` (a bare HEAD `Content-Length` can lie): ```sh curl -sL -r 0-67108863 -o cand.bin \ "https://huggingface.co///resolve/main/.gguf" ssh 'head -c 67108864 /path/to/known-good.gguf' > good.bin ``` 2. **Parse — and lock the parser against the known-good file first.** Layout (v2/v3): `magic[4]` `version u32` `n_tensors u64` `n_kv u64`, then KVs, then tensor infos. Strings are u64 length + bytes. **The value-type enum is NOT llama.cpp's `gguf.h`:** in the files seen (2026-09-28, unsloth + ajgazin UD GGUFs) type **8 = STRING, 9 = ARRAY**; llama.cpp's header has STRING=2, ARRAY=3 and desynchronises immediately. Working table: ``` 0 u8 1 i8 2 u16 3 i16 4 u32 5 i32 6 f32 7 bool 8 STR 9 ARRAY(u32 et, u64 n) 10 u64 11 i64 12 f64 ``` Validate before trusting: the parse is right iff the tensor-info block ends somewhere sane and names look real (`blk.0.attn_q.weight` `token_embd.weight`, `output_norm.weight`). EOF throw = wrong enum or offset — do not work around it. 3. **Diff what engines actually bind on**: `general.architecture`; tensor count; tensor name list in order; per-tensor `ggml_type` and dims; geometry KVs (`.block_count`, `.embedding_length`, `.feed_forward_length` `.attention.head_count[_kv]`, `.rope.*`, `.ssm.*`, `.context_length` `.nextn_predict_layers`); MTP tensors (`nextn` in the name); vision tensors (`visual`, `v_` prefixes). A same-family fine-tune quantized by the same tool usually comes out byte-identical in every one of these — that is the answer. 4. **Then read the engine's source before calling a delta a blocker.** A metadata difference only matters if the engine reads that key (`grep -rn "file_type\|recurrent_layers\|full_attention_interval" /`). Worked example (gufo v0.1.1): `general.file_type` is deliberately not consulted (UD-*_K_XL ships a stale ftype 15; the tensor table rules), and `qwen35.attention.recurrent_layers` is never read (layer kinds come from `full_attention_interval`) — both deltas were non-issues the header diff alone would have flagged wrongly. 5. **Say what you did not do.** Header analysis is strong evidence, not a load. State byte sizes, check free disk before proposing a fetch, and do not download tens of GB unasked — pointing the engine at the file is the definitive test. **Report shape:** verdict first (loads / will not / loads-with-caveats), then the side-by-side diff table, then only the load-bearing caveats: text-only vs vision tensors, chat-template differences (matters for llama.cpp, not for engines with compiled-in Jinja), license, disk headroom, and whether the base model is abliterated/uncensored. ## 5. A/B a fork's spec decode by stripping the whole flag set `--model-draft` without `--spec-type` segfaults at load (measured 2026-10-02, core dump, no diagnostic). A baseline "no-draft" arm must remove `--model-draft` AND every `--spec-draft-*` AND `--spec-type` together — the flags move as a set. ## 6. Before deleting a retired payload, read the rollback tag's env file A rollback may need a small companion dir: a drafter living in a directory named after an old quant looks like trash but is load-bearing. Same class as the gufo payload removal of 2026-10-03, which turned that rollback into a re-download. ## Related - which fork ships which custom types - the enum tables and code-level differences * the era that served the ftype-115 file, and [[frank-dragoon]] for what runs there now - [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads