Files

T

Frank 219a097b39 docs(vault): one page per endpoint, forever -- 45 era pages fold into three

Every payload swap a live endpoint has ever run is now one row in that
endpoint's Eras table, with its rollback path in the same row. No per-attempt
page, no per-swap ledger, nowhere else.

- Added: LLM/frank-archon.md (radiance, 2x R9700 TP=2), LLM/frank-dragoon.md
  (halogen flash-server, 19 eras), LLM/frank-zealot.md (llama.cpp PrismML fork
  on the 3090). Each carries Current shape, Eras, Laws, wire contract, files.
- Deleted: 45 per-payload runbooks -- the frank-dragoon line (11 including the
  pre-name halogen/kingjones/kairic/CIRU/eemin/agentionai/dflash2 pages and the
  benchmark page), the frank-zealot line (llama.cpp/vLLM/ik_llama/SGLang eras),
  the frank-archon line (vLLM/FreeToken/R9V/llama.cpp/radiance eras), plus
  frank-medium, frank-npu and the ubatch/PCIe one-offs that owned nothing.
- Rewired: every wikilink and markdown-path reference to a folded page points
  at the endpoint page instead; the three Research see-also lists and the
  ubatch sweep's runbook link included. No anchored links existed in the set.
- Regenerated both index pages as pure locators -- one row, one clause, no
  numbers -- so the docs_completeness locator law holds for all 153 pages.

tests/docs_completeness.bats green: index links every page, no unresolved
links. The pages moved to .AGENTTRASH/20261018-doc-dedupe/LLM/ (gitignored,
and git history holds every byte).

2026-10-19 00:49:03 -07:00

6.0 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

GGUF Compatibility and Runtime Lock-in

The law: a GGUF's tensor-type histogram and ftype pick the runtime — read the header before promising a file to any loader. Stock llama.cpp, Ollama and LM Studio refuse custom types at load (unknown tensor type); a fork-private type means exactly one engine can serve the file. Measured on this fleet repeatedly: Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf is ftype 115 with 311 tensors of type 103 (GGML_TYPE_Q8_0_ROCMFPX) — only charlie12345/ROCmFPX reads it. The full type tables live in Llama Cpp Runtimes and Llama Cpp Code Diff (ggml_type 100–108, ftype families 100–106 / 110–113 / 118).

1. Read the header, do not guess

Parse the GGUF KV section and histogram the tensor types plus general.architecture.file_type — one python read over the first few MiB, no model load. Counting types out of the KV section settles a runtime argument in minutes; repo names and model cards do not.

2. Compare routing variants by tensor mix, not by name

Two files of the same quant can differ in how many tensors stay at plain Q8_0 (measured: 187 vs 2 output-head tensors in an _AGENT variant of the same size). That difference is what moves spec-decode acceptance (0.953 vs 0.911) — the "same size" file is not the same file.

3. Pin the build by commit SHA

A fork that can read the file today can still crash mid-load: the 2026-08-25 ROCmFPX builds read the ftype-115 file and then SIGSEGV in ggml_compute_forward_set_rows. Record the SHA in the build script (scripts/build-rocmfpx.sh pins 36e9acd) and check it out explicitly — never a branch.

4. Header diff method (candidate vs known-good)

Answer "will engine X load this GGUF?" by diffing the candidate's header against a file the engine already loads — never by filenames or model cards; filenames lie about quantization and about MTP/vision heads.

Get the bytes: both files, from byte 0, at least 64 MiB. Tokenizer KVs (151k token strings + merges) routinely eat 5–8 MiB, so a small range desynchronises the parser. True size from curl -sIL → x-linked-size (a bare HEAD Content-Length can lie):
```
curl -sL -r 0-67108863 -o cand.bin \
  "https://huggingface.co/<owner>/<repo>/resolve/main/<file>.gguf"
ssh <host> 'head -c 67108864 /path/to/known-good.gguf' > good.bin
```
Parse — and lock the parser against the known-good file first. Layout (v2/v3): magic[4] version u32 n_tensors u64 n_kv u64, then KVs, then tensor infos. Strings are u64 length + bytes. The value-type enum is NOT llama.cpp's gguf.h: in the files seen (2026-09-28, unsloth + ajgazin UD GGUFs) type 8 = STRING, 9 = ARRAY; llama.cpp's header has STRING=2, ARRAY=3 and desynchronises immediately. Working table:
```
0 u8   1 i8   2 u16  3 i16  4 u32  5 i32  6 f32
7 bool 8 STR  9 ARRAY(u32 et, u64 n)  10 u64  11 i64  12 f64
```
Validate before trusting: the parse is right iff the tensor-info block ends somewhere sane and names look real (blk.0.attn_q.weight, token_embd.weight, output_norm.weight). EOF throw = wrong enum or offset — do not work around it.
Diff what engines actually bind on: general.architecture; tensor count; tensor name list in order; per-tensor ggml_type and dims; geometry KVs (<arch>.block_count, .embedding_length, .feed_forward_length, .attention.head_count[_kv], .rope.*, .ssm.*, .context_length, .nextn_predict_layers); MTP tensors (nextn in the name); vision tensors (visual, v_ prefixes). A same-family fine-tune quantized by the same tool usually comes out byte-identical in every one of these — that is the answer.
Then read the engine's source before calling a delta a blocker. A metadata difference only matters if the engine reads that key (grep -rn "file_type\|recurrent_layers\|full_attention_interval" <src>/). Worked example (gufo v0.1.1): general.file_type is deliberately not consulted (UD-*_K_XL ships a stale ftype 15; the tensor table rules), and qwen35.attention.recurrent_layers is never read (layer kinds come from full_attention_interval) — both deltas were non-issues the header diff alone would have flagged wrongly.
Say what you did not do. Header analysis is strong evidence, not a load. State byte sizes, check free disk before proposing a fetch, and do not download tens of GB unasked — pointing the engine at the file is the definitive test.

Report shape: verdict first (loads / will not / loads-with-caveats), then the side-by-side diff table, then only the load-bearing caveats: text-only vs vision tensors, chat-template differences (matters for llama.cpp, not for engines with compiled-in Jinja), license, disk headroom, and whether the base model is abliterated/uncensored.

5. A/B a fork's spec decode by stripping the whole flag set

--model-draft without --spec-type segfaults at load (measured 2026-10-02, core dump, no diagnostic). A baseline "no-draft" arm must remove --model-draft AND every --spec-draft-* AND --spec-type together — the flags move as a set.

6. Before deleting a retired payload, read the rollback tag's env file

A rollback may need a small companion dir: a drafter living in a directory named after an old quant looks like trash but is load-bearing. Same class as the gufo payload removal of 2026-10-03, which turned that rollback into a re-download.

Llama Cpp Runtimes — which fork ships which custom types
Llama Cpp Code Diff — the enum tables and code-level differences
frank-dragoon — the ftype-115 lock-in in production
Llama Cpp Serving Sizing — pricing what a payload costs once it loads

6.0 KiB Raw Blame History Unescape Escape