Files

T

Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts

Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.

2026-10-19 05:00:43 -07:00

6.0 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

GGUF Compatibility and Runtime Lock-in

The law: a GGUF's tensor-type histogram and ftype pick the runtime — read the header before promising a file to any loader. Stock llama.cpp, Ollama and LM Studio refuse custom types at load (unknown tensor type); a fork-private type means exactly one engine can serve the file. Measured on this fleet repeatedly: Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf is ftype 115 with 311 tensors of type 103 (GGML_TYPE_Q8_0_ROCMFPX) — only charlie12345/ROCmFPX reads it. The full type tables live in Llama Cpp Runtimes and Llama Cpp Code Diff (ggml_type 100–108, ftype families 100–106 / 110–113 / 118).

1. Read the header, do not guess

Parse the GGUF KV section and histogram the tensor types plus general.architecture.file_type — one python read over the first few MiB, no model load. Counting types out of the KV section settles a runtime argument in minutes; repo names and model cards do not.

2. Compare routing variants by tensor mix, not by name

Two files of the same quant can differ in how many tensors stay at plain Q8_0 (measured: 187 vs 2 output-head tensors in an _AGENT variant of the same size). That difference is what moves spec-decode acceptance (0.953 vs 0.911) — the "same size" file is not the same file.

3. Pin the build by commit SHA

A fork that can read the file today can still crash mid-load: the 2026-08-25 ROCmFPX builds read the ftype-115 file and then SIGSEGV in ggml_compute_forward_set_rows. Record the SHA in the build script (scripts/build-rocmfpx.sh pins 36e9acd) and check it out explicitly — never a branch.

4. Header diff method (candidate vs known-good)

Answer "will engine X load this GGUF?" by diffing the candidate's header against a file the engine already loads — never by filenames or model cards; filenames lie about quantization and about MTP/vision heads.

Get the bytes: both files, from byte 0, at least 64 MiB. Tokenizer KVs (151k token strings + merges) routinely eat 5–8 MiB, so a small range desynchronises the parser. True size from curl -sIL → x-linked-size (a bare HEAD Content-Length can lie):
```
curl -sL -r 0-67108863 -o cand.bin \
  "https://huggingface.co/<owner>/<repo>/resolve/main/<file>.gguf"
ssh <host> 'head -c 67108864 /path/to/known-good.gguf' > good.bin
```
Parse — and lock the parser against the known-good file first. Layout (v2/v3): magic[4] version u32 n_tensors u64 n_kv u64, then KVs, then tensor infos. Strings are u64 length + bytes. The value-type enum is NOT llama.cpp's gguf.h: in the files seen (2026-09-28, unsloth + ajgazin UD GGUFs) type 8 = STRING, 9 = ARRAY; llama.cpp's header has STRING=2, ARRAY=3 and desynchronises immediately. Working table:
```
0 u8   1 i8   2 u16  3 i16  4 u32  5 i32  6 f32
7 bool 8 STR  9 ARRAY(u32 et, u64 n)  10 u64  11 i64  12 f64
```
Validate before trusting: the parse is right iff the tensor-info block ends somewhere sane and names look real (blk.0.attn_q.weight, token_embd.weight, output_norm.weight). EOF throw = wrong enum or offset — do not work around it.
Diff what engines actually bind on: general.architecture; tensor count; tensor name list in order; per-tensor ggml_type and dims; geometry KVs (<arch>.block_count, .embedding_length, .feed_forward_length, .attention.head_count[_kv], .rope.*, .ssm.*, .context_length, .nextn_predict_layers); MTP tensors (nextn in the name); vision tensors (visual, v_ prefixes). A same-family fine-tune quantized by the same tool usually comes out byte-identical in every one of these — that is the answer.
Then read the engine's source before calling a delta a blocker. A metadata difference only matters if the engine reads that key (grep -rn "file_type\|recurrent_layers\|full_attention_interval" <src>/). Worked example (gufo v0.1.1): general.file_type is deliberately not consulted (UD-*_K_XL ships a stale ftype 15; the tensor table rules), and qwen35.attention.recurrent_layers is never read (layer kinds come from full_attention_interval) — both deltas were non-issues the header diff alone would have flagged wrongly.
Say what you did not do. Header analysis is strong evidence, not a load. State byte sizes, check free disk before proposing a fetch, and do not download tens of GB unasked — pointing the engine at the file is the definitive test.

Report shape: verdict first (loads / will not / loads-with-caveats), then the side-by-side diff table, then only the load-bearing caveats: text-only vs vision tensors, chat-template differences (matters for llama.cpp, not for engines with compiled-in Jinja), license, disk headroom, and whether the base model is abliterated/uncensored.

5. A/B a fork's spec decode by stripping the whole flag set

--model-draft without --spec-type segfaults at load (measured 2026-10-02, core dump, no diagnostic). A baseline "no-draft" arm must remove --model-draft AND every --spec-draft-* AND --spec-type together — the flags move as a set.

6. Before deleting a retired payload, read the rollback tag's env file

A rollback may need a small companion dir: a drafter living in a directory named after an old quant looks like trash but is load-bearing. Same class as the gufo payload removal of 2026-10-03, which turned that rollback into a re-download.

Llama Cpp Runtimes — which fork ships which custom types
Llama Cpp Code Diff — the enum tables and code-level differences

kingjones-q8agent-rocmfpx — the era that served the ftype-115 file, and frank-dragoon for what runs there now

Llama Cpp Serving Sizing — pricing what a payload costs once it loads

6.0 KiB Raw Blame History Unescape Escape