docs(vault): fleet-wide fact re-verification pass (67 pages)

The docs-fact census wave re-read the live hosts and re-keyed every page
it could: runtime/payload identities, ports, sizes, ledger figures, unit
state, and the retirement banners that drifted as the fleet moved. The
wave was interrupted mid-run, so this commit carries only what passed the
integrity gates: every touched page parses (front matter or H1+status
convention intact, balanced fences, no truncated tables, trailing
newline), the docs_completeness and link-integrity seams are green, and
bats is 384/384.

Numbers that this session proved on live hosts are keyed to the measured
value, not the page's earlier claim.
This commit is contained in:
Frank committed 2026-10-18 13:15:43 -07:00
1 parent 466fd477d8
commit e7f0d93c86
67 files changed
+2619 -1152

No files matched your search

@@ -1,7 +1,7 @@
---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp, Gemma4-12B QAT Q4_K_M"
description: "The active frank-zealot runbook — mainline llama.cpp on Dankclaw's RTX 3090 serving HauhauCS Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M with a separate MTP assistant sidecar chained behind ngram-mod, q8_0 KV, and --fit sizing the full 262k window: identity, payload, KV arithmetic, the spec-decode chain, deployed flags, the client contract, and what to check when it misbehaves."
description: "The RETIRED frank-zealot runbook (2026-09-17 → 10-01) — mainline llama.cpp on Dankclaw's RTX 3090 serving HauhauCS Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M with a separate MTP assistant sidecar chained behind ngram-mod, q8_0 KV, and --fit sizing the full 262k window: identity, payload, KV arithmetic, the spec-decode chain, deployed flags, the client contract, and what to check when it misbehaves."
tags:
- llm
- frank-zealot
@@ -16,20 +16,26 @@ timestamp: 2026-09-17
# frank-zealot — llama.cpp Gemma4-12B QAT Q4_K_M (MTP + ngram-mod)
> **ACTIVE since 2026-09-17.** Replaces [[frank-zealot-qwen38-gsq-rco]]
> (ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S, now the rollback target) and
> [[frank-zealot-vllm-gemma4]] (the same-day vLLM experiment, also the
> rollback target). Same port (7771), same alias (`frank-zealot`), same
> client contract surface.
> **RETIRED 2026-09-17 → 2026-10-01** (AGENTS.md Runbook Index). It replaced
> [[frank-zealot-qwen38-gsq-rco]] (ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S) and
> [[frank-zealot-vllm-gemma4]] (the same-day vLLM experiment).
> Same port (7771), same alias (`frank-small` — this endpoint only carried the
> `frank-zealot` name from 2026-10-06/10-09), same client contract surface.
> :7771 today runs [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML
> fork build 10754, `Ternary-Bonsai-2-27B-PQ2_0.gguf`, 2 × 200,704 q8_0 KV, MTP
> off); the audio input this page documents left the fleet with this payload.
## Identity
- **Endpoint:** `frank-zealot` at `192.168.0.143:7771` (Dankclaw).
- **Endpoint:** `frank-small` at `192.168.0.143:7771` (Dankclaw) —
`git show e23df8b:scripts/frank-small.env` → `ALIAS=frank-small`.
- **Runtime:** mainline llama.cpp master `37b53fd`, CUDA build for sm_86 with
`-DGGML_CUDA_FA_ALL_QUANTS=ON`
(`/home/dank/GitPackages/llama.cpp-mainline/build-cuda/bin/llama-server`).
The FA_ALL_QUANTS flag is required: without it, q8_0 KV + flash attention
collapses prefill ~12x on CUDA.
collapses prefill ~12x on CUDA. Host check 2026-10-18: that binary reports
`0.4.1-dev (build 1, commit 37b53fd)` and its `CMakeCache.txt` reads
`GGML_CUDA_FA_ALL_QUANTS:BOOL=ON`.
- **Service shape:** root systemd unit `scripts/frank-zealot.service`
(`User=dank`), launched by `scripts/serve-frank-zealot.sh` with tunables from
`scripts/frank-zealot.env`.
@@ -78,25 +84,29 @@ the >12.8 GiB floor does not apply.
(`mmproj-Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf`, 11 tensors,
52M params), which also carries an audio encoder. The fleet's image path
lives here. Audio works: `input_audio` content parts transcribe verbatim
(verified 2026-09-17 with a 16k mono WAV).
(verified 2026-09-17 with a 16k mono WAV). Fleet-wide audio ended with this
payload: no fleet endpoint takes audio input today.
## Payload
The main GGUF, the BF16 mmproj, and the MTP sidecar live under
`/home/dank/models/Gemma4-12B-QAT`. The serve script refuses to start if any
The main GGUF, the BF16 mmproj, and the MTP sidecar lived under
`/home/dank/models/Gemma4-12B-QAT`; they were staged to
`.AGENTTRASH/20261002-retired-llm-payloads/Gemma4-12B-QAT/` on 2026-10-02 and
that stage dir is gone as of 2026-10-18, so the payload needs a re-fetch. The serve script refuses to start if any
is missing (a missing file is an unfinished download, and the unit's restart
policy would otherwise turn it into a crash loop that reads like a runtime
fault).
## Client contract
- **Alias:** `frank-zealot` (the `id` at `/v1/models`).
- **Alias:** `frank-small` (the `id` at `/v1/models`).
- **Reasoning:** extracted to `reasoning_content` (`--reasoning-format
deepseek`, unlimited budget).
- **Tools:** supported (`--jinja` is pinned even though this build defaults
it on, so a future build that flips the default cannot silently take tools
away).
- **Vision + audio:** supported.
- **Vision + audio:** both supported in this era; audio is gone fleet-wide
today (`input: [text, image]` — `~/.omp/agent/models.yml:214`).
- **Sampling:** greedy by default (fleet contract); a client that sends a
temperature gets its own, and the harness applies per-model overrides from
`models.yml`.
@@ -110,8 +120,12 @@ pp ~965 t/s, tg 257–302 t/s, idle 23,300 MiB of 24,576. Larger ubatches
## Rollback
Payload and runtime for [[frank-zealot-qwen38-gsq-rco]] (ISTA-DASLab
Qwen3.8-27B-GSQ-RCO IQ3_S) remain intact on disk. [[frank-zealot-vllm-gemma4]]
is the same-day vLLM experiment (also a rollback target).
Qwen3.8-27B-GSQ-RCO IQ3_S) are **not on disk any more** (the 2026-10-18 census
of `/home/dank/models/` has no GSQ-RCO directory and the era fetch script is
not in the tree) — that rollback re-downloads from ISTA-DASLab.
[[frank-zealot-vllm-gemma4]] is the same-day vLLM experiment; its weights are
gone too, though its derived image is still in rootless storage
(`localhost/vllm/vllm-openai:frank-small`, id `8dad46d79c46`).
# Citations