docs(vault): fleet-wide fact re-verification pass (67 pages)

The docs-fact census wave re-read the live hosts and re-keyed every page
it could: runtime/payload identities, ports, sizes, ledger figures, unit
state, and the retirement banners that drifted as the fleet moved. The
wave was interrupted mid-run, so this commit carries only what passed the
integrity gates: every touched page parses (front matter or H1+status
convention intact, balanced fences, no truncated tables, trailing
newline), the docs_completeness and link-integrity seams are green, and
bats is 384/384.

Numbers that this session proved on live hosts are keyed to the measured
value, not the page's earlier claim.
This commit is contained in:
Frank committed 2026-10-18 13:15:43 -07:00
1 parent 466fd477d8
commit e7f0d93c86
67 files changed
+2619 -1152

No files matched your search

@@ -17,10 +17,13 @@ timestamp: 2026-09-19
# frank-zealot — vLLM on the RTX 3090 (Gemma4-12B QAT w4a16)
> **RETIRED 2026-09-19 (same-day revert).** This vLLM deployment was cut
> back to [[frank-zealot-gemma4-12b]] (llama.cpp, now the ACTIVE runbook)
> the same day it landed — this page is the rollback target. Same port
> (7771), same alias (`frank-zealot`), same client contract surface — the
> **RETIRED 2026-09-19 (same-day revert) — rollback target only in git.**
> This vLLM deployment was cut
> back to [[frank-zealot-gemma4-12b]] (llama.cpp — itself retired
> 2026-10-01; :7771 today runs [[frank-zealot-llama-prism-bonsai2-pq20]])
> the same day it landed. Same port
> (7771), same alias (`frank-small` — the name `frank-zealot` dates from
> 2026-10-06/10-09), same client contract surface — the
> runtime, the model payload, the drafter and the service shape (root
> systemd unit → rootless Quadlet) all changed.
@@ -28,12 +31,12 @@ timestamp: 2026-09-19
| what | value |
|---|---|
| runtime | vLLM `vllm/vllm-openai` dev build `gemma-x86_64-cu129` (base `sha256:e24d3d8e…001e`), audio extra pip-installed into `localhost/vllm/vllm-openai:frank-zealot` (`8dad46d7…`) |
| unit | **rootless Quadlet** `dankclaw/frank-zealot/quadlet/frank-zealot.container` (root systemd unit retired 2026-09-19) |
| runtime | vLLM `vllm/vllm-openai` dev build `gemma-x86_64-cu129` (base `sha256:e24d3d8e…001e`), audio extra pip-installed into `localhost/vllm/vllm-openai:frank-small` (`8dad46d79c46` — still in rootless storage on the 2026-10-18 `podman images` census; the tag carries the era name) |
| unit | **rootless Quadlet** `dankclaw/frank-small/quadlet/frank-small.container` (era path; deleted by the revert commit `374041c` — root systemd unit retired 2026-09-19) |
| model | `google/gemma-4-12B-it-qat-w4a16-ct` — compressed-tensors w4a16 (int4 Marlin weights, 10,264,229,896 B) |
| model dir | `/home/dank/models/gemma4-12b-qat-w4a16-ct/` |
| drafter | `google/gemma-4-12B-it-assistant` **bf16** (845,719,296 B) at `/home/dank/models/gemma4-12b-assistant-bf16/` |
| port | 7771 → 8000 (alias `frank-zealot`; `frank-dragoon` keeps 7770) |
| model dir | `/home/dank/models/gemma4-12b-qat-w4a16-ct/` (absent from the 2026-10-18 census of `/home/dank/models/` — re-fetch to use this rollback) |
| drafter | `google/gemma-4-12B-it-assistant` **bf16** (845,719,296 B) at `/home/dank/models/gemma4-12b-assistant-bf16/` (absent from the same census) |
| port | 7771 → 8000 (alias `frank-small` in this era; `frank-dragoon` keeps 7770) |
| window | 262,144; KV pool **289,647 tokens** bf16; **2 concurrent full-context sessions** |
| multimodal | native: text + image + **audio** (derived image carries `av`/`scipy`/`soundfile`/`mistral_common[audio]`) |
@@ -113,7 +116,8 @@ trade for the native w4a16 format + audio + CUDA graphs.)
## The image
`dankclaw/frank-zealot/Dockerfile` (repo) — 16 comment lines + `FROM
`dankclaw/frank-small/Dockerfile` (era path, deleted by the revert commit
`374041c`; recover with `git show 374041c^:dankclaw/frank-small/Dockerfile`) — 16 comment lines + `FROM
localhost/vllm/vllm-openai:gemma4-unified` + `RUN pip install
--no-cache-dir av scipy soundfile "mistral_common[audio]"`. **No COPY
lines** — the earlier `triton-sm89-sitecustomize.py` hack is gone:
@@ -144,9 +148,9 @@ not a hardware mystery.
## Deployed config
Repo truth: `scripts/frank-zealot.env` + `scripts/serve-frank-zealot.sh`
Repo truth **for this era** (the files carry llama.cpp knobs today): `scripts/frank-zealot.env` + `scripts/serve-frank-zealot.sh`
(mounted into the container at `/scripts/`, which is the entrypoint) +
`dankclaw/frank-zealot/quadlet/frank-zealot.container`. Key values:
`dankclaw/frank-small/quadlet/frank-small.container` (era path, deleted at `374041c`). Key values:
`KV_CACHE_DTYPE=auto` (resolves to bf16 — fp8_e5m2 is refused by
compressed-tensors checkpoints and fp8 is hardware-dead on sm_86),
`ENFORCE_EAGER=0` (CUDA graphs on), `max_model_len=262144`, MTP via
@@ -169,7 +173,9 @@ assistant, or the vllm binary are missing.
- Audio → 200, a 440 Hz WAV tone described correctly (native audio
encoder; verified in the derived image).
- Harness: `contextWindow: 262144`, `maxTokens: 32768`,
`input: [text, image, audio]`.
`input: [text, image, audio]` — era values; the endpoint reads 200,704 /
16,384 / `[text, image]` today (`~/.omp/agent/models.yml:206,212,214`), and
audio left the fleet with this payload on 2026-10-01.
## Boot ledger
@@ -186,27 +192,35 @@ container while it loads.
- **`sm_89` in an error** — stale arch override; `TRITON_OVERRIDE_ARCH`
must be absent. 8.6 is correct.
- **Audio 400/500** — the container is running the *base* image, not the
derived `:frank-zealot` (missing `av`). Check the deployed image digest
is `8dad46d7…`.
derived `:frank-small` (missing `av`). In this era, check the deployed image
id is `8dad46d7…` (`podman images`, 2026-10-18: it still exists, unused —
nothing on :7771 runs in podman today).
- **CUDA not visible in container** — a `Device=` line or a lib mount
dropped out of the unit after an edit; the unit is the source of truth,
the host-side generated unit is not.
- Logs: `journalctl --user -u frank-zealot` (rootless Quadlet; `podman
logs` covers only the current `--replace --rm` instance).
- Logs: `journalctl --user -u frank-zealot` (rootless Quadlet in this era;
`podman logs` covers only the current `--replace --rm` instance). The unit is
a ROOT unit today: `journalctl -u frank-zealot`.
## Rollback
[[frank-zealot-gemma4-12b]] — the llama.cpp deployment. Its weights,
mmproj and MTP sidecar are still on the host under
`/home/dank/models/Gemma4-12B-QAT/`, and the retired root unit is at
`scripts/.AGENTTRASH/frank-zealot.service.llama.cpp`. Restoring it means
reinstalling that unit and stopping the Quadlet (one owner per port).
mmproj and MTP sidecar are **no longer on the host** under
`/home/dank/models/Gemma4-12B-QAT/` (2026-10-18 census: the directory is
gone — it was staged to `.AGENTTRASH/20261002-retired-llm-payloads/` and that
stage dir is gone too), so this rollback re-fetches
`HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced` and restores the era
trio from git (`git show 374041c:scripts/frank-small.service`,
`…:scripts/frank-small.env`, `…:scripts/serve-frank-small.sh`); the era quadlet
lived at `dankclaw/frank-small/quadlet/frank-small.container`. One owner per
port: the llama.cpp unit is root, so stop the container unit before starting it.
## References
- `scripts/serve-frank-zealot.sh` — the actual exec line
- `dankclaw/frank-zealot/Dockerfile` — the audio-derived image
- `dankclaw/frank-zealot/quadlet/frank-zealot.container` — the unit
- `scripts/serve-frank-zealot.sh` — the actual exec line (today: llama.cpp; the
vLLM exec line is in `git show 374041c^:scripts/serve-frank-small.sh`)
- `dankclaw/frank-small/Dockerfile` — the audio-derived image (era path, deleted at `374041c`)
- `dankclaw/frank-small/quadlet/frank-small.container` — the unit (era path, deleted at `374041c`)
- [[frank-zealot-gemma4-12b]] — RETIRED predecessor (rollback)
- [[halogen-flash-server]] — the sibling endpoint on the iGPU
- [[VLLM Overview]] / [[VLLM Index]] — runtime background