docs(vault): fleet-wide fact re-verification pass (67 pages)
The docs-fact census wave re-read the live hosts and re-keyed every page it could: runtime/payload identities, ports, sizes, ledger figures, unit state, and the retirement banners that drifted as the fleet moved. The wave was interrupted mid-run, so this commit carries only what passed the integrity gates: every touched page parses (front matter or H1+status convention intact, balanced fences, no truncated tables, trailing newline), the docs_completeness and link-integrity seams are green, and bats is 384/384. Numbers that this session proved on live hosts are keyed to the measured value, not the page's earlier claim.
This commit is contained in:
1 parent
466fd477d8
commit
e7f0d93c86
67 files changed
+2619
-1152
No files matched your search
@@ -17,10 +17,13 @@ timestamp: 2026-09-19
|
||||
|
||||
# frank-zealot — vLLM on the RTX 3090 (Gemma4-12B QAT w4a16)
|
||||
|
||||
> **RETIRED 2026-09-19 (same-day revert).** This vLLM deployment was cut
|
||||
> back to [[frank-zealot-gemma4-12b]] (llama.cpp, now the ACTIVE runbook)
|
||||
> the same day it landed — this page is the rollback target. Same port
|
||||
> (7771), same alias (`frank-zealot`), same client contract surface — the
|
||||
> **RETIRED 2026-09-19 (same-day revert) — rollback target only in git.**
|
||||
> This vLLM deployment was cut
|
||||
> back to [[frank-zealot-gemma4-12b]] (llama.cpp — itself retired
|
||||
> 2026-10-01; :7771 today runs [[frank-zealot-llama-prism-bonsai2-pq20]])
|
||||
> the same day it landed. Same port
|
||||
> (7771), same alias (`frank-small` — the name `frank-zealot` dates from
|
||||
> 2026-10-06/10-09), same client contract surface — the
|
||||
> runtime, the model payload, the drafter and the service shape (root
|
||||
> systemd unit → rootless Quadlet) all changed.
|
||||
|
||||
@@ -28,12 +31,12 @@ timestamp: 2026-09-19
|
||||
|
||||
| what | value |
|
||||
|---|---|
|
||||
| runtime | vLLM `vllm/vllm-openai` dev build `gemma-x86_64-cu129` (base `sha256:e24d3d8e…001e`), audio extra pip-installed into `localhost/vllm/vllm-openai:frank-zealot` (`8dad46d7…`) |
|
||||
| unit | **rootless Quadlet** `dankclaw/frank-zealot/quadlet/frank-zealot.container` (root systemd unit retired 2026-09-19) |
|
||||
| runtime | vLLM `vllm/vllm-openai` dev build `gemma-x86_64-cu129` (base `sha256:e24d3d8e…001e`), audio extra pip-installed into `localhost/vllm/vllm-openai:frank-small` (`8dad46d79c46` — still in rootless storage on the 2026-10-18 `podman images` census; the tag carries the era name) |
|
||||
| unit | **rootless Quadlet** `dankclaw/frank-small/quadlet/frank-small.container` (era path; deleted by the revert commit `374041c` — root systemd unit retired 2026-09-19) |
|
||||
| model | `google/gemma-4-12B-it-qat-w4a16-ct` — compressed-tensors w4a16 (int4 Marlin weights, 10,264,229,896 B) |
|
||||
| model dir | `/home/dank/models/gemma4-12b-qat-w4a16-ct/` |
|
||||
| drafter | `google/gemma-4-12B-it-assistant` **bf16** (845,719,296 B) at `/home/dank/models/gemma4-12b-assistant-bf16/` |
|
||||
| port | 7771 → 8000 (alias `frank-zealot`; `frank-dragoon` keeps 7770) |
|
||||
| model dir | `/home/dank/models/gemma4-12b-qat-w4a16-ct/` (absent from the 2026-10-18 census of `/home/dank/models/` — re-fetch to use this rollback) |
|
||||
| drafter | `google/gemma-4-12B-it-assistant` **bf16** (845,719,296 B) at `/home/dank/models/gemma4-12b-assistant-bf16/` (absent from the same census) |
|
||||
| port | 7771 → 8000 (alias `frank-small` in this era; `frank-dragoon` keeps 7770) |
|
||||
| window | 262,144; KV pool **289,647 tokens** bf16; **2 concurrent full-context sessions** |
|
||||
| multimodal | native: text + image + **audio** (derived image carries `av`/`scipy`/`soundfile`/`mistral_common[audio]`) |
|
||||
|
||||
@@ -113,7 +116,8 @@ trade for the native w4a16 format + audio + CUDA graphs.)
|
||||
|
||||
## The image
|
||||
|
||||
`dankclaw/frank-zealot/Dockerfile` (repo) — 16 comment lines + `FROM
|
||||
`dankclaw/frank-small/Dockerfile` (era path, deleted by the revert commit
|
||||
`374041c`; recover with `git show 374041c^:dankclaw/frank-small/Dockerfile`) — 16 comment lines + `FROM
|
||||
localhost/vllm/vllm-openai:gemma4-unified` + `RUN pip install
|
||||
--no-cache-dir av scipy soundfile "mistral_common[audio]"`. **No COPY
|
||||
lines** — the earlier `triton-sm89-sitecustomize.py` hack is gone:
|
||||
@@ -144,9 +148,9 @@ not a hardware mystery.
|
||||
|
||||
## Deployed config
|
||||
|
||||
Repo truth: `scripts/frank-zealot.env` + `scripts/serve-frank-zealot.sh`
|
||||
Repo truth **for this era** (the files carry llama.cpp knobs today): `scripts/frank-zealot.env` + `scripts/serve-frank-zealot.sh`
|
||||
(mounted into the container at `/scripts/`, which is the entrypoint) +
|
||||
`dankclaw/frank-zealot/quadlet/frank-zealot.container`. Key values:
|
||||
`dankclaw/frank-small/quadlet/frank-small.container` (era path, deleted at `374041c`). Key values:
|
||||
`KV_CACHE_DTYPE=auto` (resolves to bf16 — fp8_e5m2 is refused by
|
||||
compressed-tensors checkpoints and fp8 is hardware-dead on sm_86),
|
||||
`ENFORCE_EAGER=0` (CUDA graphs on), `max_model_len=262144`, MTP via
|
||||
@@ -169,7 +173,9 @@ assistant, or the vllm binary are missing.
|
||||
- Audio → 200, a 440 Hz WAV tone described correctly (native audio
|
||||
encoder; verified in the derived image).
|
||||
- Harness: `contextWindow: 262144`, `maxTokens: 32768`,
|
||||
`input: [text, image, audio]`.
|
||||
`input: [text, image, audio]` — era values; the endpoint reads 200,704 /
|
||||
16,384 / `[text, image]` today (`~/.omp/agent/models.yml:206,212,214`), and
|
||||
audio left the fleet with this payload on 2026-10-01.
|
||||
|
||||
## Boot ledger
|
||||
|
||||
@@ -186,27 +192,35 @@ container while it loads.
|
||||
- **`sm_89` in an error** — stale arch override; `TRITON_OVERRIDE_ARCH`
|
||||
must be absent. 8.6 is correct.
|
||||
- **Audio 400/500** — the container is running the *base* image, not the
|
||||
derived `:frank-zealot` (missing `av`). Check the deployed image digest
|
||||
is `8dad46d7…`.
|
||||
derived `:frank-small` (missing `av`). In this era, check the deployed image
|
||||
id is `8dad46d7…` (`podman images`, 2026-10-18: it still exists, unused —
|
||||
nothing on :7771 runs in podman today).
|
||||
- **CUDA not visible in container** — a `Device=` line or a lib mount
|
||||
dropped out of the unit after an edit; the unit is the source of truth,
|
||||
the host-side generated unit is not.
|
||||
- Logs: `journalctl --user -u frank-zealot` (rootless Quadlet; `podman
|
||||
logs` covers only the current `--replace --rm` instance).
|
||||
- Logs: `journalctl --user -u frank-zealot` (rootless Quadlet in this era;
|
||||
`podman logs` covers only the current `--replace --rm` instance). The unit is
|
||||
a ROOT unit today: `journalctl -u frank-zealot`.
|
||||
|
||||
## Rollback
|
||||
|
||||
[[frank-zealot-gemma4-12b]] — the llama.cpp deployment. Its weights,
|
||||
mmproj and MTP sidecar are still on the host under
|
||||
`/home/dank/models/Gemma4-12B-QAT/`, and the retired root unit is at
|
||||
`scripts/.AGENTTRASH/frank-zealot.service.llama.cpp`. Restoring it means
|
||||
reinstalling that unit and stopping the Quadlet (one owner per port).
|
||||
mmproj and MTP sidecar are **no longer on the host** under
|
||||
`/home/dank/models/Gemma4-12B-QAT/` (2026-10-18 census: the directory is
|
||||
gone — it was staged to `.AGENTTRASH/20261002-retired-llm-payloads/` and that
|
||||
stage dir is gone too), so this rollback re-fetches
|
||||
`HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced` and restores the era
|
||||
trio from git (`git show 374041c:scripts/frank-small.service`,
|
||||
`…:scripts/frank-small.env`, `…:scripts/serve-frank-small.sh`); the era quadlet
|
||||
lived at `dankclaw/frank-small/quadlet/frank-small.container`. One owner per
|
||||
port: the llama.cpp unit is root, so stop the container unit before starting it.
|
||||
|
||||
## References
|
||||
|
||||
- `scripts/serve-frank-zealot.sh` — the actual exec line
|
||||
- `dankclaw/frank-zealot/Dockerfile` — the audio-derived image
|
||||
- `dankclaw/frank-zealot/quadlet/frank-zealot.container` — the unit
|
||||
- `scripts/serve-frank-zealot.sh` — the actual exec line (today: llama.cpp; the
|
||||
vLLM exec line is in `git show 374041c^:scripts/serve-frank-small.sh`)
|
||||
- `dankclaw/frank-small/Dockerfile` — the audio-derived image (era path, deleted at `374041c`)
|
||||
- `dankclaw/frank-small/quadlet/frank-small.container` — the unit (era path, deleted at `374041c`)
|
||||
- [[frank-zealot-gemma4-12b]] — RETIRED predecessor (rollback)
|
||||
- [[halogen-flash-server]] — the sibling endpoint on the iGPU
|
||||
- [[VLLM Overview]] / [[VLLM Index]] — runtime background
|
||||
Reference in new issue
Block a user