docs(vault): fleet-wide fact re-verification pass (67 pages)
The docs-fact census wave re-read the live hosts and re-keyed every page it could: runtime/payload identities, ports, sizes, ledger figures, unit state, and the retirement banners that drifted as the fleet moved. The wave was interrupted mid-run, so this commit carries only what passed the integrity gates: every touched page parses (front matter or H1+status convention intact, balanced fences, no truncated tables, trailing newline), the docs_completeness and link-integrity seams are green, and bats is 384/384. Numbers that this session proved on live hosts are keyed to the measured value, not the page's earlier claim.
This commit is contained in:
1 parent
466fd477d8
commit
e7f0d93c86
67 files changed
+2619
-1152
No files matched your search
@@ -13,6 +13,16 @@ This page records the full redo of the inference fleet on 2026-08-17.
|
||||
The pipeline-parallel split and the FastFlowLM NPU endpoint were retired.
|
||||
The old scripts moved to `old/scripts/` in this repository.
|
||||
|
||||
> **HISTORY — the topology below is two fleet generations dead.** Today the
|
||||
> fleet is three endpoints: [[frank-archon-radiance-qwen38-iq4r]]
|
||||
> (`192.168.0.235:7770`, radiance 1.0.13, 2×R9700 TP=2),
|
||||
> [[frank-dragoon-flash-server-v2-native]] (`192.168.0.143:7770`, halogen
|
||||
> flash-server 0.16.2 on the iGPU) and
|
||||
> [[frank-zealot-llama-prism-bonsai2-pq20]] (`192.168.0.143:7771`, llama.cpp
|
||||
> PrismML fork build 10754 on the RTX 3090 — the card that sat in Dankserver
|
||||
> until 2026-09-15). No vLLM and no ROCm endpoint serves anything; there is no
|
||||
> pipeline parallelism and no NPU endpoint.
|
||||
|
||||
## New Topology
|
||||
|
||||
| Service | Host | Runtime | Model | Port |
|
||||
@@ -20,6 +30,12 @@ The old scripts moved to `old/scripts/` in this repository.
|
||||
| frank-dragoon | Dankserver | llama.cpp (patched FastMTP) | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS | 7770 |
|
||||
| frank-zealot | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 |
|
||||
|
||||
Names as the 2026-08-17 commit wrote them: `frank-default-large` on Dankserver
|
||||
(`git show d9af7f1:scripts/frank-default-large.env:20,22` → `ALIAS=frank-default-large`,
|
||||
`LISTEN_PORT=7770`) and `frank-default-small` on Dankclaw
|
||||
(`git show d9af7f1:scripts/frank-default-small.env:16,18` → `ALIAS=frank-default-small`,
|
||||
`LISTEN_PORT=52625`); the Protoss ids date from 2026-10-06 → 10-10.
|
||||
|
||||
The RTX 3090 runs the whole 27B model standalone. No RPC worker, no
|
||||
tensor split, no NPU.
|
||||
|
||||
@@ -58,7 +74,8 @@ tensor split, no NPU.
|
||||
## Context Math (frank-dragoon, RTX 3090)
|
||||
|
||||
KV per token: 16 attention layers x 4 KV heads x 256 dim.
|
||||
K q8_0 = 1088 B/token, V f16 = 2048 B/token, total 50176 B/token.
|
||||
K q8_0 = 1088 B per layer (4 KV heads × 256 dim × 1.0625 B), V f16 = 2048 B per
|
||||
layer, ×16 layers = 50176 B/token total.
|
||||
|
||||
Budget (MiB): 24576 total - 14982 weights - 861 draft - 1024 fixed
|
||||
(CUDA context, compute graph, scratch, draft KV) - headroom.
|
||||
@@ -150,7 +167,8 @@ on gfx1151; newer vLLM releases may already include it.
|
||||
|
||||
- `models.json` (pi): two providers — `llamacpp (dankserver)` at
|
||||
http://192.168.0.235:7770/v1 and `vllm (dankclaw)` at
|
||||
http://192.168.0.143:52625/v1.
|
||||
http://192.168.0.143:52625/v1. (The harness config today has three providers
|
||||
and no vLLM: `~/.omp/agent/models.yml:14,94,173`.)
|
||||
- HF token: `~/.cache/huggingface/token` on both machines (mode 600).
|
||||
- OpenWebUI (sibling repo): retarget `OPENAI_API_BASE_URL` to the
|
||||
Dankserver endpoint.
|
||||
|
||||
Reference in new issue
Block a user