docs: distill the managed skill library into the vault (waves 1-7)
All ~173 managed skills read in full and merged, one polished reference page per skill-cluster: - LLM cross-cutting (17 pages): GGUF Compatibility, Llama Cpp Serving Sizing, Llama Cpp Multi-Device Splits, PCIe Link and Atomics, RDNA4 Container Serving, RDNA4 MoE Expert Offload VRAM Budget, vLLM Boot Forensics, vLLM Porting and Compile Triage, LLM Bench Windows, LLM Performance Diagnosis, Model Swap and Rollback, Tcclaviger Fork Ops, Heterogeneous Split Triage, FreeToken Builds + deltas folded into the retired runbooks (GLM DFlash2 port laws, Lucebox dual-R9700, R9V qualification pipeline, FLM pack registration, ik hybrid envelope, frank-medium split rebalance, llama.cpp GGUF preflight, batch-ubatch split section, bench-window D-state/stale-read notes, second-endpoint bring-up). - Architecture (3): Stale Signal Audit, Quadlet Lifecycle, Deploy State Triage. - Operations (6 + index): Remote Job Runner, Payload and Disk Reclaim, Container Host Reachability, Fleet Ops Playbook, Fleet Audit and Posture, SSH and Remote Access. - Infrastructure: Observability Provisioning + Homepage Status Probes; updates to Authentik, Anubis, LAN-DNS (per-client DoH bypass, Porkbun zone ops), Swap Configuration, OpenWebUI, Hermes (pre-existing page preserved and extended). - FrankBook-Pro: ARR-Stack filled (API split, schema-first probing, auth-posture, auth-config trap); qBittorrent E2E verification. - Desktop/Agent/Research: Noctalia and Niri, OMP Harness, Agent Docs and Edits, Command Center Dotfiles, PICO-8 Asset Standards. - Indexes: LLM Cross-Cutting References section, Operations/ and Agent/ index pages, root-index Skill Distillation section, AGENTS.md pointer. - log.md carries the wave plan. check-vault-links.py 0 unresolved (185 pages); ./tests/run.sh 366/366.
This commit is contained in:
1 parent
8618ec6dbe
commit
33eae71d33
43 files changed
+5436
-2
No files matched your search
@@ -134,6 +134,59 @@ slots of 131072, `6.44 / 6.90 GiB` free — matching the pre-deploy measurement
|
||||
0.01 GiB — a tool call and two concurrent requests answered, draft acceptance on
|
||||
both slots (0.63 / 0.48 on slot 1, 0.39 on slot 0), and no ROCm errors.
|
||||
|
||||
## The split: re-measure it at the deployed shape
|
||||
|
||||
The batch and the tensor split are **coupled**: compute buffers are not
|
||||
symmetric across the two cards, so any batch change moves the balance point.
|
||||
Order of work — sweep batch with llama-bench (above), THEN re-measure the
|
||||
split at the DEPLOYED shape (262144, two slots, drafter loaded) by loading
|
||||
the real service with a per-config env override and reading `rocm-smi`. A
|
||||
llama-bench number cannot decide the split: it has no `--parallel`, sizes
|
||||
its own context as `n_prompt + n_gen + n_depth`, and cannot load the drafter.
|
||||
|
||||
The settle gate is mandatory (above); wait for BOTH cards under ~1 GiB AND
|
||||
the port to be free (`ss -ltn | grep :7770` until gone) before the next
|
||||
load, or the transient server dies with `couldn't bind HTTP server socket`.
|
||||
Free bytes follow `card = weights_share + KV_share + constant + ubatch_term`,
|
||||
where the ubatch term costs the drafter-carrying card roughly **1.85×** the
|
||||
other (measured 2.41 GiB vs 1.30 GiB going 1536 → 512). Balance by
|
||||
measurement only: one `-ts` step is one layer, worth ~0.43 GiB on a DeltaNet
|
||||
boundary and ~1.43 GiB on an attention one, so the reachable minimum gap is
|
||||
about one DeltaNet layer, never zero. Reference: 36,29 @1536 → 4.71/4.91;
|
||||
36,29 @512 → 6.01/7.32; **35,30 @512 → 6.44/6.90 (deployed)**.
|
||||
|
||||
Re-fit both ubatch coefficients in `tests/frank_medium_unit.bats` at the new
|
||||
deployed point and re-anchor the per-card constants; check neighbouring
|
||||
splits still FAIL the balance gate or the seam has stopped discriminating
|
||||
(keep the 1.5 GiB/card floor and the one-DeltaNet-layer slack). Deploy,
|
||||
then verify live: flags in `/proc/<pid>/cmdline`, `/slots`, VRAM matching
|
||||
the pre-deploy measurement, a tool call, two concurrent requests,
|
||||
`draft acceptance` lines on BOTH slots, 0 ROCm errors.
|
||||
|
||||
**Per-card utilization (who is hot, and the split lever)** — a different
|
||||
question from batch/prefill. Measure decode tok/s AND per-card
|
||||
`rocm-smi --showuse` at 2–3 neighbouring splits in ONE session. Pitfalls
|
||||
(each cost a re-run): run the transient unit as ROOT (a `pkill -9
|
||||
llama-server` as dank auto-restarts the service and `systemctl` as dank
|
||||
fails "Access denied"); warm up 3 throwaway requests before the timed one
|
||||
(first 384-token decode after load measured 1.77 tok/s against a true ~32);
|
||||
`rocm-smi --showmeminfo vram --csv` columns are `$2`=Total, `$3`=**Used** —
|
||||
parsing the wrong column spins the settle gate the full 600 s; the util% is
|
||||
sampled post-hoc (which card finished last), so treat it as "which card is
|
||||
the pipeline tail", not a duty cycle.
|
||||
|
||||
**What the split does and does not buy (measured 2026-09-17, 393216 /
|
||||
MTP n=3):** the split is NOT a decode-throughput lever — 34:31, 35:30, 36:29
|
||||
all sit in the same ~32 tok/s band. The x1 card runs hot (~91–97 %) at
|
||||
35:30 because the MTP drafter + output layers + heavier compute buffers are
|
||||
pinned to the card holding the LAST layers — not because PCIe is the
|
||||
bottleneck (~5.2 MB of boundary activation crosses per 512-token ubatch).
|
||||
36:29 makes the x16 card the utilized worker (42 %/3 %) at no throughput
|
||||
cost but spends headroom (0.45 GiB free, under the floor) — Frank's standing
|
||||
preference is that the card with the FASTEST PCIe lane should be the more
|
||||
utilized one; deploying it requires re-fitting the seam's slack/floor and
|
||||
re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **`llama-bench` cannot run the drafter, the slots or the deployed context.**
|
||||
|
||||
Reference in new issue
Block a user