docs: distill the managed skill library into the vault (waves 1-7)
All ~173 managed skills read in full and merged, one polished reference page per skill-cluster: - LLM cross-cutting (17 pages): GGUF Compatibility, Llama Cpp Serving Sizing, Llama Cpp Multi-Device Splits, PCIe Link and Atomics, RDNA4 Container Serving, RDNA4 MoE Expert Offload VRAM Budget, vLLM Boot Forensics, vLLM Porting and Compile Triage, LLM Bench Windows, LLM Performance Diagnosis, Model Swap and Rollback, Tcclaviger Fork Ops, Heterogeneous Split Triage, FreeToken Builds + deltas folded into the retired runbooks (GLM DFlash2 port laws, Lucebox dual-R9700, R9V qualification pipeline, FLM pack registration, ik hybrid envelope, frank-medium split rebalance, llama.cpp GGUF preflight, batch-ubatch split section, bench-window D-state/stale-read notes, second-endpoint bring-up). - Architecture (3): Stale Signal Audit, Quadlet Lifecycle, Deploy State Triage. - Operations (6 + index): Remote Job Runner, Payload and Disk Reclaim, Container Host Reachability, Fleet Ops Playbook, Fleet Audit and Posture, SSH and Remote Access. - Infrastructure: Observability Provisioning + Homepage Status Probes; updates to Authentik, Anubis, LAN-DNS (per-client DoH bypass, Porkbun zone ops), Swap Configuration, OpenWebUI, Hermes (pre-existing page preserved and extended). - FrankBook-Pro: ARR-Stack filled (API split, schema-first probing, auth-posture, auth-config trap); qBittorrent E2E verification. - Desktop/Agent/Research: Noctalia and Niri, OMP Harness, Agent Docs and Edits, Command Center Dotfiles, PICO-8 Asset Standards. - Indexes: LLM Cross-Cutting References section, Operations/ and Agent/ index pages, root-index Skill Distillation section, AGENTS.md pointer. - log.md carries the wave plan. check-vault-links.py 0 unresolved (185 pages); ./tests/run.sh 366/366.
This commit is contained in:
1 parent
8618ec6dbe
commit
33eae71d33
43 files changed
+5436
-2
No files matched your search
@@ -45,6 +45,98 @@ timestamp: 2026-09-30
|
||||
> because the fork compiles its own Qwen trunk and not its GLM one. See
|
||||
> [The measured profile](#the-measured-profile-and-the-real-wall).
|
||||
|
||||
## DFlash2 target-side port — the four laws and the KV ladder
|
||||
|
||||
Distilled 2026-10-05 from the probe windows (skills `glm53-dflash-boot-ladder`
|
||||
/ `glm53-dflash-eagle3-port`); each law was proven by a real failed boot.
|
||||
|
||||
**The four EAGLE3-port laws** (make the GLM trunk satisfy `SupportsEagle3`):
|
||||
|
||||
1. **Patch the instance the runner calls.** `model_runner.py:425` calls
|
||||
`set_eagle3_aux_hidden_state_layers(self.model)` on the *loaded* model —
|
||||
GLM loads as `Glm5NextForConditionalGeneration` (multimodal wrapper;
|
||||
`self.language_model = Glm5NextForCausalLM`). Members placed only on the
|
||||
CausalLM fail the check on the conditional — same error text, wrong file.
|
||||
Patch BOTH wrappers.
|
||||
2. **Presence-checked member set.** `SupportsEagle3` (interfaces.py:1659) is
|
||||
`@runtime_checkable`; `isinstance` walks ALL `__protocol_attrs__`
|
||||
(including base-protocol members) with `getattr_static`, breaking on the
|
||||
first miss. Both wrappers need, as class attributes:
|
||||
`supports_eagle3: ClassVar[Literal[True]] = True`,
|
||||
`has_own_lm_head: bool = False`, `has_own_embed_tokens: bool = False`.
|
||||
Verify in-image with stock typing — no boot needed.
|
||||
3. **MRO forbids inheriting the Protocol.** GLM wrappers carry four/five
|
||||
Protocol-derived bases; adding `SupportsEagle3` anywhere dies with
|
||||
"Cannot create a consistent method resolution order" (sim-proven: the
|
||||
ordering rejects it in every base position). The attribute route avoids
|
||||
the MRO, so the two methods must be written by hand: the CausalLM
|
||||
unwraps to `self.model` (the `EagleModelMixin` trunk), the conditional
|
||||
delegates to `self.language_model` (pixtral.py:498 pattern);
|
||||
`get_eagle3_default_aux_hidden_state_layers` returns `(2, L//2, L-3)`,
|
||||
with the drafter config's `target_layer_ids` winning when present.
|
||||
4. **Trunk capture point:** append `aux_hidden_states` BEFORE the layer call
|
||||
— GLM's loop carries the single-stream state entering the layer.
|
||||
|
||||
**The KV blocker ladder** (patches `scripts/patches/glm53-dflash-0*`):
|
||||
|
||||
1. EAGLE3 gate → presence-checked attrs on both wrappers (above).
|
||||
2. KV page-size unifier: MLA layers cannot be padded (sparse MLA reads
|
||||
whole token rows) but CAN grow block_size when the max page is an exact
|
||||
multiple — bump `max_page_size` in `unify_kv_cache_spec_page_size` to
|
||||
the `lcm` of all MLA-family pages first (patch 02).
|
||||
3. kpool tail assert `tail_kv_cache.shape[2] == pool_size`: the unifier's
|
||||
divisible branch grew KpoolTailSpec's block; its kernel pins block ==
|
||||
index_kpool. Add a KpoolTailSpec branch that pads `page_size_padded`
|
||||
instead of growing (patch 03).
|
||||
4. KV capacity: the glm5 layout ALIASES mamba states INTO MLA pages (why
|
||||
GLM fits 32 GiB cards); the generic path (taken when draft/hidden specs
|
||||
exist) cannot alias → ~30 GiB demanded vs a 0.92 GiB reserve. Extend
|
||||
`_get_kv_cache_groups_glm5_next` to partition
|
||||
HiddenStateCacheSpec + drafter attention specs; the drafter forms its
|
||||
own uniform group (per-group pages, NO unification → the MLA block
|
||||
stays at the engine-auto-set 2,432 from interface.py:918, kpool guard
|
||||
`block_size % 512 == 0` unstressed); layout returns a 10-tuple
|
||||
(+draft_group, +hidden_groups); the builder sizes ONE shared backing
|
||||
allocation (`allocate_kv_cache` asserts all KVCacheTensor sizes equal) =
|
||||
aliased pool + appended draft/hidden tensors (patch 04). After 04 the
|
||||
capacity requirement collapsed 31.76 → 1.88 GiB at 65,536 max len and
|
||||
the engine SERVED (READY ~500 s, dflash2 fused prep ON, drafter FULL
|
||||
graphs on both TP ranks).
|
||||
5. Reserve registry: `_glm53_kv_bytes_per_token` must add 4,096 (drafter
|
||||
KV) + 5×hidden×2 (aux hidden cache) keyed on
|
||||
`speculative_config.method in {dflash, dspark, extract_hidden_states}`.
|
||||
NEVER key on `use_aux_hidden_state_outputs` — it lives on the
|
||||
gpu_model_runner instance (gpu_model_runner.py:623/678), unset at plan
|
||||
time, so the check silently fails and the reserve stays 13,600
|
||||
(patch 04b). Proven reserve math at 65,536: 13,600 + 4,096 +
|
||||
5×4096×2 ≈ 58,656 B/token → ~4.15 GiB reserve, arena ~10.3 GiB.
|
||||
|
||||
**Verify cheaply before booting:** smoke-test the pure KV logic on CPU
|
||||
torch with the image's real spec classes — stub `vllm.logger`/`vllm.utils.*`,
|
||||
register the spec classes in `KVCacheSpecRegistry` with a dummy manager, add
|
||||
`from __future__ import annotations`, and run synthetic GLM+draft+hidden
|
||||
spec sets through the patched grouping/layout/builder functions. A 7-minute
|
||||
boot per hypothesis is the alternative.
|
||||
|
||||
**Window discipline:** stop + mask `frank-primary` FIRST and gate on
|
||||
MemAvailable ≥ 230 GiB before booting the probe (the expert-offload planner
|
||||
reads MemAvailable; beside-prod boots die at plan time with `ExpertPlanError:
|
||||
host holds 5113 of 12096 swappable experts`). Save `podman logs glm-probe`
|
||||
to a file BEFORE teardown — a `podman logs | grep | head` pipeline died once
|
||||
under pipefail via SIGPIPE before log-save, losing the capture. The abort
|
||||
path must restore prod (unmask + daemon-reload + reset-failed + start),
|
||||
never bare-exit after stop+mask. Launcher knobs: `DFLASH_PATCH=1` (mounts
|
||||
the patched model.py), `DRAFT_MOUNT=1`, `SPECJSON_FILE=…` ({"method":
|
||||
"dflash","model":"/draft","num_speculative_tokens":8}) — note the launcher's
|
||||
echo prints `patch=$PATCH_MOUNT` (the COMPILE knob), not DFLASH_PATCH.
|
||||
Boot timeline: EAGLE3 gate fires ~4 min in at worker init; drafter
|
||||
torch.compile is the frontier (~380 s).
|
||||
|
||||
**Open telemetry discrepancy:** the contaminated window read 1.5–2.1 tok/s
|
||||
with acceptance 1.00–1.25; Frank live-measured ~18 tok/s decode / ~370 tok/s
|
||||
prefill. Reconcile with a clean single-request measurement before trusting
|
||||
either.
|
||||
|
||||
Related: [[dankserver-qwen38-fp8-expert-offload-tuning]] (the same fork, the same
|
||||
hardware — its conclusion was drawn from throughput arithmetic and is superseded
|
||||
by this page's kernel-time measurement),
|
||||
|
||||
Reference in new issue
Block a user