Files

T

Frank 219a097b39 docs(vault): one page per endpoint, forever -- 45 era pages fold into three

Every payload swap a live endpoint has ever run is now one row in that
endpoint's Eras table, with its rollback path in the same row. No per-attempt
page, no per-swap ledger, nowhere else.

- Added: LLM/frank-archon.md (radiance, 2x R9700 TP=2), LLM/frank-dragoon.md
  (halogen flash-server, 19 eras), LLM/frank-zealot.md (llama.cpp PrismML fork
  on the 3090). Each carries Current shape, Eras, Laws, wire contract, files.
- Deleted: 45 per-payload runbooks -- the frank-dragoon line (11 including the
  pre-name halogen/kingjones/kairic/CIRU/eemin/agentionai/dflash2 pages and the
  benchmark page), the frank-zealot line (llama.cpp/vLLM/ik_llama/SGLang eras),
  the frank-archon line (vLLM/FreeToken/R9V/llama.cpp/radiance eras), plus
  frank-medium, frank-npu and the ubatch/PCIe one-offs that owned nothing.
- Rewired: every wikilink and markdown-path reference to a folded page points
  at the endpoint page instead; the three Research see-also lists and the
  ubatch sweep's runbook link included. No anchored links existed in the set.
- Regenerated both index pages as pure locators -- one row, one clause, no
  numbers -- so the docs_completeness locator law holds for all 153 pages.

tests/docs_completeness.bats green: index links every page, no unresolved
links. The pages moved to .AGENTTRASH/20261018-doc-dedupe/LLM/ (gitignored,
and git history holds every byte).

2026-10-19 00:49:03 -07:00

4.5 KiB

Raw Blame History

type, description, tags, timestamp

type

description

Ling-3.0-flash Q6_K Trial on Dankclaw

RETIRED 2026-08-17. The frank-dragoon-ling unit is disabled under the boot policy. At this page's retirement date frank-dragoon ran the Qwen3.8-27B PP head (since superseded by the STRIX deployment — see frank-dragoon), not the Qwen3.6-27B config described in the (obsolete) switch-back section below.

Status

Service: frank-dragoon-ling (systemd).
Model: inclusionAI/Ling-3.0-flash Q6_K (MoE, ~118B total params).
Runtime: llama.cpp fork aetherbird/llama.cpp branch bailingmoe3-support (commit db4480bc8, build b10411) at /home/dank/llama.cpp-ling. Stock llama.cpp cannot load the bailingmoe3 architecture.
Backend: Vulkan0 (RADV) with AMD GTT.
Spec: draft-mtp n_max 3. The MTP block is bundled in the Ling GGUF itself (block 42) - no separate drafter file.
Context: 262144 total, 2 slots x 131072, batch 512 / ubatch 256.
Measured: load ~45s, generation ~7.7 t/s (cold, 4k prompt) to ~36 t/s (tiny cached prompt). Concurrent streams both complete.

Why Vulkan + GTT

The Strix Halo APU has 8 x 16 GB LPDDR5 modules = 128 GB physical shared memory. The firmware carves ~96 GiB as dedicated iGPU VRAM; the kernel sees the remainder (~30 GiB, MemTotal). GRUB adds no memory options (iommu=pt only).

The ROCm runtime reports 96 GiB VRAM. Vulkan (RADV) reports a 111.2 GiB heap (113910 MiB) because it can also reach system memory through the GTT. For a model near 100 GiB, Vulkan is the only backend with room. --list-devices shows the heap:

ROCm0:   AMD Radeon Graphics (98304 MiB)
Vulkan0: AMD Radeon Graphics (RADV STRIX_HALO) (113910 MiB)

Memory Fit (Q6_K = 98.26 GiB)

Item	Size
Vulkan heap (llama.cpp view)	~111.2 GiB
Model Q6_K file	98.26 GiB
Model + MTP + graphs in heap	~98.7 GiB
Free heap after load	~10 GiB

Ling uses gated MLA and KDA attention. KV cache stays tiny: 256k total context costs only ~2.5 GiB. This is why 256k context fits.

Freeze Bug (2026-08-14) - Spec Path Deadlock

Symptom

Under 2+ concurrent streams with speculative decoding, one stream hard-freezes mid-generation (zero output for minutes; a deadlock, not slowness). Freeze point varies (385-1588 tokens). Single streams never freeze.

Isolated cause

Not context exhaustion: truncated = 0, under 131072 ctx.
Not batch (3072 and 8192 both froze) and not log verbosity (0 and 4 both).
parallel=2 + draft-mtp,ngram-mod -> freeze. SPEC OFF -> no freeze.
Culprit: the speculative-decode / context-checkpoint path in llama-server with multiple slots on the Vulkan backend. Known bug family: PR #24891, #24797 (server checkpoint fixes), #25144 (MTP draft crash), plus a ggml-vulkan hang-under-sustained-compute class.

Resolution: SPEC OFF

SPEC_TYPE=none, PARALLEL=2. All concurrent streams complete (EOS). Speed cost: ~10-13 t/s under contention vs ~28-32 t/s single-stream with MTP. Fallback chain if spec-off regresses: A) PARALLEL=1 keep MTP, B) switch back to Qwen3.6-27B (frank-dragoon, MTP n_max 3).

Files

scripts/frank-dragoon-ling.service - systemd unit.
scripts/serve-ling-flash.sh - serve script (fork binary, Vulkan, draft-mtp).
scripts/frank-dragoon.ling.env - tunables.

Passwordless management is installed: /usr/local/sbin/frank-restart-ling.sh and /etc/sudoers.d/frank-ling.

Switch Back (obsolete)

The switch-back commands below are obsolete. At retirement time frank-dragoon ran the Qwen3.8-27B pipeline-parallel head. To restore a Ling trial, re-enable frank-dragoon-ling from the repo unit (scripts/frank-dragoon-ling.service) and serve script (scripts/serve-ling-flash.sh), then start it on a non-conflicting port.

Caveats

The bailingmoe3 fork validates CPU and CUDA upstream. Vulkan was validated here (load + generation + concurrent streams) on 2026-08-14.
The tokenizer logs special_eos_id is not in special_eog_ids at load. It is a warning; generation stops correctly at EOS.
Physical RAM pressure stays low because the model fits in the UMA carve-out. Generation speed on this APU is below the Qwen3.6-27B MTP config (~7.7 vs ~15-19 t/s at 4k).
The heap is effectively full (98.7 of 111.2 GiB). Do not raise the context much without checking --list-devices free MiB.

4.5 KiB Raw Blame History