Files

T

Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts

Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.

2026-10-19 05:00:43 -07:00

4.8 KiB

Raw Blame History

type, description, tags, timestamp

type

description

Ling-3.0-flash Q6_K Trial on Dankclaw

RETIRED 2026-08-17. The frank-dragoon-ling unit is disabled under the boot policy. At this page's retirement date frank-dragoon ran the Qwen3.8-27B PP head (since superseded by the STRIX deployment — see frank-dragoon), not the Qwen3.6-27B config described in the (obsolete) switch-back section below.

Status

Service: frank-dragoon-ling (systemd).
Model: inclusionAI/Ling-3.0-flash Q6_K (MoE, ~118B total params).
Runtime: llama.cpp fork aetherbird/llama.cpp branch bailingmoe3-support (commit db4480bc8, build b10411) at /home/dank/llama.cpp-ling. Stock llama.cpp cannot load the bailingmoe3 architecture.
Backend: Vulkan0 (RADV) with AMD GTT.
Spec: draft-mtp n_max 3. The MTP block is bundled in the Ling GGUF itself (block 42) - no separate drafter file.
Context: 262144 total, 2 slots x 131072, batch 512 / ubatch 256.
Measured: load ~45s, generation ~7.7 t/s (cold, 4k prompt) to ~36 t/s (tiny cached prompt). Concurrent streams both complete.

Why Vulkan + GTT

The Strix Halo APU has 8 x 16 GB LPDDR5 modules = 128 GB physical shared memory. The firmware carves ~96 GiB as dedicated iGPU VRAM; the kernel sees the remainder (~30 GiB, MemTotal). GRUB adds no memory options (iommu=pt only).

The ROCm runtime reports 96 GiB VRAM. Vulkan (RADV) reports a 111.2 GiB heap (113910 MiB) because it can also reach system memory through the GTT. For a model near 100 GiB, Vulkan is the only backend with room. --list-devices shows the heap:

ROCm0:   AMD Radeon Graphics (98304 MiB)
Vulkan0: AMD Radeon Graphics (RADV STRIX_HALO) (113910 MiB)

Memory Fit (Q6_K = 98.26 GiB)

Item	Size
Vulkan heap (llama.cpp view)	~111.2 GiB
Model Q6_K file	98.26 GiB
Model + MTP + graphs in heap	~98.7 GiB
Free heap after load	~10 GiB

Ling uses gated MLA and KDA attention. KV cache stays tiny: 256k total context costs only ~2.5 GiB. This is why 256k context fits.

Freeze Bug (2026-08-14) - Spec Path Deadlock

Symptom

Under 2+ concurrent streams with speculative decoding, one stream hard-freezes mid-generation (zero output for minutes; a deadlock, not slowness). Freeze point varies (385-1588 tokens). Single streams never freeze.

Isolated cause

Not context exhaustion: truncated = 0, under 131072 ctx.
Not batch (3072 and 8192 both froze) and not log verbosity (0 and 4 both).
parallel=2 + draft-mtp,ngram-mod -> freeze. SPEC OFF -> no freeze.
Culprit: the speculative-decode / context-checkpoint path in llama-server with multiple slots on the Vulkan backend. Known bug family: PR #24891, #24797 (server checkpoint fixes), #25144 (MTP draft crash), plus a ggml-vulkan hang-under-sustained-compute class.

Resolution: SPEC OFF

SPEC_TYPE=none, PARALLEL=2. All concurrent streams complete (EOS). Speed cost: ~10-13 t/s under contention vs ~28-32 t/s single-stream with MTP. Fallback chain if spec-off regresses: A) PARALLEL=1 keep MTP, B) switch back to Qwen3.6-27B (frank-dragoon, MTP n_max 3).

Files

scripts/frank-dragoon-ling.service - systemd unit.
scripts/serve-ling-flash.sh - serve script (fork binary, Vulkan, draft-mtp).
scripts/frank-dragoon.ling.env - tunables.

Passwordless management is installed: /usr/local/sbin/frank-restart-ling.sh and /etc/sudoers.d/frank-ling.

Switch Back (obsolete)

The switch-back commands below are obsolete. At retirement time frank-dragoon ran the Qwen3.8-27B pipeline-parallel head. To restore a Ling trial, re-enable frank-dragoon-ling from the repo unit (scripts/frank-dragoon-ling.service) and serve script (scripts/serve-ling-flash.sh), then start it on a non-conflicting port.

Caveats

The bailingmoe3 fork validates CPU and CUDA upstream. Vulkan was validated here (load + generation + concurrent streams) on 2026-08-14.
The tokenizer logs special_eos_id is not in special_eog_ids at load. It is a warning; generation stops correctly at EOS.
Physical RAM pressure stays low because the model fits in the UMA carve-out. Generation speed on this APU is below the Qwen3.6-27B MTP config (~7.7 vs ~15-19 t/s at 4k).
The heap is effectively full (98.7 of 111.2 GiB). Do not raise the context much without checking --list-devices free MiB.

The other path: vLLM-ROCm (never shipped)

A deployment plan existed for jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native (INT4 W4A16, ~62 GB) served by vLLM-ROCm on Dankclaw. It never shipped; the llama.cpp Q6_K fork trial above is what actually ran. The decision table the plan carried is in git history.

4.8 KiB Raw Blame History