Fetches into /home/dank/models/Qwen3.8-Flash-Next-GGUF/ keeping the Hub
layout: the six Q8_0 split parts (175.30 GiB — never concatenated, part
1's header advertises file_count=6), the shared Q8_0 MTP drafter head
(2.6 GiB — "shared" borrows the target's embeddings; unsloth's MTP guide
names it the recommended head), and the BF16 vision projector. Total
191,919,144,448 bytes.
Sizes and sha256s captured from the resolve endpoint's x-linked-etag/
x-linked-size on 2026-09-26; every artifact is checked for exact byte
count AND sha256, curl -f so an auth error fails instead of writing the
error body, resumable via -C -. The repo is public: no token machinery.
The frank-medium retirement tail the 2026-09-26 lucebox cutover left
uncommitted: the radiance quadlet tree, its swap/fetch tooling and its
seam test are deleted (archive copies live in dankserver/.AGENTTRASH/
20260926-frank-medium-radiance/ and scripts/.AGENTTRASH/20260926-
frank-medium-radiance/), the planner/allowlist/test-changed mappings
drop the service, and frank-large's chat template moves to scripts/
(its old home was the retired frank-medium dir).
Finished here:
- tests/frank_primary_unit.bats: the lucebox quadlet seam (boot
persistence, host-network bind, HIP_VISIBLE_DEVICES=0,1, HIP-reserved
env-name guard, checkout-mounted entrypoint/healthcheck, first-start
patience, deployer payload/port gates).
- test-changed.sh: explicit .AGENTTRASH/ case — retirement commits move
files to trash, and the wrapper refused those additions with "no
focused-test mapping".
- observability: the vllm: namespace left with the radiance runtime.
Rules keep only llamacpp:* series (prefix-cache reuse, spec-decode,
queueing, endpoint-down); the dashboard drops the vLLM KV/latency
panels; prometheus.yml documents that frank-primary (lucebox) has no
/metrics and therefore no scrape job. llm-fleet.json was left
corrupted mid-edit (missing opening brace) and is rewritten from HEAD.
Suite: 339 green (was 7 red).
The runtime's auto budget is gpu_free - kv - warm - safety (22.35 GiB), which
leaves only the 0.50 GiB safety margin free while the fit check demands the
1.50 GiB discrete headroom, so auto-sizing refuses on any 32 GiB target:
"device 0 memory: 30.97 of 31.86 GiB used, 0.89 GiB free (headroom 1.50
GiB): does not fit". 18000 MB is the value DS41.md measured its R9700 rows
at: ~1500 experts resident, ~5 GiB free on device 0.
--ds4-prefill dense made requires_monolithic_model() true, and a monolithic
load asks ggml for the entire 187 GiB expert set as one device-0 buffer:
"allocating 191932.25 MiB on device 0: cudaMalloc failed: out of memory",
followed by "monolithic HIP load required for dense prefill". Approximate
prefill (dense, sparse) is expressible only where the experts are resident
(in-process owners). exact feeds the prompt token by token, which is the
path the hybrid tier streams experts for. Chunk follows DS41.md's
cold-expert recipe (1024).
wc -c falls back to reading the whole stream when its fstat shortcut does
not apply, and this gate runs over a 377 GiB GGUF: the 2026-09-26 cutover
deploy sat in wc for minutes (30 s of CPU, NVMe-bound) before it could
reach the cutover at all. stat -c %s is metadata-only and equally exact.
The llama.cpp hunk (ggml_cuda_node_launches_nothing) was needed on main's
tree; the branch restructured that file and compiles clean, so only the
apt-list and cmake-3.31 hunks remain. Regenerated from the host clone.
The deepseek41 architecture exists only on Luce-Org/lucebox's
feat/ds41-engine branch; main rejects the checkpoint with
"unsupported model architecture 'deepseek41'". Re-aim the image build at
that branch and fix the launch shape, which had been copied from the V4
DS4.md recipe:
- drop --ds4-fused-decode and --ds4-fused-verify-f16-kv, and use
--ds4-prefill dense: V4.1's validate_model_features refuses the V4-only
prefill/fused paths (layer split and --mmproj too).
- --chunk 4096 (the V4.1 dense-prefill measurement point), --max-ctx
131072 (the V4.1 default window), --prefix/--prefill-cache-slots 0 as
the DS41.md bring-up recipe.
- drop --cache-type-k/v: the DeepSeek4 backend ignores them; the MLA
compressed caches are F16 by design.
- expose BOTH R9700s and set LUCE_EXPERT_STREAM_DEVICE=1, so hip:0 owns
the dense weights plus the hot expert stack while hip:1 holds the
streamed-expert slot cache instead of competing with it.
- quadlet: 384 routed experts per layer, Engram tables, 40 layers.
Replaces frank-medium (vllm-radiance TP=2) on :7770. The 377.2 GiB
43-layer MoE (256 routed experts, top-6) is mmapped: dense work and the
MLA KV cache on the R9700, routed experts computed on CPU from the
page cache (246 GiB RAM). The unit bypasses the image entrypoint
(Qwen/DFlash-only flag set) and execs luce_server with the DS4 launch
shape; --ds4-fused-decode's monolithic-load fallback IS the serving
mode here. deploy_llm_primary gates on image + model byte count,
stops/masks the radiance unit, waits for :7770, installs and starts.
Runbook: frank-primary-lucebox-ds4.md.
With a missing file, -C - sends Range: bytes=0-, which HF's xethub
CDN rejects with 400. Xet-backed repos (e.g. Lucebox/DeepSeek-V4.1-
Flash-ROCMFP23-GGUF) fail on the first fetch; the old LFS backend
tolerated the zero-offset range.
Dim the spec line (--dim) so the card reads pale role -> dim spec -> bright
spring-green inputs -> dim use. The inputs line now pops as the bright middle
tier and the spec/use are de-emphasized as reference detail.
Add --input: #00ff9d to the palette and point .desc-inputs at it, so the
Inputs line is visually distinct from the bright-green spec (--fg), the
pale-green role (--head), and the dim-green use (--dim).
Each LLM card now lists its accepted inputs as a line (Inputs: text · image ·
audio), sourced from the harness models.yml:
- frank-large: text (no vision projector)
- frank-medium: text, image (Gemma4 vision path)
- frank-small: text, image, audio (Gemma4 projector audio encoder; audio is
not in models.yml's input list only because OMP's schema rejects it, but the
endpoint accepts input_audio parts)
New .desc-inputs CSS + bare-mode reset; /docs model guide gains an Inputs line
per model. bats 30/30 + live render verified.
Each LLM card now carries a details object (role/spec/use) rendered as a
bold pale-green role, a bright-green spec line, and a dim 'Use:' line.
app.js descHTML() falls back to the plain description for non-LLM cards;
new .desc-role/.desc-spec/.desc-use CSS + bare-mode reset.
Roles made neutral (no nicknames): frank-large = deep reasoning,
frank-medium = primary model, frank-small = fast utility model.
The /docs model guide was fixed — it had drifted to retired model info
(frank-large as Qwen3.8-Flash-Next 87.3 GiB/200k, frank-medium as FP8
~31 GiB) — now Qwen3.8-27B UD-Q8_K_XL 31.5 GiB/262k and MXFP4/W4A8
19.4 GiB, reordered large->medium->small. bats 30/30 + live render.
frank-large: cybersecurity + debugging (requires Frank's permission)
frank-medium: primary intelligent model (surpasses Claude Sonnet 5 per
Artificial Analysis) — software engineering, sysadmin, instruction tasks
frank-small: fast/cheap utility model — agentic search, RAG, research
subagents, translation, roleplay, summarization, small utility tasks
Updated both the status cards (services.json) and the /docs model guide.
The auth works via the modern /Users/AuthenticateByName endpoint (what the
web UI uses). The 401 was a test artifact from probing the dead legacy route.
No config change needed; LDAP-Auth plugin + authentik 2026.8.3 outpost healthy.
The UserWriteStage writes prompt_data['password'] via user.set_password().
A PasswordStage only authenticates the current password. Document the
correct mechanism (password field on the default-user-settings prompt +
empty-password guard policy) and the throwaway-user verification.
The default-user-settings-flow was missing a PasswordStage, so users could
edit name/email/locale but not their password. Added a PasswordStage
(InbuiltBackend) bound at order 101. Documented the flow's stage table and
the two ORM gotchas (MTI through model, required backends array).
The local username/password path is retired (ENABLE_LOGIN_FORM +
ENABLE_PASSWORD_AUTH false, ui.enable_login_form DB override false) and
qa-e2e-admin is out of the authentik Admins group. Documented in the SSO
section and History.
The authentik OIDC round-trip is verified, so retire the local
username/password path. ENABLE_LOGIN_FORM hides the form and
ENABLE_PASSWORD_AUTH refuses password logins; the only way in is now
the OIDC flow, which folds into the local account by email. The
ui.enable_login_form config-DB override is flipped to false in the
same change (the DB wins over env, so both must agree).
Footer link opens the authentik default-user-settings-flow in a new tab,
giving users a self-service entry point to change their password and
manage MFA/credentials without admin intervention.
systemd splits unquoted Environment= values on whitespace, so the
quadlet quotes OAUTH_SCOPES to keep all four scopes; update the bats
assertion to expect the quoted form (it was checking the old
unquoted line and failing).
Records the LDAP-Auth plugin wiring: the Jellyfin LDAP provider +
outpost on Dankclaw, the svc-jellyfin-ldap service account, the
jellyfin_users/jellyfin_admins access groups, and the key gotcha that
the bind credential is a PASSWORD (set via Recovery), not an API/app
token — the default auth flow's standard-password backend rejects
tokens with 'flow error password: Invalid'. Verified end-to-end:
alex (in jellyfin_users) logs in with their authentik password and the
plugin creates the matching Jellyfin user.
systemd splits unquoted Environment= values on whitespace, so
'OAUTH_SCOPES=openid email profile roles' arrived as 'openid' and the
authorize request carried a single scope — the id_token then lacked the
email claim and the callback failed. Quoted, all four scopes flow.
Also: assign the default openid/email/profile ScopeMappings to the
provider (2026.x no longer auto-creates them; without them the token
grant is the intersection with the provider's mappings = roles only,
and userinfo 403s).
fill-verify-Enter with a single inputValue read per stage (identification
and password); the toHaveValue assertion hard-failed every attempt when the
re-render wipe landed inside its 5s window (00-auth.setup failure 2026-09-24)
Boot 2026-09-22 at 0.98 fill: KV pool auto-sized to 425,047 tokens (1.62x
the 262,144 window) with 14.42 GiB per card available for KV. The DFlash2
drafter on the Swift finetune measures mean acceptance length 7.00 / 85.7%
(easy) and 5.08 / 58.3% (varied) — it survives the base-model change.
Replaces the estimated ~420k / ~17.6 GiB figures with the measured values.
0.99 failed the vLLM startup memory check before weight load: the R9700
reports 31.86 GiB total but only 31.23 GiB free at device-init (the ROCm/HIP
context + NCCL + radiance custom all-reduce consume the 0.63 GiB gap), so
0.99 (31.54 GiB) exceeded free and the container crash-looped. 0.98 (31.22
GiB) is the ceiling and the prior production value.
Also fixed the runbook KV math that had shipped with the 0.99 change: the
~730k pool estimate was impossible (MXFP6's 25.16 GiB weights > MXFP4's
19.37 GiB, so the pool is smaller, not larger than MXFP4's 486k) and the
34.2 GiB/card figure was not the R9700 total. Recomputed ~420k tokens at 0.98
fill.
The Qwen-Image 2.1 VAE is a video VAE (5D convs, time_conv, depth-3).
ComfyUI detects it as WanVAE (latent_dim=3), but EmptyLatentImage produces
a 4D latent. The decode function's memory_used_decode lambda accesses
shape[4] which is out of range for 4D.
Fix: expand 4D latents to 5D in the decode function when latent_dim==3.
Added to the patch script for persistence across container recreations.
The serve script now passes --chat-template-file pointing at the
Qwen-Sharp v22.4.0 template. Without it the binary falls back to the
built-in peg-native template, which produces a ~73,600-token prompt for
a ~250-token request. With the sharp template wired in, the same request
produces a 62-175-token prompt.
The serve script only passed --chat-template-kwargs, so the binary fell
back to the built-in peg-native template. A 250-token request produced a
73,600-token prompt (280x the expected ~260). Wiring in --chat-template-file
drops the prompt to 62 tokens. Draft acceptance 0.79 (mean 4.14), model
answers correctly, reasoning lands in reasoning_content.
The qwen_image21 detection check and GGUF node arch list patch are
applied by a startup wrapper (comfyui-start.sh) that runs before
ComfyUI starts. The patch script and wrapper are baked into the image
so they survive container recreations. The quadlet's Exec now points
at the wrapper instead of main.py directly.
Frank
