Commit Graph
1022 Commits
Author SHA1 Message Date
Frank b06895d725 frank-primary: cutover state to Qwen3.8-Flash-Next Q8_0 (6 shards, PLE pinned, MTP4, sharp template)
- env: STATE_FILE -> r9v-state/q8xl; add dev-overlay adapter mount
  (quantized inject shard dequant, fork 363d606), FULL_MUTABLE_CACHE=0
  (80-slot tiered vs the overlay's uncensored 160-slot injection), and
  the qwen-sharp-v22.4.0.jinja chat template via the launcher's new
  R9V_CHAT_TEMPLATE_FILE/KWARGS passthrough (fork 30d4a32)
- serve/deploy: shard gates extended to SHARD4/5/6 (unset-safe, the
  uncensored rollback state carries none)
- seams: STATE_FILE/LLM_PRIMARY_STATE -> q8xl
2026-09-27 22:25:08 -07:00
Frank 928fd81dbd docs(log): record Q8_0 bring-up qualification and uncensored restore posture 2026-09-27 18:19:56 -07:00
Frank efff47a75d AGENTS.md: frank-primary section rewritten for the R9V cutover
Fleet topology row + full endpoint section now describe the R9V
radiance shape (on-host 104-key state, shim laws, launcher-driven
serve, docker-wait self-healing, measured numbers); llama.cpp section
retired into the runbook pointer. Bench note: mask fails on a relinked
unit symlink - stop is enough, redeploy restores.
2026-09-27 04:39:30 -07:00
Frank ca6ba6d0e9 frank-primary: R9V runbook + docs cutover
New ACTIVE runbook (setup/doctor PASS=34/FAIL=0, UVA transport verified
in serving workers, wire-verified alias/reasoning/metrics), index +
vault entries flipped: llama.cpp b11030 endpoint RETIRED 2026-09-27 to
rollback target. Topology diagram updated.
2026-09-27 04:35:05 -07:00
Frank 7562e41ba4 frank-primary: retire the FP8 fetch path with the vLLM 0.30 wiring
download-frank-primary-fp8.sh + the fp8 manifest fetched the vLLM 0.30
payload that never deployed; the R9V cutover's payload is the R9V setup
flow's (packages + PLE, state-gated). Scripts move to old/ per policy.
2026-09-27 04:21:47 -07:00
Frank 095128b669 frank-primary: R9V radiance cutover wiring (launcher-driven serve + state gates)
Replace the vLLM 0.30 UVA-offload wiring (never deployed; llama.cpp
b11030 endpoint replaced by Frank's order, downtime window open) with
the R9V radiance-vLLM shape on the same root unit:

- serve-frank-primary.sh: layer the on-host qualified setup state
  (104 tuned R9V_* keys, /home/dank/r9v-state/uncensored/setup.json)
  under repo production overrides, then exec the fork's own launcher
  through the docker->podman shim and block on docker wait so a crashed
  engine is a nonzero ExecStart (launch.sh --detach alone would fake
  green). Gates: pinned image by OCI manifest digest in RepoDigests,
  PLE at pinned byte count (stat -c %s), all GGUF shards + mmproj.
- frank-primary.env: thin production overrides only (LAN bind 0.0.0.0,
  :7770, alias frank-primary, container frank-primary-r9v, GPU BDFs);
  tuned engine shape stays in the setup state by design.
- frank-primary.service: same root-unit shape (User=dank,
  LimitMEMLOCK=infinity for the tiered-expert pin), plus ExecStop
  podman stop — docker wait cannot stop the container child.
- deploy-remote.sh deploy_llm_primary: state-driven gates (state ready,
  image digest present in rootless storage, PLE bytes, shard files)
  replace the fp030 shard-index gate; false-green MainPID check kept.
- tests/frank_primary_unit.bats: seam rewritten for the R9V shape.
2026-09-27 04:05:46 -07:00
Frank baba84fc90 frank-primary vllm: orphan-container self-heal + HOME + stop/rm order
Unit restart SIGKILLs conmon before --rm cleanup runs, leaving the container
name registered: every restart then dies on name-in-use while the orphan
holds the port and GPUs (observed live 2026-09-27, NRestarts=6 with the
container Up and the unit auto-restarting). The serve script now clears its
own name before exec podman run; the deployer stops the unit and removes
the container BEFORE waiting on the port.

HOME inside the container pointed at the root-owned WORKDIR /app, so
vLLM's registry cache write hit PermissionError; HOME now points at the
cache volume.

The vLLM cutover itself is blocked on a structural wall (AMD Qwen4Exp impl
creates the 51.2B PLE table as a bf16 on-GPU param: 47.69 GiB transient on a
31.86 GiB card; the pinned-host UVA path that fits this hardware is
CUDA-gated). llama.cpp is restored live at :7770; host HEAD unchanged.
2026-09-27 01:08:21 -07:00
Frank ecaad36654 frank-primary: vLLM 0.30 UVA-offload shape (pre-deploy commit)
The llama.cpp b11030 trio becomes the vLLM 0.30 trio ahead of the
FP8 cutover on :7770 — same port, same alias, no client moves.

Serve shape: locally built ROCm gfx1201 image
(localhost/frank-primary:vllm030) execs via podman as dank from the
root unit (LimitMEMLOCK=infinity legalizes the pin). UVA offload:
--offload-backend uva pins the PLE family (47.75 GiB, 138 tensors:
131 n-gram table shards + 7 per-layer conv/proj) + routed experts
(114.86 GiB, 150,528 tensors) in host RAM — 162.60 GiB combined,
~81.3 GiB per rank under TP=2, hence --cpu-offload-gb 82 (80/card
strands 2.6 GiB of experts on the cards). Parsers per the host's
own Ornith-era precedent: qwen3_xml tools + qwen3 reasoning (vLLM
0.30 emits the channel as `reasoning` — reasoning_content is
renamed server-side) + enable_thinking/preserve_thinking template
kwargs. Spec decode stays env-wired and OFF at bring-up.

Deployer deploy_llm_primary rewritten: image gate, index-driven
shard-count gate (131), stale-container cleanup, port wait, and the
false-green hardening — health is only green when the unit's own
MainPID is live behind the answer (2026-09-26 lesson).

Seam tests rewritten to the vLLM contract (15 gates): offload
arithmetic, parser wiring, no-llama.cpp-ism, deployer gates. Suite
349/349 green.
2026-09-27 00:16:42 -07:00
Frank 82960c77db frank-primary: add FP8 payload manifest + fetch wrapper
scripts/manifests/qwen3.8-flash-next-fp8.manifest: 144 entries (131
safetensors shards + aux) with Hub LFS sha256s for
Qwen/Qwen3.8-Flash-Next-FP8. download-frank-primary-fp8.sh wraps
hf-fetch.sh: idempotent fetch, full sha256 re-verify of an on-disk
payload (verified live on dankserver: all 144 files match, ~10 min).

Part of the frank-primary vLLM cutover prep; the serve trio itself
lands with the image deploy.
2026-09-26 23:50:58 -07:00
Frank 7a8d992fce searxng: wire paid Kagi engine with deploy-time key rendering
The stock SearXNG image ships a native kagi engine; enabling it needs an
API key that must never enter git, and SearXNG does not expand env vars
in settings.yml (the live secret_key literally was '${SEARXNG_SECRET}'
until now — the app tolerated the placeholder string silently).

- settings.yml: kagi engine enabled (weight 2, kg shortcut, paging off so
  exactly one API credit per aggregated query, timeout 8.0 vs 5.0 scrapers
  so a slow paid query is not a wasted credit); api_key stays ${KAGI_API_KEY}.
- new scripts/searxng-render-settings.py: deploy-time renderer, allowlisted
  to KAGI_API_KEY/SEARXNG_SECRET/SEARXNG_BASE_URL, fails the deploy loudly
  on an unresolvable placeholder, idempotent on redeploys.
- deploy-remote.sh copy_searxng_config runs the renderer against
  /opt/searxng/.env after copying settings.yml into /opt/searxng/config.
- KAGI_API_KEY placeholder added to dankclaw/.env.example (real key staged
  into the host-side /opt/searxng/.env, mode 600, out of band).
- bats: engine wiring, renderer substitution, loud-failure, deployer seam.
- docs: SearXNG.md Kagi section + renderer law; Providers.md carve-out on
  the no-credentials policy (paid engine acceptable when bounded + ordered).

Verified live on dankclaw: direct API probe via the gluetun netns (200,
data.search[] shape), then an aggregated query returned 29/29 results
tagged kagi, zero unresponsive engines.
2026-09-26 23:25:55 -07:00
Frank 4992e5a010 hf-manifest.py: follow the 307 resolve-cache redirect for non-LFS files
The first no-redirect HEAD fix read X-Linked-Size on every 3xx, but
non-LFS files on a Xet-backed repo answer 307 to HF's own
/api/resolve-cache WITHOUT any X-Linked-Size header. The fallback to the
stub's Content-Length recorded the redirect page size (254-264 bytes)
instead of the file size, so hf-fetch.sh failed the size check on every
aux file it verified against the regenerated manifest (README.md
"65400 != 254", config.json "72514 != 258", 7 files on the FP8 repo).

Distinguish the two redirect shapes: 3xx WITH X-Linked-Size = LFS/Xet,
whose X-Linked-* headers are the truth and whose redirect must NOT be
followed (the Xet bridge ETag is the CAS hash, not the sha256); 3xx
WITHOUT = non-LFS, where the Location (followed once, via urljoin)
answers 200 with the real Content-Length and a git-blob-sha ETag that
the 64-hex gate correctly rejects (size-only check for those files).

Verified against Qwen/Qwen3.8-Flash-Next-FP8: all 7 previously-failing
aux files now record their on-disk byte counts exactly (README.md 65400,
config.json 72514, merges.txt 3353259, ...), shard shas unchanged
(model-00002 6841fe21...), total 172.82 GiB across 144 files.
2026-09-26 22:50:17 -07:00
Frank 5bd12d3193 hf-manifest.py: read X-Linked-* headers, never follow the resolve redirect
On Xet-backed repos (Qwen/Qwen3.8-Flash-Next-FP8), the resolve endpoint 302s
to a CDN bridge whose final ETag is the Xet CAS chunk-dag hash, NOT the
content sha256. urllib follows redirects, so the manifest recorded the Xet
hash and every downloaded shard failed sha256 verification against a hash no
download could ever match (8/8 shards FAIL, all sizes correct, on-disk bytes
matching the Tree API's lfs.oid). Classic LFS CDNs serve ETag=sha256, which
is why this was invisible on prior fetches.

Fix: HEAD via http.client, which does not follow redirects. 200 = direct
answer (Content-Length + ETag, non-LFS); 302 = LFS/Xet redirect
(X-Linked-Size + X-Linked-ETag = content sha256). Verified against the FP8
repo: shard model-00002-of-00131 now records 6841fe21..., the sha256 of the
bytes already on disk (cross-checked against the Tree API lfs.oid).
2026-09-26 22:46:24 -07:00
Frank f41e0384dd frank-primary: measure expert routing; block choice is immaterial
Instrument the live server to answer "can -ot target the most active
layers": AMD IBS op sampling on llama-server during 4 mixed concurrent
generations, sampled load addresses mapped through /proc/<pid>/maps to
backing tensors with extents derived from consecutive GGUF header
offsets (no quant type-size tables - a wrong per-element size silently
mis-buckets tensor tails, first run lost 59% of expert samples to a
junk bucket).

Measured (75 s window, 2.7M expert-tensor samples): per-block read
volume is FLAT, 0.93x-1.08x of mean across all 40 CPU-resident blocks.
MoE top-k routing fires every block every token, so "most active
layers" do not exist at -ot granularity - any 8 GPU-resident blocks
are equivalent, the lever is COUNT, already at the 5/3 VRAM ceiling.
Real skew is per-EXPERT (top 10% of 48x512 cells carry 40.5% of
samples), below tensor-override granularity. No config change.

scripts/frank-primary-expert-activity.py: dump + sample + analyze in
one entry point; --analyze-only re-analyzes saved samples. Instruments
production serving in place; not a benchmark.
2026-09-26 21:58:03 -07:00
Frank d67768e1a5 frank-primary runbook: 5/3 block fill is the deployed shape 2026-09-26 21:40:25 -07:00
Frank d317d6b246 frank-primary: 5th expert block onto ROCm0 (operator-directed fill)
Blocks 0-4 on ROCm0 + 45-47 on ROCm1. This exact shape was probed and
battery-verified earlier the same session: card0 2.54 GiB free at boot,
2.11 GiB after the vision graph (steady under load), card1 unchanged
at ~3.3 GiB. Card1 cannot take another block (0.82 GiB would OOM the
vision graph + boot drift), so 5/3 is the physical limit at the
4x262144 / V-q8_0 shape. OT assertion pins the new value.
2026-09-26 21:38:05 -07:00
Frank e330d56a06 frank-primary runbook: record the 4-slot/V-q8/GPU-block reshape 2026-09-26 21:32:09 -07:00
Frank de85654b76 frank-primary: four full-window slots, V q8_0 KV, hot expert blocks on-GPU
- CONTEXT_SIZE=1048576 / PARALLEL_SLOTS=4: four conversations, each the
  full native 262144 window (per-slot window unchanged; slots divide
  the pool, they never multiply it).
- CACHE_TYPE_V=q8_0 / CACHE_TYPE_K=f16: main KV drops 24,576 -> 18,816
  B/token (-23.4%); the shared MTP drafter's KV ignores the V type
  (measured: 2,048 MiB draft pool = the f16 law at 1,048,576 cells).
- OT_OVERRIDES: blocks 0-3 on ROCm0 and 45-47 on ROCm1 stay
  GPU-resident (2.42/2.49 GiB per block, measured); the rest stay
  CPU-computed. Measured free under load: ~4.9 GB card0 / ~3.4 GB
  card1; a 5th block on card0 lands at 2.27 GB, under the 3 GB
  checkpoint floor, so blocks are atomic and 4 is the safe fill.
- serve script: --cache-type-k/v env-wired tunables; ONE joined
  --override-tensor flag (b11030 deprecates repeated flags, and the
  loader is first-match-wins, so the specific patterns must precede
  the broad exps=CPU sweep).
- seams: the window test asserts per-slot = 262144 at 4 slots; new
  KV-type test; OT exact-value assertion pins the ordering law.
2026-09-26 21:29:21 -07:00
Frank de4d22785d frank-primary runbook: record the KV law and per-ctx VRAM costs
29,696 B/token ctx-scaled (12 of 48 layers carry KV - full_attention_interval=4;
indexer and draft KV included); weights/RS/compute are ctx-independent.
262k = 18.75 GiB total / 9.37 per card; 100k = 14.3 / 7.1. Parallel
slots split the pool rather than multiplying it.
2026-09-26 21:01:12 -07:00
Frank 82ff9dda20 frank-primary + frank-large: export /metrics, raise primary log verbosity to 4
Both b11030 serve scripts never passed --metrics, so /metrics answered
501 and Prometheus could not scrape frank-large (silently down since
its deploy) or frank-primary. frank-primary also gets -lv 4 (VERBOSITY
env tunable, default 4) for the full per-buffer load breakdown.
2026-09-26 20:58:53 -07:00
Frank bacabdcd49 frank-primary: record measured battery numbers and the drafter-placement law
Battery 6/6: alias, reasoning_content, tool_calls, vision (probe image
described exactly), reasoning_effort high -> 200. Decode 11.8 t/s at
depth-0 prose (draft 109/accepted 72), 0.864 mean len 2.73 on tool
traffic. VRAM 29%/36% (output-head card carries the drafter). Also
records the ROCm1 drafter law in the runbook body (borrowed output.weight
+ n_copies==1 abort) and the two cutover traps: foreign-process health
false green, and llama-server SIGTERM cleanup hang.
2026-09-26 20:55:02 -07:00
Frank 5d632bbaea frank-primary: pin the drafter to ROCm1 (the last-layers card)
The shared MTP head borrows the main model output head, which a -ts 1,1
layer split places on ROCm1. With tensor overrides set, n_copies==1 in the
draft's scheduler, and a borrowed output.weight in a ROCm1 buffer aborts the
draft graph (ggml-backend.cpp:941) when the draft context is on ROCm0 —
the unit crash-looped 4x at ~12 s per attempt during the 2026-09-26 cutover.
Probed live on the host: ROCm0 aborts identically, ROCm1 loads and serves.
2026-09-26 20:49:04 -07:00
Frank a9b94a79eb frank-primary: pin the -ot override set to the real tensor names
Discovery, not guesswork: dump the GGUF v3 headers of the fetched parts
(scripts/dump-gguf.py, added — a dependency-free reader that never
touches tensor data, so a 50 GiB part scans in milliseconds) before
the first load.

What the dump says (parts 2-3, 2026-09-26):

- The 51B-param PLE n-gram table is ONE tensor: per_layer_token_embd.
  weight, dims [160, 320001536], Q8_0, 54.4 GiB — exactly the whole of
  part 3. It is the same tensor the shared MTP head borrows, so the
  =CPU override is required for placement AND for the draft load
  (GPU-resident borrow: "invalid vector subscript").
- The model is qwen4exp, 512 experts, top-10 routing
  (qwen4exp.ple.ngram_size=3, heads_per_ngram=8).
- Part 1 is metadata-only (ntensors=0); parts 2-6 carry tensors.

The placeholder pattern ngram_embeddings matched nothing — dropped from
OT_OVERRIDES (env, serve-script default, seam, AGENTS.md, runbook,
LLM index) before it could masquerade as a working override at load
time. Deployed set: exps=CPU,per_layer_token_embd.weight=CPU.

Suite: 346 green.
2026-09-26 19:51:57 -07:00
Frank 8500600d09 frank-primary: swap Dankserver to llama.cpp Qwen3.8-Flash-Next Q8_0
The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).

The deployed contract:

- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
  and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
  -ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
  over PCIe loses at 3.7 t/s, computing in RAM does not care).
  per_layer_token_embd.weight=CPU is required by the shared MTP head's
  borrow path (a GPU-resident borrow crashes the draft load).
  mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
  (mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
  draftless ngram-mod matcher chained behind it — the only legal way to
  layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
  autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
  Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
  by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
  with Qwen-recommended sampling riding compat.extraBody.

The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.

Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.

Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).

models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].

Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).

Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.
2026-09-26 19:45:28 -07:00
Frank d89813363e add frank-primary payload fetch (Qwen3.8-Flash-Next Q8_0 + MTP head + mmproj)
Fetches into /home/dank/models/Qwen3.8-Flash-Next-GGUF/ keeping the Hub
layout: the six Q8_0 split parts (175.30 GiB — never concatenated, part
1's header advertises file_count=6), the shared Q8_0 MTP drafter head
(2.6 GiB — "shared" borrows the target's embeddings; unsloth's MTP guide
names it the recommended head), and the BF16 vision projector. Total
191,919,144,448 bytes.

Sizes and sha256s captured from the resolve endpoint's x-linked-etag/
x-linked-size on 2026-09-26; every artifact is checked for exact byte
count AND sha256, curl -f so an auth error fails instead of writing the
error body, resumable via -C -. The repo is public: no token machinery.
2026-09-26 19:28:09 -07:00
Frank d7bf9f119a retire frank-medium: tests, planning and observability ground on frank-primary
The frank-medium retirement tail the 2026-09-26 lucebox cutover left
uncommitted: the radiance quadlet tree, its swap/fetch tooling and its
seam test are deleted (archive copies live in dankserver/.AGENTTRASH/
20260926-frank-medium-radiance/ and scripts/.AGENTTRASH/20260926-
frank-medium-radiance/), the planner/allowlist/test-changed mappings
drop the service, and frank-large's chat template moves to scripts/
(its old home was the retired frank-medium dir).

Finished here:
- tests/frank_primary_unit.bats: the lucebox quadlet seam (boot
  persistence, host-network bind, HIP_VISIBLE_DEVICES=0,1, HIP-reserved
  env-name guard, checkout-mounted entrypoint/healthcheck, first-start
  patience, deployer payload/port gates).
- test-changed.sh: explicit .AGENTTRASH/ case — retirement commits move
  files to trash, and the wrapper refused those additions with "no
  focused-test mapping".
- observability: the vllm: namespace left with the radiance runtime.
  Rules keep only llamacpp:* series (prefix-cache reuse, spec-decode,
  queueing, endpoint-down); the dashboard drops the vLLM KV/latency
  panels; prometheus.yml documents that frank-primary (lucebox) has no
  /metrics and therefore no scrape job. llm-fleet.json was left
  corrupted mid-edit (missing opening brace) and is rewritten from HEAD.

Suite: 339 green (was 7 red).
2026-09-26 19:25:46 -07:00
Frank 32d264ccda frank-primary: size the hot expert stack explicitly (LUCE_EXPERT_BUDGET_MB)
The runtime's auto budget is gpu_free - kv - warm - safety (22.35 GiB), which
leaves only the 0.50 GiB safety margin free while the fit check demands the
1.50 GiB discrete headroom, so auto-sizing refuses on any 32 GiB target:
"device 0 memory: 30.97 of 31.86 GiB used, 0.89 GiB free (headroom 1.50
GiB): does not fit". 18000 MB is the value DS41.md measured its R9700 rows
at: ~1500 experts resident, ~5 GiB free on device 0.
2026-09-26 17:13:05 -07:00
Frank dcdf044aa1 frank-primary: exact prefill, the only mode the streamed tier runs
--ds4-prefill dense made requires_monolithic_model() true, and a monolithic
load asks ggml for the entire 187 GiB expert set as one device-0 buffer:
"allocating 191932.25 MiB on device 0: cudaMalloc failed: out of memory",
followed by "monolithic HIP load required for dense prefill". Approximate
prefill (dense, sparse) is expressible only where the experts are resident
(in-process owners). exact feeds the prompt token by token, which is the
path the hybrid tier streams experts for. Chunk follows DS41.md's
cold-expert recipe (1024).
2026-09-26 17:11:31 -07:00
Frank de513ab080 deploy-remote: stat the frank-primary payload, do not wc it
wc -c falls back to reading the whole stream when its fstat shortcut does
not apply, and this gate runs over a 377 GiB GGUF: the 2026-09-26 cutover
deploy sat in wc for minutes (30 s of CPU, NVMe-bound) before it could
reach the cutover at all. stat -c %s is metadata-only and equally exact.
2026-09-26 17:08:13 -07:00
Frank 124175218f frank-primary: patch is Dockerfile-only on feat/ds41-engine
The llama.cpp hunk (ggml_cuda_node_launches_nothing) was needed on main's
tree; the branch restructured that file and compiles clean, so only the
apt-list and cmake-3.31 hunks remain. Regenerated from the host clone.
2026-09-26 17:03:11 -07:00
Frank 846140c2be frank-primary: V4.1 launch shape (feat/ds41-engine source)
The deepseek41 architecture exists only on Luce-Org/lucebox's
feat/ds41-engine branch; main rejects the checkpoint with
"unsupported model architecture 'deepseek41'". Re-aim the image build at
that branch and fix the launch shape, which had been copied from the V4
DS4.md recipe:

- drop --ds4-fused-decode and --ds4-fused-verify-f16-kv, and use
  --ds4-prefill dense: V4.1's validate_model_features refuses the V4-only
  prefill/fused paths (layer split and --mmproj too).
- --chunk 4096 (the V4.1 dense-prefill measurement point), --max-ctx
  131072 (the V4.1 default window), --prefix/--prefill-cache-slots 0 as
  the DS41.md bring-up recipe.
- drop --cache-type-k/v: the DeepSeek4 backend ignores them; the MLA
  compressed caches are F16 by design.
- expose BOTH R9700s and set LUCE_EXPERT_STREAM_DEVICE=1, so hip:0 owns
  the dense weights plus the hot expert stack while hip:1 holds the
  streamed-expert slot cache instead of competing with it.
- quadlet: 384 routed experts per layer, Engram tables, 40 layers.
2026-09-26 17:00:46 -07:00
Frank fc472e89d4 frank-primary: track the gfx1201 build patch (apt/cmake/ggml guard) + runbook recipe 2026-09-26 15:44:17 -07:00
Frank bed551baa6 deploy.sh: allowlist dankserver:frank-primary 2026-09-26 15:33:05 -07:00
Frank acd14ab7df frank-primary: Lucebox DeepSeek-V4.1-Flash-ROCMFP23 quadlet + deploy handler
Replaces frank-medium (vllm-radiance TP=2) on :7770. The 377.2 GiB
43-layer MoE (256 routed experts, top-6) is mmapped: dense work and the
MLA KV cache on the R9700, routed experts computed on CPU from the
page cache (246 GiB RAM). The unit bypasses the image entrypoint
(Qwen/DFlash-only flag set) and execs luce_server with the DS4 launch
shape; --ds4-fused-decode's monolithic-load fallback IS the serving
mode here. deploy_llm_primary gates on image + model byte count,
stops/masks the radiance unit, waits for :7770, installs and starts.
Runbook: frank-primary-lucebox-ds4.md.
2026-09-26 15:29:59 -07:00
Frank 441c4f1c99 hf-fetch: only pass -C - when a non-empty partial exists
With a missing file, -C - sends Range: bytes=0-, which HF's xethub
CDN rejects with 400. Xet-backed repos (e.g. Lucebox/DeepSeek-V4.1-
Flash-ROCMFP23-GGUF) fail on the first fetch; the old LFS backend
tolerated the zero-offset range.
2026-09-26 15:23:24 -07:00
Frank c5b55e1e3b frank-medium: GPU_UTIL 0.98 -> 0.97 — the 0.98 startup threshold sits within ~15 MiB of the free memory measured at engine init (worker HIP context + amdgpu per-boot baseline ~690 MiB), so 2026-09-26's boot crash-looped on every restart; 0.97 buys ~160 MiB headroom at ~5k KV tokens 2026-09-26 15:02:20 -07:00
Frank cd07a66939 homepage: flip the model card stagger to dark-light-dark-light
role/inputs now dim (--dim), spec/use now bright (--fg). Removed the
now-unused --input spring-green variable.
2026-09-26 00:21:01 -07:00
Frank d2c4c3f715 homepage: stagger the model card tiers light-dark-light-dark
Dim the spec line (--dim) so the card reads pale role -> dim spec -> bright
spring-green inputs -> dim use. The inputs line now pops as the bright middle
tier and the spec/use are de-emphasized as reference detail.
2026-09-26 00:18:48 -07:00
Frank 6fae19e285 homepage: give the model inputs line its own spring-green tone
Add --input: #00ff9d to the palette and point .desc-inputs at it, so the
Inputs line is visually distinct from the bright-green spec (--fg), the
pale-green role (--head), and the dim-green use (--dim).
2026-09-26 00:13:31 -07:00
Frank ec7677c180 homepage: add per-model input modalities to the LLM cards
Each LLM card now lists its accepted inputs as a line (Inputs: text · image ·
audio), sourced from the harness models.yml:
- frank-large: text (no vision projector)
- frank-medium: text, image (Gemma4 vision path)
- frank-small: text, image, audio (Gemma4 projector audio encoder; audio is
  not in models.yml's input list only because OMP's schema rejects it, but the
  endpoint accepts input_audio parts)

New .desc-inputs CSS + bare-mode reset; /docs model guide gains an Inputs line
per model. bats 30/30 + live render verified.
2026-09-26 00:11:01 -07:00
Frank 57fd42af6a homepage: rework LLM model cards to structured role/spec/use lines
Each LLM card now carries a details object (role/spec/use) rendered as a
bold pale-green role, a bright-green spec line, and a dim 'Use:' line.
app.js descHTML() falls back to the plain description for non-LLM cards;
new .desc-role/.desc-spec/.desc-use CSS + bare-mode reset.

Roles made neutral (no nicknames): frank-large = deep reasoning,
frank-medium = primary model, frank-small = fast utility model.

The /docs model guide was fixed — it had drifted to retired model info
(frank-large as Qwen3.8-Flash-Next 87.3 GiB/200k, frank-medium as FP8
~31 GiB) — now Qwen3.8-27B UD-Q8_K_XL 31.5 GiB/262k and MXFP4/W4A8
19.4 GiB, reordered large->medium->small. bats 30/30 + live render.
2026-09-25 23:51:41 -07:00
Frank a489d8481d docs: log the homepage model-description change (2026-09-25) 2026-09-25 22:57:39 -07:00
Frank d3c819741d homepage: add suggested uses to the three LLM model descriptions
frank-large: cybersecurity + debugging (requires Frank's permission)
frank-medium: primary intelligent model (surpasses Claude Sonnet 5 per
  Artificial Analysis) — software engineering, sysadmin, instruction tasks
frank-small: fast/cheap utility model — agentic search, RAG, research
  subagents, translation, roleplay, summarization, small utility tasks

Updated both the status cards (services.json) and the /docs model guide.
2026-09-25 22:32:56 -07:00
Frank 5df6fd1f97 docs: Jellyfin LDAP 401 resolved — legacy /Users/Auth/Password route removed in 12.1
The auth works via the modern /Users/AuthenticateByName endpoint (what the
web UI uses). The 401 was a test artifact from probing the dead legacy route.
No config change needed; LDAP-Auth plugin + authentik 2026.8.3 outpost healthy.
2026-09-25 22:30:44 -07:00
Frank 58952bd820 Bump authentik LDAP outpost to 2026.8.3 (Jellyfin auth path)
ghcr.io/goauthentik/ldap digest fa33bc8f -> 2e7b259e (2026.8.3 amd64
single-platform digest; podman 5.7.0 rejects the tag's manifest list).
2026-09-25 22:01:56 -07:00
Frank 443dde71f3 Log: fleet-wide container update 2026-09-25 2026-09-25 17:17:17 -07:00
Frank 0b1ea2efb3 Bump pinned images to latest: authentik 2026.8.3, tautulli ls245, calibre-web ls402, plex 1.43.4
- authentik 2026.8.0 -> 2026.8.3 (server, worker, outpost quadlets)
- tautulli v2.18.1-ls242 -> v2.18.1-ls245
- calibre-web 0.6.27-ls397 -> 0.6.27-ls402
- plex 1.43.3.10896-ls322 -> 1.43.4.10903-ls325
- seerr already at v3.4.1 (latest, no bump)
- sync docs + the dead compose.yaml plex entry

anubis is already at the latest stable (v1.27.0); only pre-releases exist
beyond it, so no stable bump. Local-build images (caddy, alloy, loki,
comfyui, hermes) are refreshed by forced --no-cache rebuilds, not pins.
2026-09-25 17:11:20 -07:00
Frank 650dcba0ea docs(authentik): password change is a prompt field, not a PasswordStage
The UserWriteStage writes prompt_data['password'] via user.set_password().
A PasswordStage only authenticates the current password. Document the
correct mechanism (password field on the default-user-settings prompt +
empty-password guard policy) and the throwaway-user verification.
2026-09-24 21:28:14 -07:00
Frank 8108dc4254 docs(authentik): document the self-serve user-settings flow + password stage
The default-user-settings-flow was missing a PasswordStage, so users could
edit name/email/locale but not their password. Added a PasswordStage
(InbuiltBackend) bound at order 101. Documented the flow's stage table and
the two ORM gotchas (MTI through model, required backends array).
2026-09-24 21:16:10 -07:00
Frank 201c8437dd docs(open-webui): record the SSO-only flip + qa-e2e-admin demotion
The local username/password path is retired (ENABLE_LOGIN_FORM +
ENABLE_PASSWORD_AUTH false, ui.enable_login_form DB override false) and
qa-e2e-admin is out of the authentik Admins group. Documented in the SSO
section and History.
2026-09-24 20:56:38 -07:00
Frank 33cdeca41e open-webui: flip to SSO-only (disable local login form + password auth)
The authentik OIDC round-trip is verified, so retire the local
username/password path. ENABLE_LOGIN_FORM hides the form and
ENABLE_PASSWORD_AUTH refuses password logins; the only way in is now
the OIDC flow, which folds into the local account by email. The
ui.enable_login_form config-DB override is flipped to false in the
same change (the DB wins over env, so both must agree).
2026-09-24 20:52:06 -07:00