serverAppConfigs/serverAppConfigs-docs/LLM/frank-primary-r9v-radiance-qwen38-uncensored.md

title, service, host, port, status, deployed, supersedes, tags

title	service	host	port	status	deployed	supersedes	tags
frank-primary — R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS	frank-primary	dankserver	7770	active	2026-09-27	frank-primary-llamacpp-qwen38-flash-next-q8	llm frank-primary r9v vllm radiance gfx1201

frank-primary — R9V radiance-vLLM (dankserver, ACTIVE)

The Dankserver LLM endpoint is the R9V radiance-vLLM fork (drwolfen/radiance-vllm-qwen4exp, docker-pinned prebuilt) serving Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS — the abliterated Qwen3.8-Flash-Next at IQ4_XS (92.39 GiB, 3 GGUF shards) with the blockfp8 MTP4 head and the F16 mmproj — on both R9700s (gfx1201, BDFs 0000:03:00.0 / 0000:07:00.0) as frank-primary on :7770.

Replaced llama.cpp b11030 (Qwen3.8-Flash-Next Q_8) on 2026-09-27 at Frank's direction; downtime window ~1 h 20 min (killed 02:47, healthy 04:11 PDT). Rollback target: frank-primary-llamacpp-qwen38-flash-next-q8 — the llama.cpp trio (unit/env/serve) is at commit 4992e5a+ (the repo-side wiring; host unit was the relinked one) and the GGUF payload stays on disk, so a rollback is a redeploy.

Production shape (repo-tracked)

The repo carries a thin wiring; the engine shape lives on-host:

Piece	File	Law
Unit	scripts/frank-primary.service	Root unit, User=dank, LimitMEMLOCK=infinity, ExecStop=podman stop -t 120 frank-primary-r9v, Restart=on-failure
Env	scripts/frank-primary.env	Production overrides ONLY: bind 0.0.0.0, :7770, alias frank-primary, container frank-primary-r9v, GPU BDFs
Serve	scripts/serve-frank-primary.sh	Layers setup state under repo env → execs the fork's own scripts/launch.sh via the docker→podman shim → docker wait
Deployer	scripts/deploy-remote.sh deploy_llm_primary	Gates on setup state ready, image digest, PLE bytes, shard files; MainPID false-green check

Why the tuned shape lives on-host: the profile's setup run qualified a 104-key R9V_* config (tiered-expert placement, per-rank offload budgets 146.25/78.75 GiB, CED on, MTP4, parser wiring, MAX_MODEL_LEN=196608). It is written to /home/dank/r9v-state/uncensored/setup.json and the serve script layers it UNDER the repo env (repo keys win). Copying those values into the repo would drift from the qualification — don't.

Self-healing law: the launcher --detaches; the serve script therefore blocks on docker wait. Without the wait, a crashed engine is a green unit. docker wait cannot stop the container child, hence ExecStop.

Fork checkout + shim (on-host, outside the repo)

• Checkout: /home/dank/GitPackages/radiance-vllm-qwen4exp
• State: /home/dank/r9v-state/uncensored/setup.json (0600, ready: true)
• Model dir: /home/dank/models/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS/ (92.39 GiB payload + r9v-data/per_layer_token_embd.iq4_nl.bin 26.82 GiB PLE, sha256 34fa36f8…)
• Docker→podman shim: /home/dank/GitPackages/radiance-vllm-qwen4exp/.podman-shim/docker (PATH-prepended by the serve script). The R9V tooling is docker-pinned; the shim carries three laws:
  1. Image ID law — the bundle is a containerd/OCI docker-save; docker keys the image by its OCI MANIFEST digest (sha256:2dac17a2…), podman by config digest but preserves the manifest digest in RepoDigests. The shim answers image inspect sha256:X from RepoDigests and rewrites bare sha256:X run refs to repo@sha256:X.
  2. LogConfig law — podman inspect answers HostConfig.LogConfig.Config: null where docker returns the log-opts map (the doctor crashes on .get("Config", {}) == null). The shim records run --log-opt pairs and normalizes inspect output.
  3. docker info law — podman lacks SecurityOptions/ServerVersion; the shim answers rootless + 27.5.1 (liveness probe).
• venv: ~/.r9v-venv (huggingface_hub for fetch/verify)

Verified at cutover (2026-09-27)

• Setup: PASS=28 WARN=3 FAIL=0 — 20/20 artifacts size+sha256 verified (~93 GiB), PLE derived at exact pinned size, CED projector verified.
• First-start doctor battery: PASS=34 WARN=8 FAIL=0, including runtime-worker-transport: pinned UVA reads, H2D copies, TP all-reduce verified inside serving workers on both BDFs.
• Wire: /v1/models → frank-primary; /health + /metrics 200 (Prometheus scraping); "Say PONG" (non-thinking) → 'PONG', stop; thinking requests populate reasoning (vLLM channel).
• Memlock: the full tiered-expert load ran to healthy under a plain dank session before the unit existed — the pin path does NOT hard-require LimitMEMLOCK=infinity on this host, but the root unit keeps fleet parity and headroom for the pinned reserve.
• e2e gate: 32 passed / 1 failed — the failure is 11-admin open links, which needs FrankBook-Pro (192.168.0.112) awake; the machine was down and WOL from both hosts failed. Suite minus that spec: 31 passed (the one-time 12-crawl flake re-passes in isolation).

Memory shape (measured)

• Both cards ~29–30 GiB used of 31.86; engine budget CPU_OFFLOAD_GB_BY_DEVICE=146.25,78.75 (UVA host pin ≈ 43 GiB on rank 0
  • 16 GiB PLE pinned reserve; PLE residency mode ssd mmap+trim).
• MemAvailable floor: 56.3 GiB (R9V_MIN_HOST_AVAILABLE_BYTES).
• Context envelope: 192,608 native window served as MAX_MODEL_LEN=196608; harness models.yml carries contextWindow: 196608, maxTokens: 57344 (compaction-gap math: 70% × 196,608 = 137,625 + 57,344 ≤ 196,608).

Ops

• Restart: sudo systemctl restart frank-primary (root unit; the serve script re-gates everything, relaunch through the shim, waits).
• First start after boot re-warms ~93 GiB of page cache; expect several minutes to /health; TimeoutStartSec=1800.
• Scratch/probe: NEVER probe against the prod state dir. Copy it, set R9V_HOST_PORT to a free port and R9V_CONTAINER_NAME to a scratch name, and stop the probe before any deploy — a probe holding :7770 produces a false-green deploy (the 2026-09-26 lesson, now guarded by the MainPID check).
• Qualification artifacts (kept on-host): /home/dank/r9v-state/uncensored/qualification-*, failure bundles failure-* (review before sharing — raw logs).
• Models on the wire: served alias frank-primary; thinking channel is reasoning (vLLM), tool parser qwen3_coder (not qwen3_xml like the vllm030 wiring had planned).

Known costs (accepted at selection)

• Abliterated weights (Dyluhn uncensored build) — not the reference model; answer-quality deltas are possible.
• 192K envelope — down from 262,144 on llama.cpp.
• Docker-pinned fork — updates flow through image bundles + the shim, never a bare container run; the launcher stays the single argv owner.