serverAppConfigs

869 Commits 43 Branches 8 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
Frank	095128b669	frank-primary: R9V radiance cutover wiring (launcher-driven serve + state gates) Replace the vLLM 0.30 UVA-offload wiring (never deployed; llama.cpp b11030 endpoint replaced by Frank's order, downtime window open) with the R9V radiance-vLLM shape on the same root unit: - serve-frank-primary.sh: layer the on-host qualified setup state (104 tuned R9V_* keys, /home/dank/r9v-state/uncensored/setup.json) under repo production overrides, then exec the fork's own launcher through the docker->podman shim and block on docker wait so a crashed engine is a nonzero ExecStart (launch.sh --detach alone would fake green). Gates: pinned image by OCI manifest digest in RepoDigests, PLE at pinned byte count (stat -c %s), all GGUF shards + mmproj. - frank-primary.env: thin production overrides only (LAN bind 0.0.0.0, :7770, alias frank-primary, container frank-primary-r9v, GPU BDFs); tuned engine shape stays in the setup state by design. - frank-primary.service: same root-unit shape (User=dank, LimitMEMLOCK=infinity for the tiered-expert pin), plus ExecStop podman stop — docker wait cannot stop the container child. - deploy-remote.sh deploy_llm_primary: state-driven gates (state ready, image digest present in rootless storage, PLE bytes, shard files) replace the fp030 shard-index gate; false-green MainPID check kept. - tests/frank_primary_unit.bats: seam rewritten for the R9V shape.	2026-09-27 04:05:46 -07:00
Frank	ecaad36654	frank-primary: vLLM 0.30 UVA-offload shape (pre-deploy commit) The llama.cpp b11030 trio becomes the vLLM 0.30 trio ahead of the FP8 cutover on :7770 — same port, same alias, no client moves. Serve shape: locally built ROCm gfx1201 image (localhost/frank-primary:vllm030) execs via podman as dank from the root unit (LimitMEMLOCK=infinity legalizes the pin). UVA offload: --offload-backend uva pins the PLE family (47.75 GiB, 138 tensors: 131 n-gram table shards + 7 per-layer conv/proj) + routed experts (114.86 GiB, 150,528 tensors) in host RAM — 162.60 GiB combined, ~81.3 GiB per rank under TP=2, hence --cpu-offload-gb 82 (80/card strands 2.6 GiB of experts on the cards). Parsers per the host's own Ornith-era precedent: qwen3_xml tools + qwen3 reasoning (vLLM 0.30 emits the channel as `reasoning` — reasoning_content is renamed server-side) + enable_thinking/preserve_thinking template kwargs. Spec decode stays env-wired and OFF at bring-up. Deployer deploy_llm_primary rewritten: image gate, index-driven shard-count gate (131), stale-container cleanup, port wait, and the false-green hardening — health is only green when the unit's own MainPID is live behind the answer (2026-09-26 lesson). Seam tests rewritten to the vLLM contract (15 gates): offload arithmetic, parser wiring, no-llama.cpp-ism, deployer gates. Suite 349/349 green.	2026-09-27 00:16:42 -07:00
Frank	8500600d09	frank-primary: swap Dankserver to llama.cpp Qwen3.8-Flash-Next Q8_0 The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour: Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix prebuilt, ROCm gfx120X, bundled runtime) serving unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256- verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the frank-large shape: no container, no generated unit). The deployed contract: - Placement law: layer split -ts 1,1 with -ngl 999; the routed experts and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via -ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming over PCIe loses at 3.7 t/s, computing in RAM does not care). per_layer_token_embd.weight=CPU is required by the shared MTP head's borrow path (a GPU-resident borrow crashes the draft load). mmap stays on; --cache-ram 16384. - Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head (mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the draftless ngram-mod matcher chained behind it — the only legal way to layer two spec implementations in one process. - Vision explicit: --mmproj names the BF16 projector (this build never autoloads; a server without it comes up healthy and text-only). - Reasoning: --reasoning-format deepseek → reasoning_content; the Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded by reasoningEffortMap in models.yml. - One stream, the model's full 262,144 context; greedy server default with Qwen-recommended sampling riding compat.extraBody. The deployer's deploy_llm_primary is rewritten for the root unit: it gates every artifact at its declared byte count (stat -c %s, never wc -c — which read the whole 377 GiB lucebox file for minutes on 2026-09-26), retires the lucebox user quadlet to dated home trash, waits out port 7770, relinks/enables/restarts the root unit and waits up to 1800 s for /health. The quadlet-era tree under dankserver/frank-primary/ is archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this box; git history everywhere else) and its planner mapping replaced by the scripts/ trio. Observability flips back to the llamacpp namespace: a frank-primary Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark published-port quirk that forces host.containers.internal for the two Dankclaw endpoints is same-host only), the LlmEndpointDown runbook text, and the dashboard description for three llama.cpp endpoints. Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md), RETIRED banners on the lucebox and radiance runbooks and their index entries, the AGENTS.md fleet section rewritten for the root-unit shape, and the homepage frank-medium card renamed to frank-primary with the Q8_0 Flash-Next spec (docs.html #models and the modality note updated: images on frank-primary and frank-small, audio on frank-small only). models.yml: the dankserver provider is "llamacpp (dankserver)" serving frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap law: 183,501 + 65,536 <= 262,144), input [text, image]. Suite: 346 green (frank_primary_unit.bats rewritten for the root unit — 16 tests pinning the placement law, the shared-head requirement, the explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and the deployer's byte-count gates; the quadlet-era planner and changed-tests mappings rewritten for the trio; the observability scrape test asserts three llamacpp jobs again). Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the 378 GiB payload stay on the host; the Qwen payload stays fetched, so a re-swap in either direction is a redeploy, not a refetch.	2026-09-26 19:45:28 -07:00

Author

SHA1

Message

Date

Frank

095128b669

frank-primary: R9V radiance cutover wiring (launcher-driven serve + state gates)

Replace the vLLM 0.30 UVA-offload wiring (never deployed; llama.cpp
b11030 endpoint replaced by Frank's order, downtime window open) with
the R9V radiance-vLLM shape on the same root unit:

- serve-frank-primary.sh: layer the on-host qualified setup state
  (104 tuned R9V_* keys, /home/dank/r9v-state/uncensored/setup.json)
  under repo production overrides, then exec the fork's own launcher
  through the docker->podman shim and block on docker wait so a crashed
  engine is a nonzero ExecStart (launch.sh --detach alone would fake
  green). Gates: pinned image by OCI manifest digest in RepoDigests,
  PLE at pinned byte count (stat -c %s), all GGUF shards + mmproj.
- frank-primary.env: thin production overrides only (LAN bind 0.0.0.0,
  :7770, alias frank-primary, container frank-primary-r9v, GPU BDFs);
  tuned engine shape stays in the setup state by design.
- frank-primary.service: same root-unit shape (User=dank,
  LimitMEMLOCK=infinity for the tiered-expert pin), plus ExecStop
  podman stop — docker wait cannot stop the container child.
- deploy-remote.sh deploy_llm_primary: state-driven gates (state ready,
  image digest present in rootless storage, PLE bytes, shard files)
  replace the fp030 shard-index gate; false-green MainPID check kept.
- tests/frank_primary_unit.bats: seam rewritten for the R9V shape.

2026-09-27 04:05:46 -07:00

Frank

ecaad36654

frank-primary: vLLM 0.30 UVA-offload shape (pre-deploy commit)

The llama.cpp b11030 trio becomes the vLLM 0.30 trio ahead of the
FP8 cutover on :7770 — same port, same alias, no client moves.

Serve shape: locally built ROCm gfx1201 image
(localhost/frank-primary:vllm030) execs via podman as dank from the
root unit (LimitMEMLOCK=infinity legalizes the pin). UVA offload:
--offload-backend uva pins the PLE family (47.75 GiB, 138 tensors:
131 n-gram table shards + 7 per-layer conv/proj) + routed experts
(114.86 GiB, 150,528 tensors) in host RAM — 162.60 GiB combined,
~81.3 GiB per rank under TP=2, hence --cpu-offload-gb 82 (80/card
strands 2.6 GiB of experts on the cards). Parsers per the host's
own Ornith-era precedent: qwen3_xml tools + qwen3 reasoning (vLLM
0.30 emits the channel as `reasoning` — reasoning_content is
renamed server-side) + enable_thinking/preserve_thinking template
kwargs. Spec decode stays env-wired and OFF at bring-up.

Deployer deploy_llm_primary rewritten: image gate, index-driven
shard-count gate (131), stale-container cleanup, port wait, and the
false-green hardening — health is only green when the unit's own
MainPID is live behind the answer (2026-09-26 lesson).

Seam tests rewritten to the vLLM contract (15 gates): offload
arithmetic, parser wiring, no-llama.cpp-ism, deployer gates. Suite
349/349 green.

2026-09-27 00:16:42 -07:00

Frank

8500600d09

frank-primary: swap Dankserver to llama.cpp Qwen3.8-Flash-Next Q8_0

The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).

The deployed contract:

- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
  and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
  -ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
  over PCIe loses at 3.7 t/s, computing in RAM does not care).
  per_layer_token_embd.weight=CPU is required by the shared MTP head's
  borrow path (a GPU-resident borrow crashes the draft load).
  mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
  (mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
  draftless ngram-mod matcher chained behind it — the only legal way to
  layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
  autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
  Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
  by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
  with Qwen-recommended sampling riding compat.extraBody.

The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.

Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.

Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).

models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].

Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).

Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.

2026-09-26 19:45:28 -07:00