Frank
095128b669
Replace the vLLM 0.30 UVA-offload wiring (never deployed; llama.cpp b11030 endpoint replaced by Frank's order, downtime window open) with the R9V radiance-vLLM shape on the same root unit: - serve-frank-primary.sh: layer the on-host qualified setup state (104 tuned R9V_* keys, /home/dank/r9v-state/uncensored/setup.json) under repo production overrides, then exec the fork's own launcher through the docker->podman shim and block on docker wait so a crashed engine is a nonzero ExecStart (launch.sh --detach alone would fake green). Gates: pinned image by OCI manifest digest in RepoDigests, PLE at pinned byte count (stat -c %s), all GGUF shards + mmproj. - frank-primary.env: thin production overrides only (LAN bind 0.0.0.0, :7770, alias frank-primary, container frank-primary-r9v, GPU BDFs); tuned engine shape stays in the setup state by design. - frank-primary.service: same root-unit shape (User=dank, LimitMEMLOCK=infinity for the tiered-expert pin), plus ExecStop podman stop — docker wait cannot stop the container child. - deploy-remote.sh deploy_llm_primary: state-driven gates (state ready, image digest present in rootless storage, PLE bytes, shard files) replace the fp030 shard-index gate; false-green MainPID check kept. - tests/frank_primary_unit.bats: seam rewritten for the R9V shape.
44 lines
2.0 KiB
Desktop File
44 lines
2.0 KiB
Desktop File
# frank-primary.service — R9V radiance-vLLM endpoint (Qwen3.8-Flash-Next-
|
|
# Uncensored IQ4_XS, MTP4, TP=2, tiered experts) on the 2x R9700 pair,
|
|
# dankserver :7770.
|
|
#
|
|
# Replaces the llama.cpp frank-primary unit (same port, same served alias;
|
|
# harness models.yml unchanged in id, retuned for the 196608 window). The
|
|
# serve script drives the R9V fork's own launcher through the docker->podman
|
|
# shim and then waits on the container, so a crashed engine is a nonzero
|
|
# ExecStart and Restart=on-failure is real.
|
|
#
|
|
# WHY A ROOT UNIT (not a rootless quadlet): the tiered-expert shape pins tens
|
|
# of GiB of host RAM (the UVA expert host copy plus the 16 GiB PLE pinned
|
|
# reserve). Pinned pages are charged against RLIMIT_MEMLOCK; dank's login
|
|
# session on this host carries 8 MiB and a user quadlet is capped by the user
|
|
# manager's own hard limit — raising that means restarting user@1000.service
|
|
# and bouncing every user container on the host. A root unit grants
|
|
# LimitMEMLOCK=infinity to the dank process via PID1, and the image stays in
|
|
# dank's rootless podman storage (the shim drives podman as dank).
|
|
[Unit]
|
|
Description=frank-primary — R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS on 2x R9700
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
User=dank
|
|
Group=dank
|
|
# The tiered-expert offloader pins host RAM; an RLIMIT_MEMLOCK smaller than
|
|
# the pinned set fails the load mid-start. PID1 grants infinity here —
|
|
# parity with the fleet's other LLM units.
|
|
LimitMEMLOCK=infinity
|
|
Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
|
|
# systemd stop: wait alone cannot stop the container child; stop it
|
|
# explicitly. vLLM 0.26 drains gracefully on SIGTERM.
|
|
ExecStop=/usr/bin/podman stop -t 120 frank-primary-r9v
|
|
# Boot cost: the launcher's preflight doctor (~1 min) after a warm page
|
|
# cache; cold (post-reboot) payload page-in of ~93 GiB dominates.
|
|
TimeoutStartSec=1800
|
|
Restart=on-failure
|
|
RestartSec=10
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|