Frank
095128b669
Replace the vLLM 0.30 UVA-offload wiring (never deployed; llama.cpp b11030 endpoint replaced by Frank's order, downtime window open) with the R9V radiance-vLLM shape on the same root unit: - serve-frank-primary.sh: layer the on-host qualified setup state (104 tuned R9V_* keys, /home/dank/r9v-state/uncensored/setup.json) under repo production overrides, then exec the fork's own launcher through the docker->podman shim and block on docker wait so a crashed engine is a nonzero ExecStart (launch.sh --detach alone would fake green). Gates: pinned image by OCI manifest digest in RepoDigests, PLE at pinned byte count (stat -c %s), all GGUF shards + mmproj. - frank-primary.env: thin production overrides only (LAN bind 0.0.0.0, :7770, alias frank-primary, container frank-primary-r9v, GPU BDFs); tuned engine shape stays in the setup state by design. - frank-primary.service: same root-unit shape (User=dank, LimitMEMLOCK=infinity for the tiered-expert pin), plus ExecStop podman stop — docker wait cannot stop the container child. - deploy-remote.sh deploy_llm_primary: state-driven gates (state ready, image digest present in rootless storage, PLE bytes, shard files) replace the fp030 shard-index gate; false-green MainPID check kept. - tests/frank_primary_unit.bats: seam rewritten for the R9V shape.
21 lines
1.1 KiB
Bash
21 lines
1.1 KiB
Bash
# frank-primary.env — R9V radiance-vLLM production overrides (dankserver :7770).
|
|
#
|
|
# The full tuned engine shape (104 R9V_* keys: tiered-expert placement,
|
|
# CPU offload budgets 146.25/78.75 GiB per rank, CED on, MTP4, TP=2,
|
|
# qwen3_coder tool parser, 196608 model len) lives on-host in the R9V setup
|
|
# state — /home/dank/r9v-state/uncensored/setup.json — written by the
|
|
# profile's setup run (PASS=28/FAIL=0, all 20 artifacts sha256-verified).
|
|
# serve-frank-primary.sh layers that state first and THIS FILE second, so the
|
|
# keys below win. Never copy tuned values from the state into this file —
|
|
# the state is the single source of truth for engine shape.
|
|
R9V_HOME=/home/dank/GitPackages/radiance-vllm-qwen4exp
|
|
R9V_STATE_FILE=/home/dank/r9v-state/uncensored/setup.json
|
|
# Fleet identity: LAN bind, the single frank-primary port, the harness's
|
|
# model id, our own container name (the launcher default r9v-qwen38-flash-next
|
|
# is the scratch identity).
|
|
R9V_HOST_BIND=0.0.0.0
|
|
R9V_HOST_PORT=7770
|
|
R9V_SERVED_MODEL_NAME=frank-primary
|
|
R9V_CONTAINER_NAME=frank-primary-r9v
|
|
R9V_EXPECTED_GPU_BDFS=0000:03:00.0,0000:07:00.0
|