diff --git a/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md b/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md index bbbb7d6..e6cb6c2 100644 --- a/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md +++ b/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md @@ -44,7 +44,10 @@ off): ``` MODEL=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8 (~172.8 GiB, 131 shards) -IMAGE=docker.io/tcclaviger/vllm:latest +IMAGE=docker.io/tcclaviger/vllm@sha256:da4029666668b90a0386c1ed23ea370c79916ad4e1a6ad1dd9db098b9fa069ea +# image ID 3322dbbbc96d; local alias qwen-fp8-runtime:tuned-20260929; +# the floating :latest tag moved once already — always launch by digest. +# repo tag of this state: qwen-fp8-tuned-20260929 --tensor-parallel-size 2 --max-model-len 32768 --gpu-memory-utilization 0.98 --kv-cache-memory-bytes 2147483648 --max-num-seqs 4 --enable-prefix-caching --speculative-config '{"method":"mtp","num_speculative_tokens":4}' diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index b0f0836..20b5752 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -122,6 +122,15 @@ (~165 GiB) or W4A16 payload + a glm5next-on-HIP boot probe; community GLM-5.3 quants are GGUF (llama.cpp path only), native FP8 is 328 GB (> 246 GiB RAM). Feasibility doc rows updated. + - **Revert pins installed (Frank's request):** (1) repo tag + `qwen-fp8-tuned-20260929` at f532e03 (the campaign's docs + measured + config; `git checkout qwen-fp8-tuned-20260929` restores the state); + (2) image pinned by digest — serve script now launches + `docker.io/tcclaviger/vllm@sha256:da402966...` (ID 3322dbbbc96d) with a + local alias `qwen-fp8-runtime:tuned-20260929`; the floating `:latest` + is no longer referenced anywhere (the other image on the host, + `28.06.1-Code000` = fffca39f, is a DIFFERENT build — do not confuse). + Runbook deployment-shape section updated. ## 2026-09-28 - **Dankclaw model cleanup** (disk was at 93%, 137 G free). Frank asked to