docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash)
Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index, Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0 draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16). deploy.sh now maps dankserver to frank-default-small and symlinks its unit. models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.
This commit is contained in:
1 parent
6a86a9df93
commit
35c44c4123
8 files changed
+34
-32
No files matched your search
@@ -112,7 +112,7 @@ export HF_HOME=/mnt/entropy/huggingface
|
||||
|
||||
### Active Service Conflict
|
||||
|
||||
A vLLM engine core (Podman container `vllm-qwen36-dflash`) occupies roughly 22 GB of the 24 GB VRAM. Stop it before starting a competing service:
|
||||
The active `frank-default-small` service (llama.cpp, port 7770) uses about 22 GB of the 24 GB VRAM. The retired `vllm-awq-qwen36` (Podman, ~22 GB) and `vllm-qwen36` services are not active. Stop any competing service before starting another:
|
||||
|
||||
```
|
||||
podman stop vllm-qwen36-dflash
|
||||
|
||||
Reference in new issue
Block a user