docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash)
Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index, Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0 draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16). deploy.sh now maps dankserver to frank-default-small and symlinks its unit. models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.
This commit is contained in:
1 parent
6a86a9df93
commit
35c44c4123
8 files changed
+34
-32
No files matched your search
@@ -115,7 +115,9 @@ The DFlash draft is not a standalone model. It must pair with its target model.
|
||||
|
||||
## Deployment on Dankserver (NVIDIA RTX 3090)
|
||||
|
||||
Dankserver runs vLLM in a Podman container with the RTX 3090. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
|
||||
The active Dankserver deployment runs **llama.cpp + DFlash** (`frank-default-small`), not vLLM. It serves the HauhauCS Qwen3.6-35B-A3B uncensored `Q6_K_P` on port 7770 with the `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` Q8_0 draft, 3 slots at 262144 context, and MoE experts on CPU. See [[deployment]] and `scripts/serve-llama-frank-small.sh`.
|
||||
|
||||
The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
|
||||
|
||||
- FP8 KV cache is valid on CUDA 12.
|
||||
- FP8 weights compute through a fallback path; no native FP8 tensor cores.
|
||||
|
||||
Reference in new issue
Block a user