docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash)
Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index, Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0 draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16). deploy.sh now maps dankserver to frank-default-small and symlinks its unit. models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.
This commit is contained in:
1 parent
6a86a9df93
commit
35c44c4123
8 files changed
+34
-32
No files matched your search
+5
-5
@@ -8,10 +8,10 @@ Documentation for running vLLM inference on two local machines.
|
||||
graph LR
|
||||
subgraph Dankserver[NVIDIA Dankserver - Ubuntu 24.04]
|
||||
A[RTX 3090 24GB + 62GB RAM]
|
||||
B[vLLM in Podman
|
||||
vllm-awq-qwen36]
|
||||
C[Qwen3.6-35B-A3B AWQ 4bit
|
||||
port 34401]
|
||||
B[llama.cpp DFlash
|
||||
frank-default-small]
|
||||
C[Qwen3.6-35B-A3B uncensored Q6_K_P
|
||||
experts on CPU, port 7770]
|
||||
A --> B --> C
|
||||
end
|
||||
|
||||
@@ -24,7 +24,7 @@ port 7770]
|
||||
D --> E --> F
|
||||
end
|
||||
|
||||
G[LLM Clients / API] -->|HTTP :34401| B
|
||||
G[LLM Clients / API] -->|HTTP :7770| B
|
||||
G -->|HTTP :7770| E
|
||||
```
|
||||
|
||||
|
||||
Reference in new issue
Block a user