dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash)

Browse Source 
Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index,
Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream
llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0
draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16).
deploy.sh now maps dankserver to frank-default-small and symlinks its unit.
models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.
This commit is contained in:
Frank committed 2026-08-11 01:18:29 -07:00
1 parent 6a86a9df93
commit 35c44c4123
8 files changed
+34 -32
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+5 -4
View File
@@ -14,23 +14,24 @@ Scripts for running LLM inference on two local machines.
| Service | Host | Runtime | Model | Port |
|---------|------|---------|-------|------|
| frank-default-large | dankclaw | vLLM-ROCm (CIRU) | Ling-3.0-flash CIRU INT4 MTP | 7770 |
| vllm-awq-qwen36 | dankserver | vLLM (Docker) | Qwen3.6-35B-A3B AWQ 4-bit | 34401 |
| frank-default-small | dankserver | llama.cpp (DFlash) | Qwen3.6-35B-A3B uncensored Q6_K_P | 7770 |
> The retired ROCmFPX llama.cpp service on Dankclaw was replaced by the vLLM service above. Its unit file `scripts/llama-server.service` remains for reference.
> The retired ROCmFPX llama.cpp service on Dankclaw was replaced by the vLLM service above. The retired `vllm-awq-qwen36` (AWQ 4-bit) service on Dankserver was replaced by the llama.cpp service above. Legacy unit files remain in `scripts/` for reference.
## Models
| Model | Target | Format |
|-------|--------|--------|
| Ling-3.0-flash CIRU INT4 (MTP) | dankclaw | INT4 W4A16 (Quark/vLLM) |
| Qwen3.6 35B A3B AWQ 4bit | dankserver | AWQ |
| Qwen3.6 35B A3B uncensored Q6_K_P (DFlash draft) | dankserver | GGUF Q6_K_P (llama.cpp, MoE offload) |
| Qwen3.6 35B A3B AWQ 4bit | dankserver | AWQ (retired) |
| Qwen3.6 35B A3B FP8 | dankserver | FP8 (documented, not active) |
| Qwen3.6 27B FP8 | dankclaw | FP8 (documented, not active) |
## Structure
- `docs/` — Project knowledge base with research notes on vLLM configuration, hardware specifics, speculative decoding, and deployment notes
- `scripts/` — Deployment and launch scripts (deploy.sh, serve-vllm-qwen36-dflash.sh, validate-vllm.sh, systemd unit files)
- `scripts/` — Deployment and launch scripts (deploy.sh, serve-llama-frank-small.sh, serve-ling3-vllm.sh, systemd unit files)
## Deployment