Commit Graph
5 Commits
Author SHA1 Message Date
Frank 9971bdef0f feat: initial vLLM Qwen3.6-35B-A3B-FP8 serve script with DFlash drafting on dankserver
- Podman container with CUDA GPU access
- DFlash speculative decoding via --spec-method dflash
- FP8 KV cache, 4 parallel streams, 262K context
- CPU expert offloading via --cpu-offload-gb
- systemd service for persistent runtime
2026-08-07 18:28:38 -07:00
Frank 7dce375ab4 docs: fix dankclaw specs (30GiB RAM, 96GiB GPU-visible, ROCm 7.2.4) + add validate.sh 2026-08-06 22:40:44 -07:00
Frank 5076ef828f chore: fix typo in dankclaw specs, format index table 2026-08-06 22:29:23 -07:00
Frank dc356acce7 docs: add top-level knowledge base index 2026-08-06 00:16:47 -07:00
Frank 39db415348 feat: initialize vLLM scripts knowledge base
- VLLM Overview: installation, CLI, API, architecture, quantization, parallelism
- VLLM Index: quick reference with commands, endpoints, parameters
- Dankserver System Specs: RTX 3090 24GB, 62GB RAM, Ubuntu 24.04
- Dankclaw System Specs: AMD Strix Halo APU, 61GB unified, ROCm 7.1
- RAM/CPU Offloading in VLLM: KV cache swap, gpu_memory_utilization, preemption
- Hybrid CPU/GPU Processing (Dankserver): NVIDIA GPU + system RAM configuration
- Shared Memory Architecture (Dankclaw): AMD unified memory and ROCm setup
- Running Qwen3.6 35B A3B FP8 on Dankserver: MoE model deployment guide
- Running Qwen3.6 27B FP8 on Dankclaw: dense model deployment guide
2026-08-05 23:49:02 -07:00