Commit Graph
3 Commits
Author SHA1 Message Date
Frank 9c1f777423 Fix serve-vllm-qwen36-dflash.sh: remove 'vllm serve' prefix (image entrypoint handles it) 2026-08-08 12:15:31 -07:00
Frank 9971bdef0f feat: initial vLLM Qwen3.6-35B-A3B-FP8 serve script with DFlash drafting on dankserver
- Podman container with CUDA GPU access
- DFlash speculative decoding via --spec-method dflash
- FP8 KV cache, 4 parallel streams, 262K context
- CPU expert offloading via --cpu-offload-gb
- systemd service for persistent runtime
2026-08-07 18:28:38 -07:00
Frank 5452bb63f1 Add vLLM serve script, systemd service, and validation test for Qwen3.6-35B-A3B-FP8 on dankserver 2026-08-07 18:19:40 -07:00