Frank
9227e1a8b9
Add AWQ DFlash Ampere script and improved validation
- vllm-awq-qwen36-dflash-ampere.sh: AWQ-4bit Qwen3.6-35B-A3B with DFlash
speculative decoding, optimized for Ampere hardware
- Model: cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit (AWQ Marlin)
- Draft: z-lab/Qwen3.6-35B-A3B-DFlash
- --max-model-len 32768
- --max-num-batched-tokens 1536
- --cpu-offload-gb 40 (experts)
- --reasoning-parser qwen3
- --tool-call-parser qwen3_xml
- Port: 34401
- Note: prefix caching disabled (Mamba cache align conflict)
- scripts/validate-vllm.sh: Per-stream latency tests
- Separate tests for 1, 2, 3, 4 streams
- Different prompts per stream to avoid caching effects
- Reports elapsed ms per stream count