Frank
e4fba35e89
- vLLM 0.26.0+rocm723 - --kv-cache-dtype fp8 --gpu-memory-utilization 0.80 - --max-model-len 32768 --max-num-batched-tokens 2048 - --max-num-seqs 32 --reasoning-parser qwen3 --language-model-only - Baseline response time: 219s for 50 tokens