serverAppConfigs

Files

T

Frank ced4c6f46e sync: update vllm-awq-qwen36-dflash-ampere.sh from dankserver

- Model: Avesed/Qwen3.6-35B-A3B-INT8-W8A8 (was AWQ 4bit)
- Image: vllm/vllm-openai:nightly (was :latest)
- Spec method: mtp with 2 tokens (was dflash with 12)
- Max model len: 200700 (was 32768)
- Max num seqs: 3 (was 1)
- Max batched tokens: 3072 (was 4096)
- GPU mem util: 0.97 (was 0.96)
- CPU offload: 30GB (was 40GB)
- Add --enable-prefix-caching
- Add --kv-cache-dtype auto

2026-08-10 03:23:09 -07:00

llama-server.service

Bump context to 600000 (200000/stream x 3 parallel)

2026-08-07 17:18:10 -07:00

serve-llamacpp-dflash.sh

Bump context to 600000 (200000/stream x 3 parallel)

2026-08-07 17:18:10 -07:00

serve-vllm-qwen36-dflash.sh

feat: initial vLLM Qwen3.6-35B-A3B-FP8 serve script with DFlash drafting on dankserver

2026-08-07 18:28:38 -07:00

validate-vllm.sh

Update validation script to test parallel streams (4 concurrent requests)