 Frank
|
6bdabfc65b
|
Add VLLM_USE_V2_MODEL_RUNNER=0 to skip V2 model runner MARLIN repacking OOM
|
2026-08-09 21:49:25 -07:00 |
|
|
Frank
|
83b0bea29b
|
Fix: move commented speculative config to end to not break bash line continuation
|
2026-08-09 12:27:28 -07:00 |
|
|
Frank
|
88aa18ef34
|
Reduce GPU memory to 75%, add expandable_segments to fix OOM
|
2026-08-09 12:11:48 -07:00 |
|
|
Frank
|
9dfa16fae9
|
Disable speculative decoding - OOM on stable vLLM too
|
2026-08-09 12:08:26 -07:00 |
|
|
Frank
|
578c3e8f71
|
Enable DFlash speculative decoding (12 tokens), increase CPU offload to 20GB
|
2026-08-09 11:54:40 -07:00 |
|
|
Frank
|
8c13c7bbaa
|
Add AGENTS.md git workflow rules and initial vllm-awq-qwen36-dflash-ampere.sh from dankserver
|
2026-08-09 11:54:04 -07:00 |
|