T

Frank 9971bdef0f feat: initial vLLM Qwen3.6-35B-A3B-FP8 serve script with DFlash drafting on dankserver

- Podman container with CUDA GPU access
- DFlash speculative decoding via --spec-method dflash
- FP8 KV cache, 4 parallel streams, 262K context
- CPU expert offloading via --cpu-offload-gb
- systemd service for persistent runtime

2026-08-07 18:28:38 -07:00

docs

feat: initial vLLM Qwen3.6-35B-A3B-FP8 serve script with DFlash drafting on dankserver

2026-08-07 18:28:38 -07:00

scripts

feat: initial vLLM Qwen3.6-35B-A3B-FP8 serve script with DFlash drafting on dankserver

2026-08-07 18:28:38 -07:00

.gitignore

chore: ignore obsidian workspace files

2026-08-06 00:14:06 -07:00

README.md

feat: initialize vLLM scripts knowledge base

2026-08-05 23:49:02 -07:00

README.md

vLLM Scripts

Scripts for running large language models via vLLM on two local machines.

Targets

Host	User	Codename	Architecture
dankserver	dank	DankServer	NVIDIA GPU + System RAM
dankclaw	dank	DankClaw	AMD Strix Halo (Shared Memory)

Models

Model	Target	Format
Qwen3.6 35B A3B FP8	dankserver	FP8
Qwen3.6 27B FP8	dankclaw	FP8

Structure

docs/ — Project knowledge base with research notes on vLLM configuration, hardware specifics, and deployment notes
scripts/ — Executable vLLM launch scripts (coming soon)

Knowledge Base

See docs/README.md for the project knowledge base index.