7 Commits

Include renames

Author	SHA1	Message	Date
Frank	2426ca563a	Add ngram-mod to spec pipeline: -1.2 tok/s regression ... - DFlash+ngram-mod: 17.11 tok/s vs DFlash alone: 18.34 tok/s - ngram overhead not offset by acceptance rate at current settings - Keeping in place for now, may tune ngram-mod params later	2026-08-07 16:25:49 -07:00
Frank	764a65080f	Validation: 2 parallel prompts, ~18 tok/s with DFlash ... - Add second parallel prompt (Byzantine Empire history vs ML) - Both hit 1024 tokens, combined 2048 in 111.6s = 18.34 tok/s - Parallel throughput matches single-stream baseline	2026-08-07 16:21:20 -07:00
Frank	25d57184a6	Enable DFlash speculative decoding, remove mlock ... - Add --spec-type draft-dflash (was missing, DFlash was inactive) - Change --load-mode from mlock to none (RLIMIT_MEMLOCK too small for 2.7GB) - Increase validation max_tokens from 256 to 1024 with complex ML prompt - Baseline: 18.76 tok/s (was 7.62 tok/s without DFlash)	2026-08-07 16:10:55 -07:00
Frank	1949f0651b	Pivot to llama.cpp with DFlash speculative decoding on ROCm ... - Abandon vLLM/FP8 (Triton compilation hangs, ~0.5 tok/s on Strix Halo) - llama.cpp v10133 with ROCm0 offload, DFlash draft model - Main: unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 (~22GB) - Draft: Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp:Q8_0 (~1.8GB) - 3 parallel streams, 131072 context/stream - systemd service with minimal sandboxing (ROCm requires /dev/kfd access) - Baseline: 7.62 tok/s (threshold 5.0 tok/s)	2026-08-07 15:41:27 -07:00
Frank	2bb1b2ae1d	Baseline: ThinkingCap 27B FP8 on dankclaw with enforce-eager and validation timeout	2026-08-07 08:58:20 -07:00
Frank	e4fba35e89	Baseline: ThinkingCap 27B FP8 on dankclaw ... - vLLM 0.26.0+rocm723 - --kv-cache-dtype fp8 --gpu-memory-utilization 0.80 - --max-model-len 32768 --max-num-batched-tokens 2048 - --max-num-seqs 32 --reasoning-parser qwen3 --language-model-only - Baseline response time: 219s for 50 tokens	2026-08-07 08:49:59 -07:00
Frank	7dce375ab4	docs: fix dankclaw specs (30GiB RAM, 96GiB GPU-visible, ROCm 7.2.4) + add validate.sh	2026-08-06 22:40:44 -07:00