- Podman container with CUDA GPU access - DFlash speculative decoding via --spec-method dflash - FP8 KV cache, 4 parallel streams, 262K context - CPU expert offloading via --cpu-offload-gb - systemd service for persistent runtime
- Podman container with CUDA GPU access - DFlash speculative decoding via --spec-method dflash - FP8 KV cache, 4 parallel streams, 262K context - CPU expert offloading via --cpu-offload-gb - systemd service for persistent runtime