sync scripts from remotes: ROCmFPX llama-server.service (dankclaw), dankserver vLLM serve/validate; drop failed-attempt scripts
- llama-server.service: dankclaw migrated to ROCmFPX runtime (Ling-3.0-flash, draft-mtp, no-mmap/direct-io) from regular llama.cpp Qwen3.6-27B-DFlash - serve-vllm-qwen36-dflash.sh: dankserver version (relies on image 'vllm serve' entrypoint; HF cache mounted read-write) - validate-vllm.sh: dankserver multi-stream vLLM validation (4 unique prompts, 1-4 streams, prompt+completion speed) - remove superseded scripts: serve-llamacpp-dflash.sh, validate.sh, vllm-awq-qwen36-dflash-ampere.sh, vllm-qwen36.service
This commit is contained in:
1 parent
ced4c6f46e
commit
841d872aa7
7 files changed
+76
-285
No files matched your search
@@ -1,12 +1,12 @@
|
||||
[Unit]
|
||||
Description=llama.cpp DFlash+MTP Inference Server (Qwen3.6-27B on ROCm)
|
||||
Description=llama.cpp Inference Runtime
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=dank
|
||||
Group=dank
|
||||
ExecStart=/home/dank/llama.cpp/build/bin/llama-server --hf-repo unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 --spec-draft-hf Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp:Q8_0 --spec-type draft-dflash --spec-draft-n-max 16 --device ROCm0 --spec-draft-device ROCm0 --gpu-layers all --fit on --fit-target 512 --parallel 3 --ctx-size 600000 --batch-size 4096 --port 8080 --host 0.0.0.0 --threads 16 --alias frank-default-large --load-mode none
|
||||
ExecStart=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-server -hf raulvidis/Ling-3.0-flash-ROCmFP4-STRIX-MTP-GGUF:Q4_0 --spec-type draft-mtp --spec-draft-n-max 2 --device ROCm0 --gpu-layers all --fit on --fit-target 512 --parallel 2 --ctx-size 401400 --batch-size 8192 --ubatch-size 4096 --port 8080 --host 0.0.0.0 --threads 16 --alias frank-default-large --no-mmap --direct-io -lv 4 --timeout 7200 --chat-template-file /home/dank/LingFlash/chat_template.jinja
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
|
||||
Reference in new issue
Block a user