docs + script defaults: Q6_K swap, q8_0/f16 KV, split 0.68, FA rebuild
- spec matrix + PP doc + deployment.md + AGENTS.md + index.md: current production = Uncensored Q6_K + standalone MTP draft, mismatched q8_0/f16 KV with FA retained (GGML_CUDA_FA_ALL_QUANTS rebuild at 9b05354ec), split 0.68 (45 blocks), KV 8624/3920, 273 MiB free. - PP doc: 'FA-all-quants rebuild' section (flag, cmake commands, both hosts) + new-model split-limits measurements; old Q8_K_XL limits marked superseded. - serve-pp-head.sh defaults: MODEL_FILE -> Q6_K, CACHE_TYPE_V -> f16 (env already correct; defaults now match production).
This commit is contained in:
@@ -25,7 +25,7 @@ fi
|
||||
|
||||
# --- defaults (used when the env file does not set them) -------------------
|
||||
: "${MODEL_DIR:=/home/dank/models}"
|
||||
: "${MODEL_FILE:=Qwen3.8-27B-UD-Q8_K_XL.gguf}"
|
||||
: "${MODEL_FILE:=Qwen3.8-27B-Uncensored-Q6_K.gguf}"
|
||||
: "${HEAD_BIN:=/home/dank/llama.cpp/build/bin/llama-server}"
|
||||
: "${ALIAS:=frank-pp-large}"
|
||||
: "${BIND_ADDR:=0.0.0.0}"
|
||||
@@ -43,7 +43,7 @@ fi
|
||||
: "${SPEC_NGRAM_MOD_N_MAX:=64}"
|
||||
: "${SPEC_NGRAM_MOD_N_MATCH:=24}"
|
||||
: "${CACHE_TYPE_K:=q8_0}"
|
||||
: "${CACHE_TYPE_V:=q8_0}"
|
||||
: "${CACHE_TYPE_V:=f16}"
|
||||
: "${BATCH_SIZE:=8192}"
|
||||
: "${UBATCH_SIZE:=256}"
|
||||
: "${TEMPERATURE:=1.0}"
|
||||
|
||||
Reference in New Issue
Block a user