docs: fix stale context sizes, batch/ubatch, MoE, and service names across docs
This commit is contained in:
1 parent
cd6202f4a0
commit
701e742f9f
4 files changed
+7
-7
No files matched your search
+4
-4
@@ -105,10 +105,10 @@ curl -sf http://127.0.0.1:52625/health
|
||||
The active service on Dankserver is upstream llama.cpp (b10121) serving
|
||||
`Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf` under the
|
||||
alias `frank-default-small` on port 7770. It runs the stock
|
||||
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 36 MoE
|
||||
experts on CPU (`--n-cpu-moe 36`) and 7 CPU threads, ngram-mod
|
||||
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 38 MoE
|
||||
experts on CPU (`--n-cpu-moe 38`) and 7 CPU threads, ngram-mod
|
||||
speculative decoding (n_min 48 / n_max 64 / n_match 24), f16 KV for both
|
||||
K and V, context 700000 (4 slots of ~175K each), batch 8192 / ubatch 3072,
|
||||
K and V, context 786432 (3 slots of ~262K each), batch 4096 / ubatch 2048,
|
||||
`--no-mmap`. The unit file is `scripts/frank-default-small.service`. It
|
||||
launches through `scripts/serve-frank-small.sh`, which reads tunables from
|
||||
`scripts/frank-default-small.env`.
|
||||
@@ -122,7 +122,7 @@ companions) under the alias `frank-default-large` on port 52625. It runs
|
||||
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
|
||||
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context
|
||||
(1 slot; concurrent requests queue, the server never preempts a running
|
||||
generation), batch 2048 / ubatch 2048, 16
|
||||
generation), batch 4096 / ubatch 4096, 16
|
||||
context checkpoints, 8 GiB
|
||||
prompt cache, and reasoning in deepseek format. The unit file is
|
||||
`scripts/frank-default-large.service`. It launches through
|
||||
|
||||
Reference in new issue
Block a user