dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(frank-small): record the CACHE_RAM=2048 cap in the deployed flags

Browse Source
This commit is contained in:
Frank committed 2026-09-21 15:06:05 -07:00
1 parent 2b046a21d0
commit 9e0c918b6e
1 file changed
+1 -1
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/frank-small-gemma4-12b.md
+1 -1
View File
@@ -133,7 +133,7 @@ prefill ~254k tokens / decode at ~253k context:
Repo truth: `scripts/frank-small.env` + `scripts/serve-frank-small.sh`. Repo truth: `scripts/frank-small.env` + `scripts/serve-frank-small.sh`.
Key values: `MODEL_DIR=/home/dank/models/Gemma4-12B-QAT`, Key values: `MODEL_DIR=/home/dank/models/Gemma4-12B-QAT`,
`MODEL_FILE=Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf`, `MODEL_FILE=Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf`,
`DRAFT_FILE=mtp-gemma-4-12B-it.gguf`, `CACHE_TYPE_K=V=q8_0`, `DRAFT_FILE=mtp-gemma-4-12B-it.gguf`, `CACHE_TYPE_K=V=q8_0`, `CACHE_RAM=2048`,
`SPEC_TYPE=draft-mtp,ngram-mod`, `SPEC_DRAFT_N_MAX=3`, `POLL_LEVEL=50`, `SPEC_TYPE=draft-mtp,ngram-mod`, `SPEC_DRAFT_N_MAX=3`, `POLL_LEVEL=50`,
`CTX_SIZE=1048576` with `PARALLEL=4` (four full 262,144-token slots), `CTX_SIZE=1048576` with `PARALLEL=4` (four full 262,144-token slots),
`BATCH_SIZE=8192`, `UBATCH_SIZE=1024`, `NGL=999`, `FLASH_ATTN=on`, `BATCH_SIZE=8192`, `UBATCH_SIZE=1024`, `NGL=999`, `FLASH_ATTN=on`,