docs: rename frank-default-large/small/sightsound to frank-large/small/npu across vault
Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.
This commit is contained in:
1 parent
5a083eee6c
commit
8212d4451a
68 files changed
+1053
-1207
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
title: frank-default-small Benchmark — ncmoe and Batch Tuning
|
||||
title: frank-small Benchmark — ncmoe and Batch Tuning
|
||||
description: llama-bench sweep of MoE CPU offload (ncmoe) and batch/ubatch on the RTX 3090 for Qwen3.6-35B-A3B Q6_K_P.
|
||||
tags:
|
||||
- benchmark
|
||||
@@ -10,11 +10,11 @@ tags:
|
||||
- dankserver
|
||||
---
|
||||
|
||||
# frank-default-small Benchmark — ncmoe and Batch/Ubatch Tuning
|
||||
# frank-small Benchmark — ncmoe and Batch/Ubatch Tuning
|
||||
|
||||
This page records a llama-bench sweep on Dankserver (RTX 3090, 24 GiB) for
|
||||
the HauhauCS Qwen3.6-35B-A3B Q6_K_P model. The sweep decided the final
|
||||
`--n-cpu-moe` and `--ubatch-size` values for `frank-default-small`.
|
||||
`--n-cpu-moe` and `--ubatch-size` values for `frank-small`.
|
||||
|
||||
Run date: 2026-08-13. Build: llama.cpp `555881ebc` (10121). CUDA, 7 threads,
|
||||
`--cache-type-k q8_0 --cache-type-v f16`, `--fit-target 0` (KV auto-sized to
|
||||
@@ -105,7 +105,7 @@ service runs 700416 cells (13,680 MiB).
|
||||
|
||||
## Chosen State
|
||||
|
||||
> **Live drift (2026-08-22 audit):** the deployed `frank-default-small` now runs `--n-cpu-moe 38`, `--batch-size 4096`, `--ubatch-size 2048` (ncmoe raised from 36 for stability; batch lowered). The values below are the original benchmark recommendation.
|
||||
> **Live drift (2026-08-22 audit):** the deployed `frank-small` now runs `--n-cpu-moe 38`, `--batch-size 4096`, `--ubatch-size 2048` (ncmoe raised from 36 for stability; batch lowered). The values below are the original benchmark recommendation.
|
||||
|
||||
Prefill speed is the priority, so the state keeps the top prefill tier and
|
||||
moves one expert layer to CPU for stability:
|
||||
|
||||
Reference in new issue
Block a user