frank-large: document reserve 15 result — both features in force, stall risk logged
Ledger confirms the doubled pool is actually reserved (524288, 13312.0 MiB) at max_tok 32768 with no clamp and no WARNING: 38.5 budget vs 37.0 needed. Recorded the cost honestly. The server now holds 103.5 of 121.2 GiB and the engine reports only 310 contiguous 2 MiB blocks / 7.1 GiB left for everything else (24-reserve baseline: 2081 blocks / 12.1 GiB). All 28 co-tenants stayed up, but the engine volunteers a new note: when memory runs out, frank-large AND its neighbours "can stall for minutes at a time at full CPU with no disk activity and no output, which is not a crash and needs no restart". That is the least diagnosable of the three failure modes seen today, and `free` hides it — MemAvailable still reads ~74 GiB because the pinned weights count as reclaimable page cache. Sizing rule now expresses the budget as a function of reserve, shows both endpoints (24 -> 29.5, 15 -> 38.5), and the max_tok ceiling is labelled as reserve-conditional (<= 18,130 at 24; <= 35,700 at 15).
This commit is contained in:
1 parent
8a79f9a709
commit
2969e08616
3 files changed
+93
-31
No files matched your search
@@ -32,7 +32,7 @@ host.
|
||||
| Target | Runtime | Service | Model | Port |
|
||||
|--------|---------|---------|-------|------|
|
||||
| Dankserver | FreeToken v0.1.2 | frank-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 |
|
||||
| Dankclaw|halogen-flash-server 0.5.8 (rootless Quadlet container)|frank-large|peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn 115.55 GiB + quality overlay, ~68 GiB pinned, 262144 ctx / 4 slots, pool 524288 requested / 262144 effective, vision tower on, greedy for silent clients)|7770|
|
||||
|Dankclaw|halogen-flash-server 0.5.8 (rootless Quadlet container)|frank-large|peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn 115.55 GiB + quality overlay, ~68 GiB pinned, 262144 ctx / 4 slots, pool 524288 / max_tok 32768 / reserve 15, vision tower on, greedy for silent clients)|7770|
|
||||
| Dankclaw | *(retired 2026-09-11)* FastFlowLM v1.0.1 on the XDNA2 NPU | frank-npu | Gemma4-E4B-IT-NPU2 pack (vision + audio). Retired with the `amd_iommu=off` decision: the driver's SVA/PASID bind needs an enabled IOMMU | 7771 |
|
||||
|
||||
History: the 2026-08-17 redo retired the pipeline-parallel split and the
|
||||
@@ -396,17 +396,23 @@ published) and the OpenAI front-end on :8731 published as host **7770** —
|
||||
with `HALOGEN_MODEL_ID=frank-large`, so no client wiring changed.
|
||||
|
||||
Deployed flags: `HALOGEN_CTX=262144`, `HALOGEN_KV_POOL_POSITIONS=524288`
|
||||
(requested 2026-09-12, but **the engine serves 262144** — with
|
||||
`HALOGEN_MAX_TOK=32768` the prefill arena eats the device budget and
|
||||
`HALOGEN_KV_POOL_FIT` clamps the pool to the floor at every boot; set
|
||||
`HALOGEN_MAX_TOK=16384` to make the doubled pool real), `HALOGEN_KV_SLOTS=4`,
|
||||
`HALOGEN_HOST_RESERVE_GIB=24`, `HALOGEN_VISION_TOWER=1`, and
|
||||
(**in force** since 2026-09-12 — two full-length conversations resident, or
|
||||
four at 131k), `HALOGEN_KV_SLOTS=4`, `HALOGEN_MAX_TOK=32768`,
|
||||
`HALOGEN_HOST_RESERVE_GIB=15`, `HALOGEN_VISION_TOWER=1`, and
|
||||
`HALOGEN_DOWNLOAD=peonist-ai/halogen-qwen3.8-flash-next` for the first-start
|
||||
fetch; everything else is upstream's default. The weights stay pinned
|
||||
(~68 GiB) and the kernel counts those locked pages as reclaimable page cache,
|
||||
so `free`, `MemAvailable` and anything built on them (Grafana) overstate free
|
||||
memory by the size of the model — believe the server's own startup ledger,
|
||||
not the dashboards.
|
||||
fetch; everything else is upstream's default. **Pool and reserve are a matched
|
||||
pair:** the budget is `121.2 host − 67.7 weights − reserve`, and that one
|
||||
budget carries the KV pool, the prefill arena and the 1.97 GiB vision tower
|
||||
together. Raise the reserve back to 24 and `HALOGEN_KV_POOL_FIT` silently
|
||||
clamps the pool to 262144 while the deploy still reports healthy. At 15 the
|
||||
server holds 103.5 of 121 GiB, leaving ~7.1 GiB on the host, and the engine
|
||||
warns that when it runs out frank-large and its neighbours can stall for
|
||||
minutes at full CPU with no disk activity — not a crash, and invisible to
|
||||
`free`, which still reads ~74 GiB available. The weights stay pinned (~68 GiB)
|
||||
and the kernel counts those locked pages as reclaimable page cache, so `free`,
|
||||
`MemAvailable` and anything built on them (Grafana) overstate free memory by
|
||||
the size of the model — believe the server's own startup ledger, not the
|
||||
dashboards.
|
||||
|
||||
Client-visible contract: **no server-side sampler defaults** (a request that
|
||||
sends no `temperature` decodes greedy); the per-request budget defaults to
|
||||
|
||||
Reference in new issue
Block a user