frank-large: halogen container cutover, rootless rlimit law, and the DFlash2 drafter shape

serverAppConfigs-docs/LLM/VLLM Overview.md

@@ -237,6 +237,46 @@ vLLM supports multiple parallelism strategies:
 - Expert Parallelism (EP) distributes expert networks in MoE models
 - Context Parallelism (CP) splits context across GPUs
 
+
+## Chunked Prefill — the flag surface, and what it actually does
+
+**YES, the runtime exposes chunked-prefill options, and the running shape
+already uses them.** The tcclaviger fork is a vLLM fork, so the standard
+scheduler flags are present in the shipped image (verified by reading the
+`vllm.engine.arg_utils` and `vllm.config.scheduler` source in the live
+container `frank-primary-qwen`, not from a stub):
+
+| Flag | Default | What it controls |
+|---|---|---|
+| `--enable-chunked-prefill` | `True` in `SchedulerConfig`; the CLI arg is an auto-flag (`"default": None`) | whether prefill may be split into chunks |
+| `--max-num-batched-tokens` (MNBT) | 2048 (`DEFAULT_MAX_NUM_BATCHED_TOKENS`) | tokens processed per prefill iteration |
+| `--prefill-schedule-interval` | 0 | gap between prefill iterations |
+
+The running shape's argv carries `--max-num-batched-tokens 2048` alongside
+`--max-model-len 200704`, so chunked prefill is **ON and already active**: a
+200704-token prompt is prefilled in ~98 iterations of 2048 tokens, not in one
+shot.
+
+**It is a TTFT lever, not a throughput lever.** MEASURED 2026-09-29 on the
+ad hoc qwen-fp8 container (the same fork, same payload, same TP=2):
+
+- MNBT 8192 vs the default 2048: **neutral** on decode at every depth.
+- Small values re-chunk prefill only. They hurt TTFT (each chunk is its own
+  iteration, so the first token waits on more iterations) and they never touch
+  decode: a decode step is `max_num_seqs x (spec+1)` tokens, independent of
+  MNBT.
+
+**There is no `--ubatch-size` equivalent in vLLM.** `--ubatch-size` is a
+llama.cpp compute-buffer concept (`--batch-size`/`--ubatch-size`); the tcclaviger
+fork has no ubatch knob at all. The llama.cpp compute-buffer pair that frank-small
+was doubled on (2026-10-05, +4.6 % prefill at 8,928 tokens for 616 MiB) has no
+counterpart here, and the two backends should not be reasoned about by analogy.
+
+**MNBT is not a lever on this shape either.** The deployed 2048 is the fork's
+default, so a change is a deliberate departure from the shipped value, not a
+tune. The one thing MNBT does cost: every change re-enters the compile RANGE
+path, so `FRANK_PRIMARY_CACHE_DIR` pays one recompilation.
+
 ## See Also
 
 - [[VLLM Index]]

serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md

@@ -48,7 +48,7 @@ bearing numbers:
 - **`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200`** — coupled
   pair (FLAGS.md table): a request at the cap must not outlive the timeout.
   65536 > the harness's 57,344 `maxTokens` so an eager request never meets
-  the engine's 413.
+  the engine's 400.
 - **KV math:** 1 slot x 200,700 x 64 KiB/token (d2 KV = 2x fp8) = 12.2 GiB
   pool, inside the ~65 GiB left after 33.4 GiB mmap'd weights + engine working
   set + host baseline on the 121.2 GiB box. The engine sizes the pool itself
@@ -56,17 +56,23 @@ bearing numbers:
   that ledger, not a formula.
 - **No server-side sampler:** an absent temperature is GREEDY. The card's
   thinking recipe (temp 1.0 / top_p 0.95) lives in `models.yml` `extraBody`
-  — on the wire only.
+  — on the wire only. The template defaults `reasoning_effort` to **xhigh**
+  when a request sends none, and **thinking tokens consume the
+  `max_tokens` budget**: a budget exhausted mid-thought returns
+  `finish_reason: "length"` with EMPTY `content` and the partial reasoning
+  in `reasoning_content` — clients must read both fields.
 
 ## Client contract deltas vs llama.cpp
 
 |Behavior|halogen|
 |---|---|
 |Prompt over slot|400 naming the limit (no truncation)|
-|`max_tokens` over cap|413, max_tokens over queue timeout|503s for the queue|
+|`max_tokens` over cap|400 naming the limit (never silent truncation)|
+|full-length request outliving `HALOGEN_QUEUE_TIMEOUT`|503s for everyone queued|
+|`timings`|every response carries llama-server-shape `timings` (`draft_n`, `draft_n_accepted`) — the drafter check reads the RESPONSE, and it proves speculation fired|
 |`response_format`|NOT supported (no constrained decoding)|
 |thinking field|`reasoning_content` (same as before)|
-|`reasoning_effort`|same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays|
+|`reasoning_effort`|same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays; unset defaults to xhigh, `none` disables thinking|
 |tools|server-side tool calls advertised — round-trip MUST be re-probed after deploy|
 |image input|ignored/error — route to frank-small|