From 025216fc841184d5548b3521bb05533c716b1e85 Mon Sep 17 00:00:00 2001
From: Frank <frank@dankware>
Date: Sat, 12 Sep 2026 23:40:34 -0700
Subject: [PATCH] docs(frank-small): document that FreeToken samples by default
 from the checkpoint

Answering "does the FreeToken runtime use sampling arguments?" -- yes, and
nothing in our config revealed it. serve-frank-small.sh passes no sampling
flags; the behaviour is the runtime default:

    freetoken/server/args.py:31  sampling_defaults: str = "model"

which fills temperature/top_k/top_p from the checkpoint's
generation_config.json -- this one ships do_sample:true, temp 1.0, top_k 20,
top_p 0.95. Proved live: three identical requests with no sampling params
produced three different reasoning traces (487/540/577 chars).

The asymmetry with frank-large is the part that will bite someone: a silent
client samples on frank-small and decodes greedy on frank-large, so the same
request is reproducible on one endpoint and not the other. Documented with the
determinism options (explicit temperature:0, or --sampling-defaults none).

Also recorded two request-level traps found during the probe: thinking eats the
whole max_tokens budget and returns empty content with finish_reason length,
and the API accepts unknown fields with a 200, so a 200 proves nothing about
whether a key was understood.
---
 .../LLM/freetoken-frank-small.md              | 52 +++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/serverAppConfigs-docs/LLM/freetoken-frank-small.md b/serverAppConfigs-docs/LLM/freetoken-frank-small.md
index 3473796..b326e41 100644
--- a/serverAppConfigs-docs/LLM/freetoken-frank-small.md
+++ b/serverAppConfigs-docs/LLM/freetoken-frank-small.md
@@ -125,6 +125,58 @@ tensor b (...)` in `_dequant_fp8_weight`, which multiplies
 `weight × weight_scale` directly instead of expanding the block scales.
 modelopt block-fp8 remains the only FP8 path that loads.
 
+## Sampling
+
+**frank-small samples by default, and our config never said so.**
+`scripts/serve-frank-small.sh` passes no sampling arguments at all — only
+`--model`, `--served-model-name`, `--host`, `--port`, `--moe-cache-size`,
+`--num-tokens` and `--max-running-requests`. The behaviour comes from the
+runtime's own default:
+
+```python
+# freetoken/server/args.py:31
+sampling_defaults: str = "model"
+```
+
+`--sampling-defaults model` fills `temperature` / `top_k` / `top_p` from the
+**checkpoint's** `generation_config.json` for any request that omits them.
+This checkpoint ships:
+
+```json
+{ "do_sample": true, "temperature": 1.0, "top_k": 20, "top_p": 0.95 }
+```
+
+Verified behaviourally 2026-09-12 — three identical requests with **no
+sampling parameters sent at all** produced three different reasoning traces
+(sha `13ca02…`/487 chars, `d49928…`/540, `7b81b8…`/577). Greedy decoding
+would be byte-identical.
+
+### This is the opposite of frank-large
+
+| | frank-small (FreeToken) | frank-large (halogen) |
+|---|---|---|
+| Silent client (no `temperature`) | **samples** at temp 1.0 / top_k 20 / top_p 0.95 | **greedy** — no server-side sampler defaults |
+| Deterministic across runs | No | Yes |
+| Where the default comes from | checkpoint `generation_config.json` | none; the request decides |
+
+A client that talks to both endpoints and sends no sampler gets
+non-deterministic answers from :7770 on Dankserver and reproducible ones from
+:7770 on Dankclaw. If a workflow needs determinism from frank-small it must
+send `temperature: 0` explicitly, or the service must be relaunched with
+`--sampling-defaults none` (framework defaults only).
+
+### Related request behaviour
+
+- **Thinking consumes the budget.** At `max_tokens: 200` all three probe runs
+  returned `finish_reason: "length"` with **empty `content`** and the whole
+  budget spent in `reasoning_content`. Budget for thinking, or the answer
+  never appears.
+- **The API does not validate unknown fields** — a request carrying
+  `totally_bogus_field: 123` returns 200, as does
+  `chat_template_kwargs: {"preserve_thinking": true}`. A 200 is not evidence
+  that a key was understood.
+
+
 ## Files
 
 | File | Purpose |