live: heartbeat sync 2026-10-18T20:54:41-07:00 (Refs #11)

This commit is contained in:
Frank committed 2026-10-18 20:54:41 -07:00
1 parent 0c7fe8b4bf
commit 0258ef70f8
22 files changed
+269 -56

No files matched your search

@@ -1,6 +1,6 @@
{
"v": 1,
"lastConsolidatedAt": 1792368375152,
"lastConsolidatedAt": 1792381479415,
"lastResult": {
"at": 1791599865775,
"sessions": 1,
@@ -1,4 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792359704968
"lastConsolidatedAt": 1792381499057
}
@@ -1,4 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792378334862
"lastConsolidatedAt": 1792379255984
}
@@ -22,7 +22,7 @@
**Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated** (`.hgn` format)
- Runtime: halogen-flash-server 0.16.2 (digest-pinned `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`)
- Runtime: halogen-flash-server 0.16.2 (digest-pinned `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`)
- Format: v2 `.hgn` (flash-server exclusive)
- MTP head: INSIDE checkpoint (`HALOGEN_MTP_HEAD` unset)
- Vision: ON (v2 tower loads beside checkpoint)
@@ -62,7 +62,7 @@ Every deployed, working Podman container MUST have a self-healing, boot-persiste
Unit requirements:
- **Boot persistence** — `[Install] WantedBy=default.target`, installed to `~/.config/containers/systemd/<name>.container`, enabled in the user manager, with linger on for the deploy user (`loginctl show-user dank -p Linger` → `Linger=yes`).
- **Boot persistence** — `[Install] WantedBy=default.target`, installed to `~/.config/containers/systemd/<name>.container`, enabled in the user manager, with linger on for the deploy user (`loginctl show-user dank -p Linger` → `Linger=yes`).
- **Self-healing** — `Restart=always` (or `on-failure`) with a short `RestartSec`; quadlet health check where the service has one.
- **Repo provenance** — the `.container` file lives in this repo (`<host>/<svc>/quadlet/`), is installed by the deployer, and is covered by the quadlet test seam. The host-side unit is generated state; the repo file is the source of truth.
@@ -82,7 +82,7 @@ Scratch ports are not used on LLM hosts — a test process cannot share the sing
A gufo-era law: **`gufo bench` cannot rank speculative-decode settings, and will invert the ranking.**
Measured 2026-09-28 on frank-dragoon at depth 32768: its corpus reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across draft widths, i.e. the same highly-predictable text every run, and 93 % of every proposal accepted. It reported 29–37 t/s decode where real traffic gives ~15 t/s, and it made `--draft-tokens 16` look 6 % faster than 7 when real text made it 14 % *slower*. Use it for shape, not for A/B. For an A/B, replay **real** text through the live server and read the `Server-Timing` header — but note the unit rides in the **name** half (`ttft;dur=402.485`), so splitting on `=` alone keys the dict `ttft;dur` and every lookup silently returns nothing. Generate **at least 128 tokens** for the measurement: at 8 tokens, `inter_token` is first-token noise and the run-to-run spread swamps every real difference.
Measured 2026-09-28 on frank-dragoon at depth 32768: its corpus reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across draft widths, i.e. the same highly-predictable text every run, and 93 % of every proposal accepted. It reported 29–37 t/s decode where real traffic gives ~15 t/s, and it made `--draft-tokens 16` look 6 % faster than 7 when real text made it 14 % *slower*. Use it for shape, not for A/B. For an A/B, replay **real* text through the live server and read the `Server-Timing` header — but note the unit rides in the **name** half (`ttft;dur=402.485`), so splitting on `=` alone keys the dict `ttft;dur` and every lookup silently returns nothing. Generate **at least 128 tokens** for the measurement: at 8 tokens, `inter_token` is first-token noise and the run-to-run spread swamps every real difference.
This covers **any GPU experiment, not just benchmarks** — including a bare 2-rank RCCL/`torch.distributed` probe. On 2026-09-16 one such probe was run against the live service "because it only needs 200 MB of VRAM"; the host went hard down ~20 minutes later (no ARP reply, 100% ICMP loss from two subnets, no recovery — a power cycle was required). A hung AQL dispatch wedging the GPU driver is the plausible mechanism, and this host has precedent for going unreachable until a power cycle. If an experiment needs GPUs, stop and mask first, or use a host that serves nobody. Telling an off machine from a wedged one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a merely wedged kernel usually still answers ARP.
@@ -115,4 +115,4 @@ This rule exists because subagent hangs were observed in early deployments (2026
Before declaring a subagent deployed: verify the heartbeat endpoint returns within the timeout window, and confirm the recovery path is wired (retry/fallback/escalation logic exists in the codepath).
A subagent without a heartbeat or recovery path is a gap: wire the heartbeat, add the recovery path, then deploy.
A subagent without a heartbeat or recovery path is a gap: wire the heartbeat, add the recovery path, then deploy.`,
@@ -1,7 +1,27 @@
## Voice Generation
- Custom voice source material must exclude the first 10 seconds of any audio sample file used to generate voices; this is a project-wide invariant across all source files, not a per-file exception.
- Voice selector valid options: af_heart, af_alloy, af_aoede, af_bella, af_jessica, af_kore, af_nicole, af_nova, af_river, af_sarah, af_sky, am_adam, am_echo, am_eric, am_fenrir, am_liam, am_michael, am_onyx, am_puck, am_santa, bf_emma, bf_isabella, bm_george, bm_lewis, bf_alice, bf_lily, bm_daniel, bm_fable.
## Podcast
- Podcast functionality is sourced from open-notebook; this is a project decision established before this document was created.
- Podcast functionality is sourced from open-notebook; this is a project decision established before this document was created.
## Audio Input Policy
- There is no audio input anywhere in the fleet: that capability left with the FLM NPU endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's audio encoder, and left again with the Gemma4 retirement on 2026-10-01.
- This policy mirrors the architecture-level decision documented in `architecture.md` regarding the fleet's input capabilities.
## Input Capabilities
All endpoints accept text input with optional image input: `frank-dragoon` returned to vision with the flash-server v2 swap, while `frank-archon` takes them natively and `frank-zealot` through its BF16 mmproj.
There is no audio input anywhere in the fleet: that capability left with the FLM NPU endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's audio encoder, and left again with the Gemma4 retirement on 2026-10-01.
This rule exists because the Gemma4 projector was retired in 2026-10-01, removing the audio input channel from the fleet. The 2026-09-11 to 2026-10-01 period contained the Gemma4 projector which enabled audio, but that endpoint is now retired.
## Voice Usage Notes
- Voice generator output does not affect token counting or inference costs — all voices are rendered post-inference.
- Voice quality varies by source sample: exclude the first 10 seconds of any audio sample file used to generate voices; this is a project-wide invariant across all source files, not a per-file exception.
- Voice selection is client-side only; the server does not generate or return audio data.`,
@@ -1,8 +1,8 @@
{
"v": 1,
"lastConsolidatedAt": 1792378149820,
"lastConsolidatedAt": 1792381956673,
"lastResult": {
"at": 1792374778678,
"at": 1792379504653,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
@@ -1,3 +1,3 @@
## top bar
- Preserve top bar customizations; customizations lost or obliterated on subsequent runs
- Todo list updates as tasks change, not waiting for entire bill of tasks to finish
- Todo list updates as tasks change, not waiting for entire bill of tasks to finish
@@ -1,8 +1,8 @@
{
"v": 1,
"lastConsolidatedAt": 1792378276529,
"lastConsolidatedAt": 1792381281460,
"lastResult": {
"at": 1792374324899,
"at": 1792381281460,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
@@ -1,2 +1,2 @@
## window outlines
- Window outline color is amber, and outline thickness is thickened
- Window outline color is amber, and outline thickness is thickened
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792381246498
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792381842302
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792381487662
}
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1792381870670,
"lastResult": {
"at": 1792381870670,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
}
}