Files
serverAppConfigs/shared/homepage/docs.html
T
Frank 8500600d09 frank-primary: swap Dankserver to llama.cpp Qwen3.8-Flash-Next Q8_0
The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).

The deployed contract:

- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
  and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
  -ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
  over PCIe loses at 3.7 t/s, computing in RAM does not care).
  per_layer_token_embd.weight=CPU is required by the shared MTP head's
  borrow path (a GPU-resident borrow crashes the draft load).
  mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
  (mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
  draftless ngram-mod matcher chained behind it — the only legal way to
  layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
  autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
  Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
  by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
  with Qwen-recommended sampling riding compat.extraBody.

The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.

Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.

Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).

models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].

Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).

Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.
2026-09-26 19:45:28 -07:00

128 lines
6.1 KiB
HTML

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>frank — docs</title>
<base href="/">
<link rel="stylesheet" href="style.css">
</head>
<body>
<header class="top">
<span class="title">frank@home:~$ docs</span>
<a id="bare-toggle" class="toggle" href="?bare=1">bare</a>
</header>
<main class="docs">
<section id="getting-in">
<h1>Getting in</h1>
<p>Everything here lives behind two doors. The first is <strong>Anubis</strong>, a
quiet bot-check: the first time you visit any guarded page in a session you see a
short "making sure you're not a bot" screen. It runs once. The second door is
<strong>Authentik</strong>, the family single sign-on: log in once and every
guarded service (this homepage, search, chat, media requests, and friends) opens
for you without a second password.</p>
<ul>
<li>Need an account? Ask Frank. Accounts are issued by hand; there is no signup.</li>
<li>Your clearance lives in a cookie. Clearing cookies means re-clearing both doors.</li>
<li>If a page asks you to log in repeatedly, that is a server-side problem — tell Frank, don't brute-force it.</li>
</ul>
</section>
<section id="chat">
<h1>Talking to the AI</h1>
<p>The everyday front door is <a href="https://ai.franksdank.cloud">Open WebUI</a>
(it has its own login the first time; use the same email you use everywhere).
Pick a model in the chat's model selector — the three models the fleet runs are
described <a href="#models">below</a>. Open WebUI keeps your chat history; the raw
model endpoints do not.</p>
<p>Images go to <code>frank-primary</code> or <code>frank-small</code>
(both see them). Audio (WAV) goes to <code>frank-small</code> only.</p>
</section>
<section id="models">
<h1>The running models</h1>
<p>Three models serve the family. The homepage status cards show whether
each one is up right now.</p>
<h2>frank-large — deep reasoning</h2>
<ul>
<li><strong>Model:</strong> Qwen3.8-27B (UD-Q8_K_XL, 31.5 GiB).</li>
<li><strong>Capacity:</strong> 262k-token context, exactly one conversation at a time. Text only.</li>
<li><strong>Inputs:</strong> text.</li>
<li><strong>Best for:</strong> cybersecurity and debugging. <em>Use requires Frank's permission.</em></li>
<li><strong>Not for:</strong> images. It cannot see them.</li>
</ul>
<h2>frank-primary — the primary model</h2>
<ul>
<li><strong>Model:</strong> Qwen3.8-Flash-Next (Q8_0, 175.3 GiB) with MTP + n-gram speculative drafting on both R9700s.</li>
<li><strong>Capacity:</strong> 262k-token context, exactly one conversation at a time; sees images.</li>
<li><strong>Inputs:</strong> text, image.</li>
<li><strong>Best for:</strong> software engineering, system administration, and instruction tasks.</li>
</ul>
<h2>frank-small — the fast utility model</h2>
<ul>
<li><strong>Model:</strong> Gemma4-12B QAT (Balanced, Q4_K_M, 7.4 GiB).</li>
<li><strong>Capacity:</strong> one million tokens of pooled context across four simultaneous conversations (up to 262k each); fastest at images.</li>
<li><strong>Inputs:</strong> text, image, audio.</li>
<li><strong>Best for:</strong> agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence, and anything with a picture (screenshots, photos, diagrams).</li>
</ul>
<p class="foot-note">Which one to pick? Small utility work, RAG, search, and
screenshots: <code>frank-small</code>. Software engineering, sysadmin, and
instruction tasks: <code>frank-primary</code>. Cybersecurity and debugging:
<code>frank-large</code> (with Frank's permission).</p>
</section>
<section id="media">
<h1>Media &amp; files</h1>
<ul>
<li><strong>Movies &amp; TV:</strong> request with <a href="https://seerr.franksdank.cloud">Seerr</a>;
watch with <a href="https://plex.franksdank.cloud/web/index.html">Plex</a> or
<a href="https://jelly.franksdank.cloud">Jellyfin</a>. Don't ask the *arrs directly —
Seerr is the front door, and requests ride the tunnel from there.</li>
<li><strong>Books:</strong> <a href="https://books.franksdank.cloud">Calibre-Web</a> — the e-book library, with its own login.</li>
<li><strong>Games:</strong> <a href="https://romm.franksdank.cloud">RomM</a> — ROM library and emulator collection.</li>
<li><strong>Raw files:</strong> <a href="https://files.franksdank.cloud">Copyparty</a> — direct downloads and shares.</li>
<li><strong>Search:</strong> <a href="https://search.franksdank.cloud">SearXNG</a> — metasearch, VPN-routed, no AI snoop.</li>
</ul>
</section>
<section id="troubleshooting">
<h1>When something looks broken</h1>
<ul>
<li><strong>A status card says DOWN.</strong> Reload the page once; the checker
re-probes every few minutes. If it stays down, that service really is unhappy — tell Frank.</li>
<li><strong>"status unavailable" on the homepage.</strong> The page couldn't read
its status file. Hard-reload (Ctrl/Cmd-Shift-R).</li>
<li><strong>Connection timeout on a home device.</strong> Nine times out of ten
the device isn't using the house DNS (Pi-hole at 192.168.0.143) and resolved
our names to the public address, which doesn't answer. Point the device's DNS at
192.168.0.143 and retry.</li>
<li><strong>The bot-check spins forever.</strong> Let it finish; heavy JS blocking
(privacy extensions) can starve it. Pause the blocker, reload, resume.</li>
</ul>
</section>
</main>
<footer class="foot">
<span><a href="/">← services</a></span>
<span>docs · written by the chief designer · ask Frank for anything missing</span>
</footer>
<script>
// The same bare-mode toggle the service pages use (?bare=1), for this static page.
(function () {
var bare = new URLSearchParams(window.location.search).get('bare') === '1';
document.documentElement.setAttribute('data-bare', bare ? '1' : '0');
var t = document.getElementById('bare-toggle');
if (t) {
t.textContent = bare ? 'styled' : 'bare';
t.href = bare ? '?bare=0' : '?bare=1';
}
})();
</script>
</body>
</html>