The conversation battery (scripts/bench-conversation.py, salted, same 10
prompts, 2026-09-20) measured three arms on the two R9700s:
arm I llama.cpp layer split 32:33 + DFlash2, Q8_0 geo decode 45.7 t/s, TTFT 0.72 s, burst-4 71.7 t/s
arm L llama.cpp tensor split 1:1 (RCCL), no drafter geo decode 30.5 t/s, TTFT 0.71 s, burst-4 54.3 t/s
arm R radiance vLLM TP=2 + DFlash2-FP8, MXFP4 geo decode 95.8 t/s, TTFT 0.21 s, burst-4 259.4 t/s
Radiance wins every axis, so it takes the alias. The board swap is what made
arm R possible: TP=2 was archived on 2026-09-16 because the second card sat
behind B650 chipset bridges with AtomicOpsCap: Routing-, and the ProArt
B850-Creator put both cards on CPU GPP root ports (Gen5 x8, Routing+, ReqEn+).
The R4D one-shot all-reduce needs no RCCL collective anyway.
- un-retire dankserver/frank-medium/: the TP=2 quadlet (banner rewritten —
it is the deployed shape now), radiance-entrypoint.sh, the healthcheck.
The replica pair and its affinity proxy stay archived; one process owns
both cards now.
- retire scripts/frank-medium.{service,env} + serve-frank-medium.sh and
bench-spec-ab.py to scripts/.AGENTTRASH/20260920-llama-layersplit/ as the
rollback path. build-llama-rdna4.sh and download-frank-medium-gguf.sh
stay: they are the tooling that rollback needs.
- deploy-remote.sh: deploy_llm_medium now gates on the image (pull once,
re-tag to the localhost name the quadlet resolves), the MXFP4 checkpoint,
the drafter, the pinned patch clone and the patched libr4d; retires the
llama root unit into the home trash instead of relinking it; then
install_quadlet + wait_container_healthy 1800s (first start compiles).
retire_vllm_medium_quadlets becomes retire_vllm_replica_quadlets — the
a/b/proxy trio only, since frank-medium is installed again.
- planner + test-changed map dankserver/frank-medium/** to frank-medium
and its seam; the scripts-trio mapping is gone with the files.
- tests/frank_medium_unit.bats rewritten for the container contract: boot
persistence of a GENERATED unit (no enable on this host's v1 generator),
TP=2 with the atomics-era NCCL workarounds kept out, the DFlash2 depth vs
the MXFP4 decode-kernel row band, the KV pin's coupling to the shape it
was calibrated at plus pool/VRAM arithmetic, the endpoint/tool/reasoning
contract, and the archive invariants.
- homepage card + docs updated, and they were stale twice over: they still
described the ajgazin Swift 35:30 / 393k payload, not the deployed shape.
Known regression, documented in the unit: vision. The MXFP4 checkpoint is
language_model_only and the launcher loads no projector, so images are a 400
here; the harness's vision role already routes to frank-small.
127 lines
6.3 KiB
HTML
127 lines
6.3 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
<title>frank — docs</title>
|
|
<base href="/">
|
|
<link rel="stylesheet" href="style.css">
|
|
</head>
|
|
<body>
|
|
<header class="top">
|
|
<span class="title">frank@home:~$ docs</span>
|
|
<a id="bare-toggle" class="toggle" href="?bare=1">bare</a>
|
|
</header>
|
|
<main class="docs">
|
|
|
|
<section id="getting-in">
|
|
<h1>Getting in</h1>
|
|
<p>Everything here lives behind two doors. The first is <strong>Anubis</strong>, a
|
|
quiet bot-check: the first time you visit any guarded page in a session you see a
|
|
short "making sure you're not a bot" screen. It runs once. The second door is
|
|
<strong>Authentik</strong>, the family single sign-on: log in once and every
|
|
guarded service (this homepage, search, chat, media requests, and friends) opens
|
|
for you without a second password.</p>
|
|
<ul>
|
|
<li>Need an account? Ask Frank. Accounts are issued by hand; there is no signup.</li>
|
|
<li>Your clearance lives in a cookie. Clearing cookies means re-clearing both doors.</li>
|
|
<li>If a page asks you to log in repeatedly, that is a server-side problem — tell Frank, don't brute-force it.</li>
|
|
</ul>
|
|
</section>
|
|
|
|
<section id="chat">
|
|
<h1>Talking to the AI</h1>
|
|
<p>The everyday front door is <a href="https://ai.franksdank.cloud">Open WebUI</a>
|
|
(it has its own login the first time; use the same email you use everywhere).
|
|
Pick a model in the chat's model selector — the three models the fleet runs are
|
|
described <a href="#models">below</a>. Open WebUI keeps your chat history; the raw
|
|
model endpoints do not.</p>
|
|
<p>There is no audio input anywhere on the fleet. Text in, and — on
|
|
<code>frank-small</code> only — images too.</p>
|
|
</section>
|
|
|
|
<section id="models">
|
|
<h1>The running models</h1>
|
|
<p>Three models serve the family, each on its own hardware. The homepage status
|
|
cards above show whether each one is up right now.</p>
|
|
|
|
<h2>frank-large — the big brain</h2>
|
|
<ul>
|
|
<li><strong>Model:</strong> Qwen3.8-Flash-Next, w4b quantization, served by halogen-flash-server.</li>
|
|
<li><strong>Hardware:</strong> Dankclaw's Strix Halo iGPU (gfx1151), shared with the whole host.</li>
|
|
<li><strong>Best for:</strong> long, thoughtful text conversations. 200k-token context per conversation.</li>
|
|
<li><strong>Capacity:</strong> exactly two conversations resident at once. A third waits — politely, but it waits.</li>
|
|
<li><strong>Not for:</strong> images. It cannot see them.</li>
|
|
</ul>
|
|
|
|
<h2>frank-small — the eyes</h2>
|
|
<ul>
|
|
<li><strong>Model:</strong> Gemma4-12B QAT (HauhauCS Balanced, Q4_K_M) with MTP + n-gram speculative decoding.</li>
|
|
<li><strong>Hardware:</strong> Dankclaw's RTX 3090.</li>
|
|
<li><strong>Best for:</strong> anything with a picture — screenshots, photos, diagrams. Fastest decode of the fleet at long context, and the endpoint that gets most daily traffic.</li>
|
|
<li><strong>Capacity:</strong> one million tokens of pooled context across four simultaneous conversations (up to 262k each).</li>
|
|
</ul>
|
|
|
|
<h2>frank-medium — the 27B generalist</h2>
|
|
<ul>
|
|
<li><strong>Model:</strong> Qwen3.8-27B-Uncensored at MXFP4, on vllm-radiance with a DFlash2 draft head.</li>
|
|
<li><strong>Hardware:</strong> Dankserver's two Radeon R9700s as ONE tensor-parallel engine (TP=2).</li>
|
|
<li><strong>Best for:</strong> general chat and reasoning when the big brain is busy; the biggest pure-language model we run, and the fastest we have measured.</li>
|
|
<li><strong>Capacity:</strong> 262k-token context, shared by up to eight conversations at once. Text only — pictures go to <code>frank-small</code>.</li>
|
|
</ul>
|
|
|
|
<p class="foot-note">Which one to pick? Screenshots and photos: <code>frank-small</code>.
|
|
Deep long-text work: <code>frank-large</code> (mind the two-seat limit).
|
|
When in doubt: <code>frank-medium</code>.</p>
|
|
</section>
|
|
|
|
<section id="media">
|
|
<h1>Media & files</h1>
|
|
<ul>
|
|
<li><strong>Movies & TV:</strong> request with <a href="https://seerr.franksdank.cloud">Seerr</a>;
|
|
watch with <a href="https://plex.franksdank.cloud/web/index.html">Plex</a> or
|
|
<a href="https://jelly.franksdank.cloud">Jellyfin</a>. Don't ask the *arrs directly —
|
|
Seerr is the front door, and requests ride the tunnel from there.</li>
|
|
<li><strong>Books:</strong> <a href="https://books.franksdank.cloud">Calibre-Web</a> — the e-book library, with its own login.</li>
|
|
<li><strong>Games:</strong> <a href="https://romm.franksdank.cloud">RomM</a> — ROM library and emulator collection.</li>
|
|
<li><strong>Raw files:</strong> <a href="https://files.franksdank.cloud">Copyparty</a> — direct downloads and shares.</li>
|
|
<li><strong>Search:</strong> <a href="https://search.franksdank.cloud">SearXNG</a> — metasearch, VPN-routed, no AI snoop.</li>
|
|
</ul>
|
|
</section>
|
|
|
|
<section id="troubleshooting">
|
|
<h1>When something looks broken</h1>
|
|
<ul>
|
|
<li><strong>A status card says DOWN.</strong> Reload the page once; the checker
|
|
re-probes every few minutes. If it stays down, that service really is unhappy — tell Frank.</li>
|
|
<li><strong>"status unavailable" on the homepage.</strong> The page couldn't read
|
|
its status file. Hard-reload (Ctrl/Cmd-Shift-R).</li>
|
|
<li><strong>Connection timeout on a home device.</strong> Nine times out of ten
|
|
the device isn't using the house DNS (Pi-hole at 192.168.0.143) and resolved
|
|
our names to the public address, which doesn't answer. Point the device's DNS at
|
|
192.168.0.143 and retry.</li>
|
|
<li><strong>The bot-check spins forever.</strong> Let it finish; heavy JS blocking
|
|
(privacy extensions) can starve it. Pause the blocker, reload, resume.</li>
|
|
</ul>
|
|
</section>
|
|
|
|
</main>
|
|
<footer class="foot">
|
|
<span><a href="/">← services</a></span>
|
|
<span>docs · written by the chief designer · ask Frank for anything missing</span>
|
|
</footer>
|
|
<script>
|
|
// The same bare-mode toggle the service pages use (?bare=1), for this static page.
|
|
(function () {
|
|
var bare = new URLSearchParams(window.location.search).get('bare') === '1';
|
|
document.documentElement.setAttribute('data-bare', bare ? '1' : '0');
|
|
var t = document.getElementById('bare-toggle');
|
|
if (t) {
|
|
t.textContent = bare ? 'styled' : 'bare';
|
|
t.href = bare ? '?bare=0' : '?bare=1';
|
|
}
|
|
})();
|
|
</script>
|
|
</body>
|
|
</html>
|