This website requires JavaScript.
71fe8c008f
SearXNG: move the trio out of compose into quadlet deployment
Frank
2026-08-18 23:34:08 -07:00
6e9bfd5d3e
SearXNG: define the gluetun+searxng+valkey trio as quadlet units
Frank
2026-08-18 23:20:28 -07:00
aef7eefa35
Gluetun: healthcheck via the tun0 tunnel interface
Frank
2026-08-18 18:12:04 -07:00
39f43d010a
Gluetun: fix healthcheck (no curl in image; /health is not an endpoint)
Frank
2026-08-18 18:00:10 -07:00
c390135d56
SearXNG: trusted group needs Caddy XFF; trust the podman bridge
Frank
2026-08-18 17:52:44 -07:00
9ca22d1c34
Gluetun/Valkey: use fully-qualified docker.io image names
Frank
2026-08-18 16:39:06 -07:00
9fd82dc6e1
Reduce CTX_CHECKPOINTS from 32 to 16 in env file to free ~16 GiB VRAM and prevent OOM crashes
Frank
2026-08-18 16:12:52 -07:00
8c391fe120
Reduce ctxcp from 32 to 16 to free ~16 GiB VRAM headroom and prevent OOM crashes during prompt processing
Frank
2026-08-18 16:11:58 -07:00
a046d2c89d
Gluetun: pass full single-peer WireGuard config to the container
Frank
2026-08-18 15:16:50 -07:00
f846b8a36e
test: skip ComfyUI GPU migration tests until the ROCm build lands
Frank
2026-08-18 13:55:37 -07:00
b4f1043b1a
Gitea: document rootless privileged-port sysctl fix
Frank
2026-08-18 13:55:37 -07:00
e3cd2aca1b
SearXNG: add limiter.toml trusted group (LAN /24), wire into deploy
Frank
2026-08-18 13:55:37 -07:00
e5f21a5562
AGENTS.md: document the Pi models.json location and resolution rules
Frank
2026-08-18 09:00:05 -07:00
a08be1641f
frank-default-large: full native 262144 context per slot (524288 total)
Frank
2026-08-18 08:01:17 -07:00
819b9d448a
Docs: reflect the 2026-08-18 second swap (rename + Qwen3.6 restore)
Frank
2026-08-18 01:29:21 -07:00
1ba9618fcc
Swap fleet naming + restore Qwen3.6-35B-A3B Q6_K_P on Dankserver
Frank
2026-08-18 00:32:26 -07:00
fe1d834d37
docs: note 128K context and thinking on for frank-default-small
Frank
2026-08-17 23:12:32 -07:00
2e1a78aeb4
frank-default-small: 128K context and thinking on
Frank
2026-08-17 23:11:13 -07:00
64cdfc8529
docs: reflect CIRU runtime on Dankclaw in specs and deployment pages
Frank
2026-08-17 21:14:16 -07:00
3b51b5a94a
dankclaw: replace vLLM with CIRU ROCmFPX llama.cpp runtime
Frank
2026-08-17 21:12:35 -07:00
4941d911f2
docs: correct Dankclaw RAM figure (64 GiB, 61 GiB visible)
Frank
2026-08-17 20:34:51 -07:00
044dd49011
docs: note prefix caching enabled on frank-default-small (opt-in for hybrid; ~4x TTFT on repeated prefix)
Frank
2026-08-17 18:45:49 -07:00
b854ec78ad
frank-default-small: enable prefix caching (opt-in for hybrid models; big TTFT win for pi's repeated system prompt)
Frank
2026-08-17 18:25:28 -07:00
1de43f6777
docs: update sibling KB for the 2026-08-17 fleet redo (llama.cpp moved to Dankserver, vLLM on Dankclaw; OpenWebUI/Hermes backend addresses)
Frank
2026-08-17 18:10:57 -07:00
8c42771877
docs: mark retired FastFlowLM/NPU pages with the port/name handoff; tighten Dankclaw system specs runtime row
Frank
2026-08-17 18:10:57 -07:00
115b0af8dc
docs: reflect the 2026-08-17 evening changes (Quark model swap, tool calling, batch 8192)
Frank
2026-08-17 18:01:02 -07:00
1cebbf47f6
frank-default-small: swap to nameistoken Quark W8A8 INT8, batch 8192
Frank
2026-08-17 17:49:27 -07:00
a17e638a1f
frank-default-small: stronger tool template with a concrete worked example
Frank
2026-08-17 17:38:00 -07:00
20108e5b7a
frank-default-small: use the repo chat_template.jinja so tools render in Qwen3 XML
Frank
2026-08-17 17:13:35 -07:00
5108af11aa
frank-default-small: enable tool calling (--enable-auto-tool-choice --tool-call-parser qwen3_xml)
Frank
2026-08-17 16:50:23 -07:00
c81933ccb4
OpenWebUI: retarget to Dankserver llama.cpp (:7770 moved in 2026-08-17 redo)
Frank
2026-08-17 16:39:30 -07:00
57264f7faa
record performance fixes: FA_ALL_QUANTS build flag (dankserver), RDNA triton FA env (dankclaw)
Frank
2026-08-17 16:30:22 -07:00
27788f6f2c
fix: make serve scripts + deploy.sh executable
Frank
2026-08-17 15:24:40 -07:00
d9af7f1d19
fleet redo 2026-08-17: retire PP+NPU, new topology
Frank
2026-08-17 15:23:46 -07:00
136ee080c9
docs: measured PP link math (H=5120 verified, 20.5 MB/ubatch wire), NIC inventory
Frank
2026-08-17 05:06:41 -07:00
968066ff23
docs: boot policy (only PP units enabled), deploy.sh dankserver hazard, stale-claim cleanup
Frank
2026-08-17 03:30:00 -07:00
3f88b8e1ed
docs + script defaults: Q6_K swap, q8_0/f16 KV, split 0.68, FA rebuild
Frank
2026-08-17 02:08:08 -07:00
11f37dafd8
frank-default-large: split 0.68,0.32 = 45 blocks on 3090 (Q6_K phase 2)
Frank
2026-08-17 02:05:04 -07:00
1bc39c43ce
frank-default-large: swap to Qwen3.8-27B-Uncensored Q6_K, q8_0/f16 KV
Frank
2026-08-17 02:01:22 -07:00
e9ec5cd13f
docs: NPU endpoint now serves qwythos:9b (2026-08-17 swap, FLM 1.0.1)
Frank
2026-08-17 01:24:43 -07:00
aa1941d592
docs + guard: single 262k slot state, split-limits analysis, cache incident
Frank
2026-08-17 01:24:06 -07:00
c85ff6c6fd
frank-default-small: NPU model swap qwen3.6-moe:35b-a3b -> qwythos:9b (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated)
Frank
2026-08-17 01:22:32 -07:00
2b96043740
frank-default-large: split 0.53,0.47 - provably-safe max (34 blocks)
Frank
2026-08-17 01:21:21 -07:00
193034fdb8
frank-default-large: split 0.54,0.46 - 35 blocks on 3090 (0.55 overran to 36 -> OOM)
Frank
2026-08-17 00:58:06 -07:00
accf90b6fa
frank-default-large: split 0.55,0.45 = 35 blocks on the 3090 (true max)
Frank
2026-08-17 00:34:58 -07:00
53ada02a63
frank-default-large: PARALLEL=1, CTX 262144, split 0.50 (max 3090 weight)
Frank
2026-08-16 23:53:09 -07:00
c353725ea5
docs: cross-link the vllmScripts project (model endpoints + git host)
Frank
2026-08-16 23:19:54 -07:00
7fe271e65d
docs: llama.cpp stack at production state (2026-08-16) + sibling-project links
Frank
2026-08-16 23:19:21 -07:00
f5a78db017
docs: FLM rejects developer role — root cause of pi 'Stream ended without finish_reason'
Frank
2026-08-16 20:54:13 -07:00
04112e88e6
fix: gitea boot persistence via quadlet (incident 2026-08-16)
Frank
2026-08-16 18:31:38 -07:00
9ec5678ef0
docs: qwen3.6-moe unreachable incident (2026-08-16) + topology reconciliation
Frank
2026-08-16 15:50:15 -07:00
50c55ee220
frank-default-small: swap to qwen3.6-moe:35b-a3b at 262144 ctx (native max), keep turbo+asr
Frank
2026-08-16 14:48:33 -07:00
2f1999b2e9
docs: frank-default-small live config — turbo, ASR, 131072 ctx verified
Frank
2026-08-16 14:38:37 -07:00
fd4746422d
frank-default-small: pmode turbo, ASR (Whisper-V3-Turbo) on, ctx-len 131072 (max verified)
Frank
2026-08-16 14:37:32 -07:00
02a1447602
head alias: frank-default -> frank-default-large (match renamed service)
Frank
2026-08-16 14:27:48 -07:00
d0ade28f6d
docs: fleet topology update — PP split renamed frank-default-large, NPU endpoint promoted to frank-default-small, 4-model NPU bakeoff matrix (2026-08-16)
Frank
2026-08-16 14:22:13 -07:00
3f42210922
fix: mark serve-flm-small.sh executable (unit failed EXEC 203)
Frank
2026-08-16 14:20:17 -07:00
f6737f234d
promote NPU endpoint to frank-default-small: FLM serve unit + wrapper + env (gemma4-it:e4b default), LimitMEMLOCK=infinity (2026-08-16)
Frank
2026-08-16 14:19:37 -07:00
48142f5d8e
rename frank-pp-head to frank-default-large: PP split is the new production large endpoint (2026-08-16)
Frank
2026-08-16 13:44:31 -07:00
79c913aaa7
docs: NPU parallelism measured — single-lane FIFO serialization, aggregate t/s flat at N=1/2/4
Frank
2026-08-16 13:12:46 -07:00
d48227c7f7
docs: gpt-oss-20b landed on the Dankclaw NPU — 19.3 t/s decode measured, memlock fix, serve state (2026-08-16)
Frank
2026-08-16 12:57:44 -07:00
0799fc5eb4
docs: FastFlowLM NPU research — AMD NPU runtime, 50 TOPS Strix Halo fit analysis, verified install state, test runbook (2026-08-16)
Frank
2026-08-16 12:29:11 -07:00
934c6d8be1
pp-head: 2x200700 f16 KV (ctx 401400, parallel 2), split 0.40,0.60 = max 3090 capacity
Frank
2026-08-16 12:23:08 -07:00
3ad4c6921c
pp-head: alias frank-default, verbosity 4 (interactive use)
Frank
2026-08-16 10:42:29 -07:00
750730e4d9
pp-test: restore drops caches before production load; all-mode EXIT trap auto-restores; ssh ConnectTimeout
Frank
2026-08-16 10:17:01 -07:00
1716bf37c9
pp-test: preflight production check via health endpoint (no sudo needed)
Frank
2026-08-16 05:48:17 -07:00
9544c36afc
PP kit + Strix research: Qwen3.8-27B RPC split (Dankclaw head / Dankserver worker)
Frank
2026-08-16 05:40:27 -07:00
c775077016
dankclaw: add MTP=1 toggle - prepend draft-mtp before --spec-default ngram-mod fallback
Frank
2026-08-15 22:31:29 -07:00
b6f2ee2945
docs: record --spec-default + Vulkan0 + PARALLEL=2 final state; 262k-ctx ROCm-vs-Vulkan bench
Frank
2026-08-15 17:13:18 -07:00
5296cf11d7
dankclaw: switch backend to Vulkan0 (48% faster decode at 262k ctx); add bench-high-ctx.sh
Frank
2026-08-15 17:03:01 -07:00
1dbc70cb32
dankclaw: retire DSpark, use llama.cpp --spec-default (ngram-mod) preset; PARALLEL=2 trial
Frank
2026-08-15 15:15:06 -07:00
020e874a5c
docs: record DSpark single-slot switch + ROCm parallel crash (2026-08-15) across KB; refs AGENTS/deployment/benchmark/matrix/index
Frank
2026-08-15 14:36:34 -07:00
d04cc69e09
dankclaw: add accurate DSpark stream/queue test (PARALLEL=1) - reports true prompt_tokens; single+concurrent 16k pass
Frank
2026-08-15 14:32:21 -07:00
6266e43f10
dankclaw: DSpark parallel crash - ROCm illegal memory access under 2x16k prose, pin PARALLEL=1 per user
Frank
2026-08-15 14:24:27 -07:00
b94bad066b
dankclaw: switch frank-default-large spec to DSpark sidecar draft (magnitudedev Qwen3.8-27B-DSpark Q8_0, n_max 7)
Frank
2026-08-15 14:00:30 -07:00
40b079fe1e
unit: refresh frank-default-large description (Qwen3.8-27B UD-Q8_K_XL, MTP+ngram-mod+ngram-map-k4v)
Frank
2026-08-14 21:01:22 -07:00
8d64adbcbe
dankclaw: production spec -> 3-layer chain draft-mtp,ngram-mod,ngram-map-k4v (user request; ~equal real-prose speed)
Frank
2026-08-14 21:00:05 -07:00
2d7290a2a8
AGENTS: MANDATORY benchmarking policy - stop production service before any bench (serves ~3 real users); restart + health-check after
Frank
2026-08-14 20:54:34 -07:00
0eda208705
docs: Qwen3.8 spec-matrix benchmark page; add matrix tools; final env n_max=5
Frank
2026-08-14 14:28:51 -07:00
a818be70dc
dankclaw Qwen3.8: winner n_max=5 (13.3 t/s real), not matrix n_max=10
Frank
2026-08-14 14:26:14 -07:00
75bb9bb53e
dankclaw Qwen3.8: batch 8192/ubatch 256 - fixes MTP collapse + premature-EOS
Frank
2026-08-14 13:59:20 -07:00
57b0be037c
dankclaw: switch frank-default-large to unsloth Qwen3.8-27B UD-Q8_K_XL
Frank
2026-08-14 13:26:17 -07:00
d72092b2c7
docs: record dual-f16 KV refit on frank-default-small (2026-08-14)
Frank
2026-08-14 12:59:10 -07:00
f96aa2b692
frank-default-small: refit ctx 800000->700000 for dual-f16 KV pool
Frank
2026-08-14 12:55:44 -07:00
f70460b002
frank-default-small: KV K+V to f16, add verbosity tunable (F16 KV diag)
Frank
2026-08-14 12:50:59 -07:00
bea52138d2
docs: record Dankserver VRAM/RAM growth investigation (2026-08-14)
Frank
2026-08-14 12:44:24 -07:00
9d7643ec5d
frank-default-small: ncmoe 35->36 for prefill stability
Frank
2026-08-14 10:56:40 -07:00
7daf9896e5
docs: record Ling freeze diagnosis - spec path deadlock, resolution spec-off parallel=2
Frank
2026-08-14 03:49:42 -07:00
139686043b
ling B-test: spec off (--spec-type none), keep parallel=2 - isolate spec path as freeze trigger
Frank
2026-08-14 03:39:15 -07:00
7ba86812f7
ling A/B: verbosity 4->0, drop log file to isolate hang trigger (match known-good run)
Frank
2026-08-14 02:32:08 -07:00
6a6d673dca
ling A/B: revert batch 8192->3072 to isolate mid-stream freeze (suspect batch)
Frank
2026-08-14 02:20:52 -07:00
a5444e2c4d
ling: batch 8192 (ubatch 3072); add -lv verbosity + log file for stop-investigation
Frank
2026-08-14 02:12:35 -07:00
321f946660
ling: ubatch 512 -> 3072 (batch 3072) - using heap headroom, watch for OOM
Frank
2026-08-14 01:46:36 -07:00
a5b12e1062
ling: keep draft-mtp,ngram-mod active (A/B wash vs MTP alone, parallel-safe)
Frank
2026-08-14 01:45:22 -07:00
6647690ebd
ling A/B: bench draft-mtp alone vs draft-mtp,ngram-mod
Frank
2026-08-14 01:43:53 -07:00
3204b4b29e
ling trial: layer ngram-mod under draft-mtp; add ngram args to serve script
Frank
2026-08-14 01:40:46 -07:00
c275cbaf57
docs: Ling-3.0-flash Q6_K trial page (Vulkan+GTT memory research, switch-back steps)
Frank
2026-08-14 01:22:02 -07:00
923e775354
dankclaw: Ling-3.0-flash Q6_K trial config (Vulkan+GTT+draft-mtp)
Frank
2026-08-14 01:20:30 -07:00
db69fc8192
docs: pipeline parallelism research + educational sample script set
Frank
2026-08-14 00:22:23 -07:00
8d1d9baa82
docs: MTP wins the DFlash replacement; record parallel premature-EOS incident
Frank
2026-08-13 23:27:06 -07:00
Frank