Frank
d9af7f1d19
fleet redo 2026-08-17: retire PP+NPU, new topology
...
- Move all previous scripts to old/scripts/ (retired PP/NPU-era)
- frank-default-large: llama.cpp FastMTP on dankserver (HauhauCS
Qwen3.8-27B IQ4_XS + FastMTP 32K draft + ngram-mod, K q8_0/V f16,
128K ctx, batch 2048/ubatch 1024, port 7770)
- frank-default-small: vLLM ROCm on dankclaw (Avesed Qwen3.6-35B-A3B
INT8 W8A8, full 262144 ctx, no KV quant, 4 seqs, batch 16384,
ngram spec, port 52625)
- retarget deploy.sh (dankserver/dankclaw, inline sudo, no helpers)
- docs: deployment + index + AGENTS.md + README for new topology,
superseded banners on retired docs, add frank-redo-2026-08-17 runbook
2026-08-17 15:23:46 -07:00
Frank
136ee080c9
docs: measured PP link math (H=5120 verified, 20.5 MB/ubatch wire), NIC inventory
...
- Ethernet Bandwidth Math: new section 0 with live measurements
(iperf 281-294 MB/s, ping RTT both paths, prefill 457.9 t/s at
2.5GbE, per-ubatch wire 20.5 MB, 98k KV snapshot 3.45 GB blocking
the slot 13.6 s, validated link-speed model: 386/458/505 t/s)
- System specs: NIC tables for both boxes; the 1 GbE USB-C adapter on
the claw was silently routing the RPC path (unplugged 2026-08-17)
- PP Deployment: fix hidden-state size (10 KiB/direction, H=5120) and
RTT-vs-bandwidth claim (decode RTT-bound; prefill+snapshot BW-bound)
2026-08-17 05:06:41 -07:00
Frank
968066ff23
docs: boot policy (only PP units enabled), deploy.sh dankserver hazard, stale-claim cleanup
...
- deployment.md: Boot Policy (2026-08-17) section (disabled units table,
re-enable recipes, stale frank-restart-small.sh helper, broken
deploy.sh dankserver target); Update Dankserver -> frank-pp-worker;
condensed retired 35B history block and 2026-08-13 incident
- AGENTS.md: boot-policy note, target map fix (Q6_K, split 0.68),
deploy.sh dankserver warning, obsolete quirk entries condensed
- README.md: rewrite active services/models tables (PP head + worker)
- index.md: topology fix (qwythos:9b, Q6_K split 0.68, NPU disabled)
- qwen3.8-spec-matrix.md: superseded configs merged into compact
history; stale inline env block removed; bundled-head MTP note
- FastFlowLM, PP Deployment: boot-policy + enablement notes
- System Specs x2, DFlash doc, 35B FP8 doc, Ling docs: stale active
claims fixed, retired banners, never-shipped plan stubbed
2026-08-17 03:30:00 -07:00
Frank
3f88b8e1ed
docs + script defaults: Q6_K swap, q8_0/f16 KV, split 0.68, FA rebuild
...
- spec matrix + PP doc + deployment.md + AGENTS.md + index.md: current
production = Uncensored Q6_K + standalone MTP draft, mismatched
q8_0/f16 KV with FA retained (GGML_CUDA_FA_ALL_QUANTS rebuild at
9b05354ec), split 0.68 (45 blocks), KV 8624/3920, 273 MiB free.
- PP doc: 'FA-all-quants rebuild' section (flag, cmake commands, both
hosts) + new-model split-limits measurements; old Q8_K_XL limits
marked superseded.
- serve-pp-head.sh defaults: MODEL_FILE -> Q6_K, CACHE_TYPE_V -> f16
(env already correct; defaults now match production).
2026-08-17 02:08:08 -07:00
Frank
aa1941d592
docs + guard: single 262k slot state, split-limits analysis, cache incident
...
- spec matrix + PP doc: production state = 1 slot @ 262144, split 0.53
(34 blocks), KV 8192/8192, 3090 at 23460/24576. New 'Split limits
(measured 2026-08-17)' section: 0.50/0.54/0.55 boundary data, the
KV-per-device rounding rule (round(f x 66)), and why 35 blocks is
unreachable (9-FA flip at f~0.538).
- PP doc: cache incident writeup - full Dankserver disk -> 0-byte cache
entries -> corrupt weights -> degenerate output; fix + signature.
- deployment.md + AGENTS.md + index.md: numbers brought current.
- scripts/prune-rpc-cache.sh: cron guard capping the worker cache at
40 GiB (oldest first) to prevent the disk-full recurrence.
2026-08-17 01:24:06 -07:00
Frank
7fe271e65d
docs: llama.cpp stack at production state (2026-08-16) + sibling-project links
...
- spec matrix: new 'Production state' section — PP head (RPC0+ROCm0,
split 0.40/0.60), pool 401400 / n_ctx_slot 200704 live, MTP n_max 4
over ngram-mod fallback (journal acceptance stats), f16 KV.
2026-08-15 standalone state marked superseded (Vulkan0 verdict was
standalone-only; PP head runs ROCm0 by default).
- deployment.md: fix frank-default-large section (MTP not 'no draft',
ROCm0 not Vulkan0, 200704/slot not 262144, f16 not q8_0 KV,
serve-pp-head.sh not serve-llama-frank-large.sh).
- PP deployment doc: production-state section; test-day open questions
resolved with journal evidence.
- index.md: NPU line now qwen3.6-moe:35b-a3b @262k ctx; Sibling
projects section -> serverAppConfigs.
- AGENTS.md: overview reflects PP+MTP reality; sibling pointer.
2026-08-16 23:19:21 -07:00
Frank
d0ade28f6d
docs: fleet topology update — PP split renamed frank-default-large, NPU endpoint promoted to frank-default-small, 4-model NPU bakeoff matrix (2026-08-16)
2026-08-16 14:22:13 -07:00
Frank
9544c36afc
PP kit + Strix research: Qwen3.8-27B RPC split (Dankclaw head / Dankserver worker)
...
- scripts: serve-pp-head/worker, envs, systemd units, pp-test.sh orchestrator, pp-bench.py
- head design verified in b10433 source + live 4B smoke test:
* no --device with --rpc (RPC device silently dropped otherwise)
* default device selection = [RPC0, ROCm0] (HIP registers before Vulkan)
* default split proportional to reported free mem = ~20/80 here, pin 1,1
* -dio required for large model over RPC
- docs: qwen3.8-strix-optimizations (ranked no-quant-downgrade levers,
quant landscape, 2026-08-16 test matrix), PP deployment runbook,
index updates, Dankclaw Improvements corrections (Vulkan0, no power_profile sysfs)
- worker built at 9b05354ec in /home/dank/llama-cpp-rpc worktree (CUDA+RPC)
- second head build build-vulkan-rpc (HIP off) for backend A/B
2026-08-16 05:40:27 -07:00
Frank
db69fc8192
docs: pipeline parallelism research + educational sample script set
...
Compiles the 5-subagent research fan-out (llama.cpp RPC, vLLM PP,
bandwidth math, mixed-vendor experiments, Qwen3.6-27B spec methods) into
the KB under docs/Pipeline Parallelism/ with an index, expected
performance estimates (1/2.5/10 GbE), and an EDUCATIONAL-ONLY two-node
Qwen3.6-27B Q8 split script set (MTP/DFlash/ngram-mod variants, DSpark
noted as vLLM-only). Nothing here is deployed or intended to run.
2026-08-14 00:22:23 -07:00
Frank