Frank
12c5e885f1
Finishes the purge. These two surfaces were left out of the earlier commit
because the memory store is auto-generated, but both are read back into every
future session, so the name was still reaching the agent.
58 sites across 16 files: hostname, both LAN addresses, the SSID. Special-cased
the compound names a blind token swap breaks ('dank@the workstation/dankclaw',
'on the workstation/Dankclaw/Dankserver') and reworded the four that came out
ungrammatical. Facts kept: the hardware inventory of the box, the noctalia
architecture history, and today's own purge lessons all survive intact — only
the identifiers are gone.
Non-memory: the tracked .pre-pull-backup snapshot of a managed skill, whose
live counterpart was already clean.
132 lines
14 KiB
Markdown
132 lines
14 KiB
Markdown
# Consolidated long-term memory
|
||
|
||
## Operating context
|
||
- Memory root: memory://root.
|
||
- Primary repo: /home/dank/Projects/serverAppConfigs on the workstation, Dankclaw, and Dankserver contexts. Current repository state always wins over this memory.
|
||
- Primary hosts:
|
||
- Dankclaw: 192.168.0.143. Main Caddy edge host, Authentik/Pi-hole/Hermes, frank-default-large, FLM/Gemma NPU endpoint.
|
||
- Dankserver: 192.168.0.235. Media services and frank-default-small/media stack pieces.
|
||
- FrankBook-Pro: 192.168.0.112 / FrankBook-Pro.local, user dank, macOS x86_64 with Homebrew, Podman 5.4.2, applehv/vfkit machine.
|
||
- Workstation: the operation host.
|
||
- Passwordless SSH was verified among Dankclaw, Dankserver, FrankBook-Pro, and Hermes containers. Hermes containers use /root/.ssh/id_ed25519 for root/dank host-to-host checks.
|
||
- Common inventory commands:
|
||
- cd /home/dank/Projects/serverAppConfigs && find . -path ./.git -prune -o -name '*.container' -print | sort
|
||
- ssh HOST 'systemctl --user list-units --type=service; podman ps -a --format "{{.Names}} {{.Status}} nets=[{{.Networks}}]"'
|
||
|
||
## Deployment and container standards
|
||
- Long-running services should be Quadlet .container units, usually rootless user-systemd units started via systemctl --user, generating <name>.service. Some LLM/model services may be system units or custom services instead.
|
||
- Locked standardization decisions:
|
||
- Quadlet .container units for long-running services; compose is retired to reference where practical.
|
||
- Bind mounts only; avoid named volumes for durable service data.
|
||
- Linux service state roots under /home/dank/Containers/<svc> unless a service has a justified legacy root.
|
||
- Secrets stay out of tracked repo files; use machine-local env files.
|
||
- Verify user namespace settings after daemon reload with systemctl --user cat <svc>.service when relevant.
|
||
- Preferred Quadlet pattern: [Container] Image, ContainerName, EnvironmentFile, Volume bind mounts, Network/PublishPort as needed; [Service] Restart=always; [Install] WantedBy=default.target.
|
||
- Rootless Podman gotchas:
|
||
- Use UserNS=host in Quadlet, not compose-style UsernsMode=.
|
||
- For render/GPU group access, rootless containers may need keep-groups.
|
||
- Data-root mkdir/chown is handled by deployer in newer flows, but verify before assuming.
|
||
- Deployer: scripts/deploy.sh computes changes, pushes, target pulls, dispatches, wires configs, and runs health checks. It uses target-side FF-only pulls and state tracking. It can start services only if inactive for selected services to preserve sessions.
|
||
- Tooling caution: bash tool cwd must exist; /home/dank/Projects/serverAppConfigs is known valid. The read tool handles relative paths well.
|
||
|
||
## Network, DNS, TLS, and auth chain
|
||
- Domain: franksdank.cloud.
|
||
- Public DNS: Porkbun. Public A records point WAN clients to 71.227.227.172 when exposed.
|
||
- LAN DNS: Pi-hole on homelab, resolving the same internal service names to Dankclaw 192.168.0.143 via addn-hosts/local records. G36 router DHCP hands out Pi-hole to LAN clients.
|
||
- Removing an ordinary Porkbun A record makes a subdomain unavailable/NXDOMAIN externally while LAN Pi-hole resolution continues. This does not remove router port forwards or direct-IP exposure by itself.
|
||
- Keep public DNS for names needed externally, especially auth.franksdank.cloud and likely gitea.franksdank.cloud. Be cautious removing public DNS for auth because external login/debug flows have depended on it.
|
||
- Caddy runs as a single container on Dankclaw and fronts services on both Dankclaw and Dankserver by Host header.
|
||
- TLS uses real Let’s Encrypt wildcard DNS-01 certificates through a franksdank/caddy-porkbun image with Porkbun API credentials from dankclaw/.env, deployed to /home/dank/Containers/caddy/env. Never commit Porkbun credentials.
|
||
- Caddy admin must remain enabled on 127.0.0.1:2019 inside the container. Turning admin off disables the config file watcher and caused stale basicauth hashes to remain loaded.
|
||
- Auth chain for protected vhosts: browser -> Caddy TLS -> Anubis JS challenge -> internal Caddy/Auth routing -> Authentik outpost forward_auth -> app.
|
||
- Authentik outpost provider matching is by exact Host header. Curl without the right Host often produces misleading 404s.
|
||
- CLI verification limitation: unknown user agents like curl may receive Anubis’s HTML JS challenge with HTTP 200, not real app/outpost success. End-to-end auth must be verified in a real browser after solving Anubis.
|
||
- Apps with native auth generally bypass Authentik forward_auth: Grafana, Gitea, OpenWebUI, Jellyfin, *arr, Pi-hole. Protected via forward_auth include homepage, SearXNG, Hermes, Prometheus, RomM, Copyparty.
|
||
|
||
## Hermes on Dankclaw
|
||
- Hermes data persists via bind mount /home/dank/hermes-data:/data. HERMES_DATA_DIR and HERMES_HOME must point to /data.
|
||
- Hermes requires an entrypoint override equivalent to hermes gateway run --force -v to avoid default TUI crash-loop on exit.
|
||
- Hermes WebUI has used host networking. Dashboard runs separately on port 9119. Browser automation uses chrome-devtools-protocol with CDP bound to 127.0.0.1:9222.
|
||
- Live Hermes config is /home/dank/hermes-data/config.yaml inside host data. Repo file dankclaw/hermes/config.yaml.example is only a sanitized baseline; do not overwrite live secrets/state from it. Do not commit dashboard password hashes, phone numbers, OAuth credentials, API keys, sessions, memories, or generated state.
|
||
- Live config quirk: /data/config.yaml may be owned by an unmapped UID, so direct edits as dank may fail; use sudo-safe editing if needed.
|
||
- NPM npx cache for Hermes tools lives in container layer: first tool call in a fresh container may download; restarts keep cache, recreation resets it.
|
||
- Hermes FLM/Gemma rollout:
|
||
- Added provider dankclaw-npu at http://192.168.0.143:7771/v1.
|
||
- Model gemma4-it:e4b.
|
||
- auxiliary.vision routed to provider custom, model gemma4-it:e4b, base_url http://192.168.0.143:7771/v1.
|
||
- Main Hermes model remained frank-default-large via dankclaw-large at http://192.168.0.143:7770/v1.
|
||
- ~/.omp/agent/models.yml frank-default-large context length adjusted to 200000 for agent behavior.
|
||
- Recent Hermes/Signal diagnostic on Dankclaw:
|
||
- hermes, hermes-webui, and hermes-dashboard were active/generated.
|
||
- signal-cli-daemon.service was inactive/disabled.
|
||
- Hermes WebUI listened on 0.0.0.0:8787, dashboard/rootlessport on 9119, CDP on 127.0.0.1:9222.
|
||
- No listener on 8080, so Signal HTTP transport was likely unavailable despite healthy WebUI.
|
||
- If WebUI cannot reach Hermes/Signal, first check signal-cli-daemon.service with systemctl --user status and journalctl --user -u signal-cli-daemon.service; enable/start only if appropriate, then verify port 8080.
|
||
|
||
## frank-default-large current state and pitfalls
|
||
- Current durable configuration supersedes older agentionai/ciru/charlie-only experiments: frank-default-large on Dankclaw serves kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF via kingjones30/ROCmFPX/qwen4exp ROCmFPX runtime.
|
||
- Service alias: frank-default-large.
|
||
- Serve script and env: /home/dank/Projects/serverAppConfigs/scripts/serve-frank-large.sh and scripts/frank-default-large.env.
|
||
- Model path: /home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf.
|
||
- Runtime binary: /home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server, built for HIP/ROCm on Strix Halo gfx1151, ROCm 7.2.4. Stock llama.cpp and charlie12345/ROCmFPX alone cannot load this GGUF because ROCmFP4 tensor types and qwen4exp architecture support must be present in the same tree.
|
||
- Hardware target: Dankclaw Strix Halo gfx1151 unified memory. Env uses HSA_OVERRIDE_GFX_VERSION=11.5.1, HIP_VISIBLE_DEVICES=0, device ROCm0.
|
||
- Model details: Qwen3.8-Flash-Next base / qwen4exp MoE, 512 experts/10 used, 48 layers (12 attention + 36 SSM), QSA indexer, hyper-connections, PLE at layer 1. Single GGUF about 113.47 GiB, file_type 105, Q4_0_ROCMFP4_STRIX. PLE table about 50.66 GiB Q8_0.
|
||
- Critical pitfalls:
|
||
- PLE n-gram table must remain mmap/page-cache streamed and stay out of GPU memory. Do not add --no-mmap or tensor override/offload for the PLE table.
|
||
- --fit must remain off; auto-fit has segfaulted on qwen4exp hyper-connection config.
|
||
- Avoid bare environment variable names TEMP, HOST, and GPU_MAX_HEAP_SIZE; HIP/ROCm may read them. Pass llama.cpp host/port/temp as CLI flags.
|
||
- If graph build aborts on QSA F16 mask with flash attention, port scripts/patches/qwen4exp-qsa-f32-mask.patch into the active kingjones ROCmFPX tree and rebuild.
|
||
- Do not assume Qwen3.8-Flash-Next MTP heads are compatible by name. Prior MTP heads failed missing blk.0.hc_attn_norm.weight. Speculative decoding is currently disabled in latest durable config (SPEC_TYPE=none), despite older ngram-mod/MTP experiments.
|
||
- Latest serving defaults captured: host 0.0.0.0, port 7770, ctx 401408, parallel 2 slots, batch 4096, ubatch 512, threads 16, threads-batch 16, f16 KV, flash-attn on, split-mode none, gpu-layers 999, ctx-checkpoints 16, cache-ram 8192 MiB, prompt cache and idle-slot cache enabled, metrics enabled.
|
||
- Sampling defaults: temp 0.7, top_p 0.8, top_k 20, min_p 0.0, presence_penalty 1.5, reasoning on, reasoning-format deepseek, reasoning-budget -1.
|
||
- mmproj: STRIX repo ships no mmproj. Serve script conditionally passes /home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf if readable.
|
||
- Benchmark lessons: separate prefill/TTFT from decode tok/s. Prior stable prefill setting was batch/ubatch 4096/4096, but current durable env may differ; check repo. Decode sweep found THREAD_BATCH=16 best with 8K prompt and 256 gen tokens. Use 256 gen tokens for decode comparisons unless testing thermal/stability.
|
||
|
||
## Media and service state
|
||
- Plex homepage URL was updated to http://plex.franksdank.cloud/web/index.html in shared homepage configs. Deployed commit 8895a02 with 233/233 tests green.
|
||
- Jellyfin mount: /mnt/entropy:/data:ro. Health was 200 and libraries verified. DB path: /home/dank/Containers/jellyfin/config/data/jellyfin.db. Relevant table is BaseItems, not TypedBaseItems. User directive: stop touching Jellyfin/Copyparty after verification unless explicitly asked.
|
||
- Copyparty:
|
||
- Durable mount intent: /mnt/entropy:/w:ro or copyparty volume arg equivalent to browsable/read-only.
|
||
- Important permission fix: copyparty bare A means everyone full access, not anonymous-deny. Correct read-only behavior used -v /w::r:ra,dank, omitting w/m/d. Authentik-protected traffic reaches copyparty as unauthenticated, so anonymous must retain r for web reads.
|
||
- Do not preserve old compose secret in memory; rotate any exposed compose-interpolated copyparty password if host/container inspect access was possible.
|
||
- Earlier bare compose stack lacked Quadlet boot-persistence; newer standardization prefers Quadlet.
|
||
- Plex/Tautulli migration on Dankserver:
|
||
- Goal achieved as zero-bytes-moved Podman Quadlets.
|
||
- Plex image pin: lscr.io/linuxserver/plex:1.43.3.10896-cb3ebc72d-ls322.
|
||
- Tautulli image pin: lscr.io/linuxserver/tautulli:v2.18.1-ls242.
|
||
- Rootless UserNS=host. Plex PUID=0/PGID=0 because rootless Podman cannot map real uid 997 natively; /var/lib/plexmediaserver chowned to 1000:1000 for container operation. Rollback chowns back to 997:984.
|
||
- Tautulli PUID=1000/PGID=1000.
|
||
- Plex volumes: /var/lib/plexmediaserver:/config rw, /mnt/entropy:/mnt/entropy:ro.
|
||
- Tautulli volume: /home/dank/Containers/tautulli:/config.
|
||
- Network: dankserver_appnet with aliases plex and tautulli. Ports: Plex 32400, Tautulli 8181.
|
||
- Tautulli seed config uses PMS_Web_Host=plex, PMS_Web_Port=32400, PMS_Secure=0.
|
||
- Native Plex unit stopped/masked and Tautulli snap disabled; decommission packages only after explicit sign-off.
|
||
- Plex health gate widened to 600s due to large 32GB DB migrations; no HealthCmd to avoid cold-load flapping.
|
||
|
||
## *arr and FrankBook-Pro migration
|
||
- *arr stack (Sonarr, Radarr, Lidarr, Readarr, Prowlarr) migrates to FrankBook-Pro and is decommissioned from Dankserver.
|
||
- Keep /Volumes/COMPLETE and /Volumes/COMPLETE/Finished Torrents paths verbatim; rename deferred.
|
||
- VPN egress via gluetun container. qBittorrent shares gluetun netns or a related service netns as configured; ensure VPN egress gate passes before publishing ports.
|
||
- Podman machine init on Mac requires bind mounts for /Volumes/COMPLETE and /Volumes/INCOMPLETE. Adding mounts later may require machine re-init.
|
||
- Config dirs are archived to /home/dank/archive/<date>-arr-configs.tar.gz before removal; virgin installs can skip archive.
|
||
- Exposure model: private LAN-only direct IP:port and Pi-hole DNS; no Caddy vhost on Mac.
|
||
- Seerr target: Dankclaw, SQLite, Authentik OIDC SSO, provider created before route flip.
|
||
|
||
## Volume security and miscellaneous technical notes
|
||
- POSIX single-writer/multi-reader pattern: create dedicated writer user and readers group; set volume owner writer:readers and chmod 2775 so new files inherit group. Restrict SSH/SFTP via Match block or ForceCommand internal-sftp.
|
||
- FreeToken build on Dankclaw is build-only, no deploy/run. Environment has ROCm 6.3, NVIDIA driver 575.57.02, CMake, Python, git. Vulkan or ROCm builds are feasible.
|
||
- GGUF loader constraint in one context: restricted to Gemma4GGUFForCausalLM. Supports F32/F16/BF16 native views and Q4_0/Q6_K dequant. Q8_0 mapped but lacks dequant and raises NotImplementedError.
|
||
|
||
## ASUS TUF GAMING B650-E WIFI BIOS workflow
|
||
- Use the exact ASUS TUF GAMING B650-E WIFI support page; avoid wrong CAP files for similar boards such as B650M-E or B650-PLUS.
|
||
- Target latest memory-compat BIOS captured: 3886 beta (2026-07), AGESA 1.2.0.3a Patch A. Current referenced BIOS was 2613.
|
||
- Backup current BIOS before flashing:
|
||
- sudo pacman -S flashrom
|
||
- sudo flashrom --programmer internal -r bios-2613-backup-1.bin
|
||
- sudo flashrom --programmer internal -r bios-2613-backup-2.bin
|
||
- sha256sum bios-2613-backup-*.bin; hashes must match.
|
||
- Expected AM5 SPI dump size about 32 MB. Copy backups to USB and another machine.
|
||
- Never write BIOS with flashrom from Linux. Use UEFI/EZ Flash normal path. Use USB BIOS FlashBack only for no-POST rescue and exact renamed CAP from manual/support page.
|
||
- Dankserver hosts media stack and frank-default-small, so BIOS flash implies roughly 15–20 minutes downtime. First boot after AGESA update may black-screen 1–5 minutes for DDR5 retraining; wait.
|
||
- Post-flash: settings reset. Re-apply memory profile/manual settings, rerun memtest, verify BIOS version with BIOS screen or sudo dmidecode -s bios-version.
|