agent: repoint roles to frank-medium/dankclaw-3090; sync niri tweaks, memories, managed skills

- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh
- models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes
- niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0
- memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills
- managed-skills: 40+ new fleet runbook skills
- agent/skills: import built-in skill set
This commit is contained in:
Frank
2026-09-17 12:27:08 -07:00
parent 83c9c92b3f
commit 135bb2194d
103 changed files with 5333 additions and 170 deletions
@@ -1,22 +1,34 @@
# dankclaw Deployment & Model Services
## System & Hardware Topology
- **dankserver**: Dual AMD R9700 GPUs.
- *Issue*: RCCL fails on chipset-attached PCIe (PCIEX16_2) because lanes lack required atomic-op routing.
- *Fix*: Use a $30 M.2→PCIe x4 riser to move card #2 to CPU-attached M.2_2 slot.
- **dankclaw**: RTX 3090 (PCIe 4.0 x4 dock) + AMD Strix Halo (gfx1151, Vulkan).
- *Kernel*: Ubuntu 24.04, `amd_iommu=off`.
## frank-large (Quadlet Service)
Deployed as a rootless Quadlet service (`frank-large.service`) on dankclaw. Hosts ~68 GiB weights and a 524,288-position KV pool (~14.4 GiB). HALOGEN_KV_SLOTS reduced to 3, stabilizing total memory at ~103.4 GiB within the 121 GiB budget. Admission is pool-bound, capping at 3 simultaneous full-length requests.
## LLM & Software Stack
- **vLLM / ROCm**: Version 0.27.1 / torch 2.11.0 / ROCm 7.14.
- *Constraint*: HIP runtime reads bare `TEMP` and `HOST` env vars; naming these explicitly can cause collisions.
- *Constraint*: `GGML_CUDA_FA_ALL_QUANTS` required for flash-attention with quantized KV.
- **llama.cpp (ik_llama.cpp)**:
- *Model*: Qwen3.8-Flash-Next-Uncensored-i1-GGUF (i1-Q4_K_M).
- *Performance*: ~3.8-4.0 t/s decode (bottlenecked by PCIe x4 sync/graph splits), ~192 t/s prefill.
- *Note*: 30 t/s decode is unreachable via current PCIe x4/split-mode setup; requires mainline PR #28901 for capture-safe graphs.
### Ports & Logging
- Engine: :8730 (loopback), Front-end: :8731, Published: :7770
- Logging: `podman logs frank-large` (full/filtered), `journalctl --user -u frank-large`
- Env Vars: Avoid `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE` (HIP runtime reads bare `TEMP`).
## Service Management (Quadlet & Podman)
- **Persistence**: `restart: unless-stopped` survives crashes, not explicit stops. Every container must have a Quadlet `.container` file for boot-persistence.
- **frank-large (HALOGEN)**: Rootless Quadlet. Weights pinned (~68 GiB). KV pool: 524288 positions (~14.4 GiB).
- *Stall Behavior*: Multi-minute full-CPU stalls (no disk/output) are documented behavior under co-tenancy with `HALOGEN_FLASH_PIN_TRUNK=1`. Do not restart.
- *First-touch*: 45-60s silence after restart as engine first-touches fresh KV pool.
- *Constraints*: `maxTokens` <= 65536. Prompt + max_tokens must fit KV pool.
- **frank-small**: Defaults to sampling (temp 1.0, top_k 20, top_p 0.95).
### Sampling & API Quirks
- `frank-large` defaults to greedy; `frank-small` defaults to sampling (temp=1.0, top_k=20, top_p=0.95). Force small determinism via `temperature: 0` or `--sampling-defaults none`.
- `preserve_thinking` is per-request (client-side `chat_template_kwargs`), not controlled by HALOGEN_* env vars.
- `max_tokens: 200` with empty content indicates thinking consumed budget. frank-small returns 200 for unknown fields. File-cache shortage causes uninterruptible sleep, not OOM. `host_reserve_gib=24` protects n-gram table.
- Tests: 285/286 pass (test 73 pre-existing comfyui selector bug). `pyyaml` missing on system python3; `uv` available.
## Networking & Auth
- **Marreta Auth Chain**: Caddy TLS → Anubis (:8923) → Authentik Outpost → Marreta (:8017).
- *Fix*: Populated `invalidation_flow`, `authentication_flow`, and `authorization_flow` to prevent Rust outpost serde crashes.
- *Probing*: Host-to-listener returns HTTP 000 (Caddy netns); probe inside netns or grep config for internal status.
## Marreta & Authentik Guard Chain
Deployed via podman compose, sharing gluetun egress with searxng. Caddy TLS edge routes through Anubis (:8923) → Authentik outpost → internal :8017 → marreta upstream.
- Auth Flow: Provider-first creation → route flip (vhost `reverse_proxy anubis:8923`) → deploy.
- Resolved Login-Wall: Null `ProxyProvider.application` and missing auth/authz/invalidation flows crashed the Rust outpost. Fix: populated flows, linked `Application(slug=marreta)`, restarted services.
- Pitfalls: Host probe (`localhost:8017`) returns HTTP 000 due to Caddy netns; probe internally or grep config. Non-browser clients fail JS challenges. Audit service-to-service traffic for backend URLs hitting guarded hostnames. IdP, Gitea (git smart-HTTP), and self-auth services are exempt.
- Verification: E2E test (`tests/e2e/specs/16-marreta.spec.ts`) validates homepage tile routing.
## Operational Lessons
- **Memory**: `free` command lies about available memory (doesn't account for pinned/reserved weights).
- **Skills Discovery**: `omp` scans `~/.omp/agent/skills/`.
- **Root-Cause**: PCIe x4 dock provides ~29 GB/s, not 130 GB/s.
- **Logs**: Use `podman logs` or `journalctl --user -u` for rootless Quadlets.