0 Fork 0

Files

T

Frank 872e86399a agent: repoint FLM-NPU model roles off the retired provider; sync memory/skills

config.yml: vision -> llamacpp (dankclaw)/frank-large:auto,
tiny -> llamacpp (dankserver)/frank-small:off,
task -> llamacpp (dankclaw)/frank-large:medium (was flm dankclaw-npu),
matching the flm (dankclaw-npu) provider removal in models.yml.

Memories/managed skills: rollouts and learned notes from the frank-large
KV-pool + sampler work and the noctalia recolor; merged the duplicate
noctalia-palette-recolor skill into noctalia-custom-palette-recolor (adds
the dotfiles-tracking step and the +show-config delta gotcha) and dropped
the superseded legacy memory skills whose ghostty USR1 reload recipe is
wrong for this fleet.

2026-09-13 23:51:43 -07:00

2.2 KiB

Raw Blame History

dankclaw Deployment & Model Services

frank-large (Quadlet Service)

Deployed as a rootless Quadlet service (frank-large.service) on dankclaw. Hosts ~68 GiB weights and a 524,288-position KV pool (~14.4 GiB). HALOGEN_KV_SLOTS reduced to 3, stabilizing total memory at ~103.4 GiB within the 121 GiB budget. Admission is pool-bound, capping at 3 simultaneous full-length requests.

Ports & Logging

Engine: :8730 (loopback), Front-end: :8731, Published: :7770
Logging: podman logs frank-large (full/filtered), journalctl --user -u frank-large
Env Vars: Avoid TEMP, HOST, GPU_MAX_HEAP_SIZE (HIP runtime reads bare TEMP).

Sampling & API Quirks

frank-large defaults to greedy; frank-small defaults to sampling (temp=1.0, top_k=20, top_p=0.95). Force small determinism via temperature: 0 or --sampling-defaults none.
preserve_thinking is per-request (client-side chat_template_kwargs), not controlled by HALOGEN_* env vars.
max_tokens: 200 with empty content indicates thinking consumed budget. frank-small returns 200 for unknown fields. File-cache shortage causes uninterruptible sleep, not OOM. host_reserve_gib=24 protects n-gram table.
Tests: 285/286 pass (test 73 pre-existing comfyui selector bug). pyyaml missing on system python3; uv available.

Marreta & Authentik Guard Chain

Deployed via podman compose, sharing gluetun egress with searxng. Caddy TLS edge routes through Anubis (:8923) → Authentik outpost → internal :8017 → marreta upstream.

Auth Flow: Provider-first creation → route flip (vhost reverse_proxy anubis:8923) → deploy.
Resolved Login-Wall: Null ProxyProvider.application and missing auth/authz/invalidation flows crashed the Rust outpost. Fix: populated flows, linked Application(slug=marreta), restarted services.
Pitfalls: Host probe (localhost:8017) returns HTTP 000 due to Caddy netns; probe internally or grep config. Non-browser clients fail JS challenges. Audit service-to-service traffic for backend URLs hitting guarded hostnames. IdP, Gitea (git smart-HTTP), and self-auth services are exempt.
Verification: E2E test (tests/e2e/specs/16-marreta.spec.ts) validates homepage tile routing.

2.2 KiB Raw Blame History