Commit Graph
55 Commits
Author SHA1 Message Date
Frank 8fefe6e1a9 feat(frank-tts): deploy wiring for the 7772 voice endpoint, and voice naming written down
Stage 2 of the voice stack. Repo state only: nothing deployed, no host touched,
no container started.

- `frank-tts` is now registered everywhere a dankclaw service has to be:
  `SERVICE_ORDER` (after `frank-tts-engine`, before the consumers), the planner's
  dankclaw set, `deploy-remote.sh`'s allowlist + dispatch + a
  `deploy_frank_tts` handler, and `deploy.sh`'s dispatch regex — plus the
  `build_compose_image` arm naming `dankclaw/voiceservice/Containerfile`, without
  which the build silently returns 0 and the unit's `Pull=never` tag never
  exists.
- The unit joins TWO networks, in order: `dankclaw_default` first (default route,
  where consumers and open-notebook live), then the voice bridge. `TTS_UPSTREAM`
  moves from `host.containers.internal:8880` to `http://frank-tts-engine:8880` —
  the actual fix for a shape that could never work, since a bridge container
  cannot reach a host-loopback bind and the engine publishes nothing at all.
- `Wants=`/`After=` on the engine, never `Requires=`: the `llama-tts-cli` rollback
  backend needs no upstream, so a broken engine may delay this endpoint but must
  not take its unit down.
- The handler installs the engine's quadlet pair too, because a referenced
  `Network=` file the generator cannot find means no façade unit is produced at
  all, and `--only frank-tts` has no other way to get it. It creates
  `/home/dank/voices` as the deploy user (rootless podman would auto-create a
  missing bind source root-owned, which reads as "no voices"), and gates on
  podman's recorded health.
- Voice naming is now written down where it can be found: a voice is a lowercase
  directory name matching `^[a-z0-9][a-z0-9._-]*$`, at most 64 chars. That shape
  is the intersection of two rules that disagree — the shim would accept mixed
  case, the engine lowercases names — so a mixed-case pack resolves on one side
  only, and the boot warmer refuses it loudly. `narrator1/2/3` are load-bearing
  for open-notebook.
- Homepage card for the pair, `wip: down` and hidden, probing `127.0.0.1:7772`;
  the engine gets no card because it publishes nothing and could never be probed.
- **No Prometheus job, deliberately.** No `/metrics` exists on the shim, this
  Prometheus 3 refuses HTML/JSON bodies (the existing `comfyui` and `services`
  probes are documented down for exactly that), and `LlmEndpointDown` is scoped
  `up{job=~"frank-.*"}` — so a `/health` job would page on Signal for a healthy
  endpoint. Recorded in the page instead of quietly skipped.
- Vault: new [[Frank TTS]] page linked from the index; [[Frank TTS Engine]] now
  points at it for the pair's public signal; log entry for the day.
- Seam grows to 25 tests: the generator's rendering of the pair (both
  attachments, first-network order, one publish, Wants-not-Requires, bounded
  per-unit blocks), the env file's upstream checked as a network JOIN rather than
  a string, planner separation and order, the handler's steps, the card's gating,
  routing that selects both seams for either deployable.
2026-10-18 17:43:08 -07:00
Frank 8d76854a1e feat(frank-tts-engine): the arm A engine as a boot-persistent quadlet unit
Repo state only: nothing deployed, no host touched, no container started.

- `frank-tts-engine` on Dankclaw: `qwentts.cpp` commit
  5916bf7c79dbb4e66fc059cdc507a79a23a28bb4, resident `tts-server`, 1.7 B base
  talker Q4_K_M + Q8_0 tokenizer, container port 8880. It is deliberately NOT a
  fleet endpoint — `frank-tts` on 7772 is the endpoint consumers address.
- Publishes NOTHING. A host-loopback publish would have been useless to the
  façade (netavark DNATs published ports only, so a bridge container cannot
  reach a host loopback bind — the shape that 502'd hermes) and would have been
  the one LAN-reachable surface of an unauthenticated synthesiser. Both voice
  containers sit on one committed quadlet network, `frank-tts-engine.network`
  pinned to 10.89.5.0/24 (next free /24 after open-notebook's 10.89.4.0/24), and
  peers address the engine by container name. Diagnostics and the deployer's
  health gate run inside the netns (`podman exec`, `wait_container_healthy`).
- Cold-voice law honoured in the unit: the first request for a registered voice
  decodes its whole reference to snapshot codec state (6.2-8.0 s, first audio
  byte 5 530 ms), later ones restore in 0.5 ms, and the registry is memory-only.
  `ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
  with `TimeoutStartSec=900`; the `-` prefix keeps a failed warm from
  crash-looping the endpoint under `Restart=always`. It logs cold=/warm= per
  voice, so the under-a-second second request is proven on every boot.
- Thread law recorded: `tts-server` has no `--threads`; src/backend.h uses
  hardware_concurrency()/2, so a cpuset pin silently changes the thread count.
- Image builds from the pinned commit with the proven line (GGML_BLAS=OFF
  forced by this host, every GPU backend off, -j 8, no -DGGML_NATIVE because
  ggml defaults to -march=native for the building host). Two stages; the runtime
  stage COPYs both the entrypoint and the warmer, because a container that
  publishes nothing is reached with `podman exec`, not a port.
- `scripts/download-frank-tts-engine-qwentts.sh` fetches the two GGUFs with byte
  and LFS-oid checks (talker 1 219 245 248 B, tokenizer 291 150 624 B); the
  deployer gates on --verify-only instead of pulling 1.4 GiB mid-deploy.
- Registered in SERVICE_ORDER, DANKCLAW_SERVICES, the deploy-remote.sh dispatch,
  its pre-deploy requirement, deploy.sh's regex, the planner's path map (the
  whole `dankclaw/voiceservice/` tree mapped to nothing before this), and
  test-changed.sh ABOVE the `dankclaw/*.container` case whose `*` crosses `/`.
- New seam tests/frank_tts_engine.bats (19): the Quadlet generator's own output
  for network/publish/boot/warm shape, entrypoint argv driven from the tracked
  env file, the warmer against a stub tts-server over a real socket
  (registration keys, ref_text absent, late engine, hung synthesis, name
  refusal), the fetcher's verifier, the pinned cmake line, and both routing
  tables. Suite: 422 passed.
- Vault: [[Frank TTS Engine]] linked from the index; log.md entry for the day.
2026-10-18 17:26:50 -07:00
Frank 8816a9d498 feat(deploy): wire open-notebook through both deployers, gated on secrets
Registration is load-bearing, not cosmetic: deploy-plan.py refuses any service
it does not know, so SERVICE_ORDER + DANKCLAW_SERVICES are what let `--only` and
`--auto` address the pair at all; the dispatch case in deploy-remote.sh runs it;
the requirement in check_prerequisites gates a manual deploy; and the dispatch
regex in deploy.sh's host-mode branch decides whether a changed path dispatches
a service or falls through to "no service mapped" — an omission there is silent,
which is exactly how the AGENTS.md deployer-registration law was earned.

The deploy seeds /home/dank/Containers/open-notebook/env (0600) from the host
env and refuses to start the pair while OPEN_NOTEBOOK_ENCRYPTION_KEY or
OPEN_NOTEBOOK_PASSWORD is blank: without the key no provider credential saves
and without the password the API — including the episode-audio route — is open
to the LAN with no auth. A seeded file is never rewritten, so re-deploying
cannot clobber a rotated key, which would silently make every stored credential
undecryptable. SURREAL_PASS is derived from SURREAL_PASSWORD in the same write.

The gate names paths that answer, not invented ones: the database is judged by
its own container health status (/surreal is-ready, verified against the pinned
digest) and the app by wait_for_health on /health at the published 5056 mapping —
the route is upstream's and is on the auth middleware's exclude list, so it
stays 200 after the password is set. 180 s covers Next.js build + startup and a
degraded-but-listening first boot, matching the sibling LLM deploys.

Both digest pins are checked against local storage before restart, because
Pull=never means a missing image would otherwise surface as a unit stuck in
"Start request queued" (the 203/EXEC trap's sibling). The interim episode
directory is created before the app starts: a missing bind source makes podman
create it as root-owned and the app's mkdir would then fail.
2026-10-18 16:26:52 -07:00
Frank 55f87fcdce llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot
The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
2026-10-10 17:03:14 -07:00
Frank 87721ab0fe rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:

- Service trio: scripts/{frank-small.env,frank-small.service,
  serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
  frank-secondary-*.md (9 files); the FreeToken runbook stays
  freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
  LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
  3090, and the Dankserver service that became frank-medium) are
  preserved verbatim — they describe what the service was called
  at the time.
2026-10-06 15:37:13 -07:00
Frank 9c508218e9 e2e: scope the FrankBook-Pro leg to its own trigger-gated Playwright project
The 11-admin spec walked links that terminate on FrankBook-Pro
(192.168.0.112), which sleeps when idle — every deploy gate timed out
there (measured 2026-10-05, two deploy windows). Split: the frankbook
links move to tests/e2e/specs/19-frankbook.spec.ts under a frankbook
project that run.sh excludes by default and deploy.sh triggers ONLY
when a frankbook blast-radius path changed (the spec, frankbook-pro/,
the homepage registry, the Caddyfile arr vhosts). deploy.sh computes
the trigger from the same BASE..HEAD revision range the plan uses.
2026-10-05 18:04:02 -07:00
Frank d7bf9f119a retire frank-medium: tests, planning and observability ground on frank-primary
The frank-medium retirement tail the 2026-09-26 lucebox cutover left
uncommitted: the radiance quadlet tree, its swap/fetch tooling and its
seam test are deleted (archive copies live in dankserver/.AGENTTRASH/
20260926-frank-medium-radiance/ and scripts/.AGENTTRASH/20260926-
frank-medium-radiance/), the planner/allowlist/test-changed mappings
drop the service, and frank-large's chat template moves to scripts/
(its old home was the retired frank-medium dir).

Finished here:
- tests/frank_primary_unit.bats: the lucebox quadlet seam (boot
  persistence, host-network bind, HIP_VISIBLE_DEVICES=0,1, HIP-reserved
  env-name guard, checkout-mounted entrypoint/healthcheck, first-start
  patience, deployer payload/port gates).
- test-changed.sh: explicit .AGENTTRASH/ case — retirement commits move
  files to trash, and the wrapper refused those additions with "no
  focused-test mapping".
- observability: the vllm: namespace left with the radiance runtime.
  Rules keep only llamacpp:* series (prefix-cache reuse, spec-decode,
  queueing, endpoint-down); the dashboard drops the vLLM KV/latency
  panels; prometheus.yml documents that frank-primary (lucebox) has no
  /metrics and therefore no scrape job. llm-fleet.json was left
  corrupted mid-edit (missing opening brace) and is rewritten from HEAD.

Suite: 339 green (was 7 red).
2026-09-26 19:25:46 -07:00
Frank bed551baa6 deploy.sh: allowlist dankserver:frank-primary 2026-09-26 15:33:05 -07:00
Frank cb96525f78 fix(deploy): add calibre-server to deploy.sh's service allowlist 2026-09-21 12:47:04 -07:00
Frank 590853c133 alerting: Alertmanager + Signal relay deliver firing rules to Frank's phone
- dankclaw/observability/quadlet/alertmanager.container: loopback :9093,
  config from shared/prometheus/alertmanager.yml, family conventions
- shared/prometheus/alertmanager.yml: single signal receiver, group_by
  alertname, 30s/5m/4h intervals, webhook to host.containers.internal:18093
- scripts/signal-relay.py + .service: host-side user unit, loopback :18093,
  15-min dedupe, sends via signal-cli account 318311
- prometheus.yml: alerting block to alertmanager:9093; stale comments fixed
- llm-fleet.yml: delivery header; LlmSpecDecodeOff annotation now drafter-agnostic
- deploy.sh/deploy-remote.sh/deploy-plan.py: alertmanager service, config
  copy, relay install, planner routing for config + relay files
- homepage: alertmanager status card (loopback probe)
- tests: alertmanager in UNITS/loopback/health/PAIRS/planner + five new
  delivery-path seams (webhook<->relay contract, prometheus target, relay
  unit, signal-cli account, planner routing)
- docs: Alertmanager.md component page, Observability.md, Prometheus.md,
  index.md, AGENTS.md delivery line
2026-09-17 13:09:59 -07:00
Frank 43ec77fee6 inference: frank-large returns to halogen, frank-small takes the 3090
Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.

frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.

The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.

Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.
2026-09-16 13:58:00 -07:00
Frank 6085a99dd8 frank-medium: serve the new R9700s, retire frank-small
The RTX 3090 left Dankserver; two AMD Radeon AI PRO R9700 cards (Navi 48,
gfx1201, 32,624 MiB each) replaced it. FreeToken/CUDA cannot run there at all,
so the `frank-small` service is retired and `frank-medium` takes the endpoint.

New service: charlie12345/ROCmFPX llama.cpp (built for gfx1201 against the
fork's self-contained ROCm 7.14 toolchain, so the host's stale /opt/rocm-6.3.3
never participates) serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF --
a Q8_0_ROCMFPX_AGENT (ftype 115) checkpoint that stock llama.cpp cannot load.
Placeholder config: --split-mode tensor across both cards, 131072 context,
1 slot, f16 KV, MTP draft head on ROCm0.

deploy-remote.sh grows retire_root_llm_small(): the CUDA-era unit is stopped,
disabled and moved to ~/.AGENTTRASH/20260915-frank-small-retire/host-units
before frank-medium starts, because `systemctl disable` has been observed to
delete the unit file itself on this host (2026-08-26 incident).

The retired frank-small repo files (env, unit, serve script) move to
scripts/.AGENTTRASH/, and the planner, dispatcher, service allow-list and the
deployment test seams now name frank-medium.
2026-09-15 18:53:20 -07:00
Frank 09c54113b1 deploy: make Playwright e2e a mandatory post-deploy gate
The 2026-09-12 homepage outage slipped through because the live-fleet
Playwright suite was a manual 'MUST invoke' that deploys skipped. Wire it
into deploy.sh as the [3/3] gate: after a behavior-affecting deploy,
run tests/e2e/run.sh against the live fleet; a failure exits DEPLOY
INCOMPLETE. Add --skip-e2e as an emergency escape hatch. Update the
AGENTS.md deployment-completeness contract and flow diagram to match.

Verified: bash -n clean, --skip-e2e parsed, unknown flags rejected,
seam suite green (only the pre-existing comfyui test 73 fails).
2026-09-12 11:50:40 -07:00
Frank 91e9410c5e deploy.sh: allowlist dankclaw:marreta in the top-level service gate
The scoped runner (deploy-remote.sh) already knew marreta; the top-level
deploy.sh valid_service() gate did not, so --service marreta was rejected
before reaching the planner.
2026-09-12 10:10:01 -07:00
Frank 69f87c8483 feat(llm): rename inference services dropping 'default' from names
frank-default-large -> frank-large (Dankclaw :7770), frank-default-small ->
frank-small (Dankserver :7770), frank-default-sightsound -> frank-npu
(Dankclaw :7771). Units, env files, serve scripts, deploy-plan mappings,
deploy-remote dispatch (stop/disable old units + relink + enable new), and the
test seams move together. Retired frank-default-npu.* artifacts moved to
scripts/.AGENTTRASH/ (tracked-in-history; ignored on disk).

Served aliases become frank-large / frank-small via --alias; the NPU endpoint
serves the whole FLM registry, so its rename changes no served tags.
2026-09-03 16:15:43 -07:00
Frank 93f16f68be deployer: route plex and tautulli as dankserver quadlet services
- deploy_plex(): install unit, stop+mask native plexmediaserver.service on
  first cutover (mask, never disable), gate on /identity for 180s
- deploy_tautulli(): install unit, stop+disable snap on first cutover,
  seed config.ini with PMS_Web_Host=plex only if absent
- allow-lists (deploy.sh valid_service, deploy-remote.sh service_is_allowed),
  planner routing (SERVICE_ORDER, DANKSERVER_SERVICES, quadlet_services),
  test-changed cutover case, quadlet seam enumeration + new
  userns/PUID=997/PGID=984 ownership-contract test
2026-09-01 22:31:14 -07:00
Frank 7457a9c607 deploy.sh: accept dankclaw:homepage as a valid explicit service (drift vs deploy-plan.py) 2026-09-01 16:43:06 -07:00
Frank 4ad85fae70 authentik: add LDAP outpost service for Jellyfin LDAP-Auth
- New authentik-outpost-ldap quadlet: ghcr.io/goauthentik/ldap:2026.8.0,
  publishes host 389/636 -> container 3389/6636 for cross-host binds from
  dankserver's Jellyfin LDAP-Auth plugin; outpost token in host .env.
- deploy-plan.py: register authentik-outpost-ldap in SERVICE_ORDER and
  DANKCLAW_SERVICES; map dankclaw/authentik/quadlet/ unit path.
- deploy-remote.sh: deploy_authentik_outpost_ldap handler + dispatch case.
- deploy.sh: valid-service entry.
2026-09-01 15:25:56 -07:00
Frank f4c9b4f381 seerr: quadlet on dankclaw + guard-chain vhost + deployer wiring
Unit: ghcr.io/seerr-team/seerr:v3.4.1, loopback-only 5055 publish
(homepage probe; users go through Anubis->Authentik->caddy:8017->seerr),
Containers data root, no env secrets. Guard chain: seerr.franksdank.cloud
vhost behind anubis:8923 with post-auth handler on 8017; addn-hosts record;
homepage widget; deploy-remote/deploy-plan/deploy.sh routing; seam test.
2026-09-01 14:40:10 -07:00
Frank 949f68e2b5 frankbook-pro: *arr stack + qbittorrent quadlets inside podman machine
Seven units installed INSIDE the Mac VM (/home/core/.config/containers/
systemd) by a new deploy_frankbook_stack handler: gluetun (ProtonVPN
WireGuard, env-file credentials, publishes qbit's 8080/6881), qbittorrent
(Network=gluetun.container, verbatim /Volumes data roots, first-boot
config seed with fresh machine-local WebUI password and the Finished
Torrents category), and prowlarr/sonarr/radarr/lidarr/readarr (grab-only,
config migrated from dankserver at provisioning, readarr pinned to
amd64-1.0.980). deploy.sh routes the new target through the ssh-config
alias with REMOTE_SHELL=/usr/local/bin/bash (macOS bash 3.2 lacks
associative arrays); the planner learns FRANKBOOK_SERVICES and the
gluetun->qbittorrent netns pairing; new seam test pins the contract.
2026-09-01 14:10:23 -07:00
Frank b206f477ad dankserver: quadlet units + deploy handlers for romm trio, jellyfin, calibre-web, copyparty
Cutover of the six surviving dankserver services from compose-era bare
containers to boot-persistent Quadlet units (WantedBy=default.target; podman
5.x v1 generator honors it at user-manager start).

Units are faithful to live inspect ground truth (image pins = running tags,
not compose inventory):
- romm trio shares one data root: /home/dank/Containers/romm mounted whole at
  /romm; mariadb datadir and redis data dir live inside it. Aliases mariadb/
  redis preserved on dankserver_appnet via Network=...:alias=. Secrets stay in
  the host-local env file (EnvironmentFile + ${VAR} expansion in the generated
  ExecStart, gitea precedent). copyparty passes its admin account via
  Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD} (Exec= not Command=: the v1
  generator hard-fails on Command=); entropy stays rw only there.
- jellyfin keeps every creation-time env as Environment= lines (zero drift);
  Device=/dev/dri + GroupAdd=keep-groups preserve hardware transcoding.
- calibre-web pins the exact running image tag (id 986277563e76 =
  0.6.27-ls397), pasta network preserved.

Deployer: first-cutover handlers retire the old container, rescue unshadowed
anonymous-volume content (podman volume export + tar --skip-old-files), mv
data roots on the same fs, then start units in dependency order with health
gates. Re-deploys restart only the selected unit. deploy-plan.py maps compose
keys mariadb/redis to the new unit names; allowlists updated. New seam test
tests/dankserver_quadlet.bats pins boot-persistence + v1-generator contract;
test-changed.sh maps the new dirs to it. Full suite green (209 tests).
2026-09-01 04:14:08 -07:00
Frank c3ed3a6e12 sightsound: new FLM NPU sight+sound service on XDNA2 (gemma4-it:e4b)
New Dankclaw service frank-default-sightsound: FLM 1.0.1 serving the
Gemma4-E4B-IT-NPU2 pack (vision+audio+reasoning, 131072 ctx) on the XDNA2
NPU at port 52626. Model choice rationale: gemma4-it is the only NPU2 model
family with BOTH image and audio towers; the uncensored candidate qwythos:9b
is text-only, so sight+sound wins over uncensored here.

Deployer wiring: planner maps env/unit/serve files to the new service name;
deploy-remote gains deploy_llm_sightsound (relink + daemon-reload + enable +
restart + health gate on /v1/version — FLM has no /health route). Unlike the
retired frank-default-npu relink-only path, this unit is boot-enabled.
2026-08-30 04:37:42 -07:00
Frank 999cecda7b fix(deploy): allow caddy in the explicit-service gate
deploy-remote.sh already implements dankclaw:caddy (render + reload);
the top-level gate rejected it before dispatch could reach the handler.
2026-08-28 22:30:44 -07:00
Frank 3a16375cfe deploy: accept authentik-outpost in the top-level service gate 2026-08-28 19:10:35 -07:00
Frank 0ae3650414 anubis: bot-challenge proxy in front of the guarded vhosts (two-tier chain)
Bots and AI scrapers hit the SSO-fronted vhosts and eat Authentik auth
work for 403s. Anubis verifies clients first: DENY for known AI scraper
UAs, JS challenge for everyone else; verified traffic resumes the normal
Authentik forward_auth chain on Caddy's internal listener (:8016).

Chain: client -> Caddy (TLS) -> Anubis :8923 (verify) -> Caddy :8016
(auth + routing resumes; Host survives both hops) -> app. Reverse-proxy
through Anubis, not forward_auth: at the pinned tag a CHALLENGE answers
HTTP 200 + HTML, and forward_auth treats 200 as "authenticated" — the
page would pass through unchallenged. Exempt by construction (never
routed through Anubis): plex (plain HTTP legacy), auth.* (IdP login
must be reachable ungated), gitea (git smart-HTTP clients cannot solve
JS challenges), and every app with its own login.

Pinned ghcr.io/TecharoHQ/anubis:v1.13.0 — the tag was source-verified
before pinning: at this tag the policy parser is JSON-only with a
minimal bots[] schema (no imports, no CEL expressions), flags are the
config surface via flagenv (TARGET/POLICY_FNAME map to --target/
--policy-fname), and the image's exec-form HEALTHCHECK is inherited by
the quadlet unit (dodges the v1 generator's HealthCmd space-split quirk).

Rollback nuance: stopping anubis 502s everything behind the chain (the
public listeners proxy through it); a true rollback is reverting the
Caddyfile and redeploying, not just systemctl --user stop anubis.
2026-08-28 06:38:24 -07:00
Frank 105a4cdfef deploy.sh: accept hermes-webui in the top-level service inventory 2026-08-26 08:22:23 -07:00
Frank d52bcc15f8 Migrate Gitea proxy to canonical Caddy 2026-08-25 21:56:04 -07:00
Frank bc718f04f4 Fix scoped deployment state and config ownership 2026-08-25 20:24:47 -07:00
Frank 1ba47e4d22 Deploy services independently by changed paths 2026-08-25 20:13:33 -07:00
Frank 835996a67b Keep deployment moving when Caddy restart is blocked 2026-08-25 16:55:10 -07:00
Frank eae5bc2565 Continue LLM deployment when compose restart fails 2026-08-25 16:54:18 -07:00
Frank c7a72b465e Fix privileged shared config deployment 2026-08-25 16:53:13 -07:00
Frank 3cc99d81cc Authentik SSO + homepage naked domain
Auth layer (Authentik 2026.8.0):
- compose: 3-service stack (authentik-postgres, authentik-server, authentik-worker),
  postgres health-gated, bind mounts under /home/dank/authentik
- Caddyfile: auth.franksdank.cloud vhost; forward_auth on the no-auth vhosts
  (homepage, SearXNG, Hermes, Prometheus, RomM, Copyparty); outpost path
  proxied first in a route block
- .env.example: AUTHENTIK_PG_PASS + AUTHENTIK_SECRET_KEY
- deploy.sh: render only CADDY_BASICAUTH_HASH (the homepage per-user hashes
  are retired; grafana + pihole keep basicauth)
- tests/authentik.bats (new) + updated homepage/caddyfile/pihole seams
- docs: Authentik.md + per-service pages; Caddy.md + Homepage.md updated

Homepage (naked domain):
- Caddyfile: dankclaw.franksdank.cloud -> franksdank.cloud www.franksdank.cloud;
  dankclaw + dankserver redirect to it; bare-IP catch-all updated
- addn-hosts: A records for the naked domain, www, and auth (all -> 143)
- tests/pihole.bats: vhost_names handles multi-address blocks; TLD check
  allows the naked domain

All 146 tests pass.
2026-08-24 23:48:27 -07:00
Frank c58dab0a89 Homepage: give the user page its own login (dankuser)
The homepage user page + shared data now use a dedicated CADDY_HOMEPAGE_HASH
(username dankuser), separate from the shared family password (CADDY_BASICAUTH_HASH,
username dank, still used by the other vhosts) and the operator password
(CADDY_ADMIN_HASH, username frank, the /admin/ page). This lets the homepage be
re-logged independently without touching the other sites. deploy.sh renders the
new placeholder; the .env.example documents all three hashes.
2026-08-23 21:08:21 -07:00
Frank b485a27784 deploy: run the /srv/homepage steps under sudo
The homepage deploy's mkdir/cp/write of /srv/homepage ran as the deploy user
and hit 'Permission denied' (/srv is root-owned). Prefix them with the sudo
wrapper, write deployed.js via a temp file + sudo cp, and chmod the tree
a+rX so Caddy can serve it. Run the one-off checker as root too (it writes
status.json into /srv/homepage).
2026-08-23 15:44:46 -07:00
Frank f400be76c6 deploy: make the Caddy hash grep defensive under set -e
A missing CADDY_*_HASH in the .env made the 'grep | tail | cut' pipeline
return non-zero, which aborted the deploy under 'set -e' before it could warn
and keep the placeholder. Add '|| true' so an absent hash warns instead of
aborting. (The homepage deploy aborted here because CADDY_ADMIN_HASH was not
in the repo-root .env the deploy reads.)
2026-08-23 15:25:53 -07:00
Frank 2e6c44f3ba Homepage (TDD slice 6): the git-driven deploy
Red tests first.
- tests/homepage_deploy.bats: the deploy ships the homepage (assets to
  /srv/homepage, the checker timer, both Caddy hashes, deployed.js) and is
  git-driven (pulls main); the checker runs as a systemd timer.
- scripts/homepage-status.service / .timer: the checker as a oneshot service
  + a 3-minute timer (Q2/Q35).
- scripts/deploy.sh: the Caddyfile render now substitutes BOTH hashes
  (REPLACE_WITH_HASH <- CADDY_BASICAUTH_HASH, REPLACE_WITH_ADMIN_HASH <-
  CADDY_ADMIN_HASH). A new homepage step copies the assets to /srv/homepage,
  generates deployed.js from the deployed commit's datetime (Q43), installs
  the checker timer, and runs the checker once so status.json exists at once.
- tests/llm_units.bats: scoped the three LLM-unit tests to
  frank-default-*.service (the homepage unit is a different pattern: it
  installs to /usr/local/bin and runs from /srv at deploy time).
2026-08-23 12:59:56 -07:00
Frank 6a9edd4ee0 deploy.sh: keep the relink comment free of a literal loop-var token
The explanatory comment contained an unescaped loop-variable token, which
the local shell (set -u) tried to expand while building the double-quoted
remote script ('unit: unbound variable'). Reword the comment.
2026-08-22 22:36:04 -07:00
Frank 33438f223e deploy.sh: fix LLM unit relink (unquote source path so ${unit} expands on the remote)
The relink loop wrapped the source path in single quotes, which shipped a
literal ${unit} to the remote and created a dangling symlink
(/etc/systemd/system/frank-default-large.service -> .../${unit}.service),
so 'systemctl enable' failed with 'unit does not exist'. The path has no
spaces; unquoting lets ${unit} expand on the remote for both source and
destination.
2026-08-22 22:33:16 -07:00
Frank fbfbc5c6db Move frank-default-large from :52625 to :7770 (dankclaw)
Operator request (2026-08-22): the primary inference server
(frank-default-large, the 27B CIRU model on Dankclaw) moves to the
standard llama.cpp port 7770. Both LLM services now listen on :7770,
distinguished by host (dankserver vs dankclaw).

Functional changes:
- scripts/frank-default-large.env: LISTEN_PORT 52625 -> 7770
- scripts/serve-frank-large.sh: default LISTEN_PORT 52625 -> 7770
- scripts/deploy.sh: LLM_HEALTH[dankclaw] -> :7770
- open-webui quadlet: OPENAI_API_BASE_URL -> 192.168.0.143:7770
  (re-pointed so chat keeps working; data dir preserved)
- scripts/bench-fast-greedy.sh: PORT 52625 -> 7770

Tests updated to the new topology (both LLM endpoints on :7770 by host;
open-webui -> :7770). Docs + pi KB updated; dated historical runbooks
keep their point-in-time ports. models.json (pi client) re-pointed to
:7770 separately, effective on the next client restart.
2026-08-22 22:11:53 -07:00
Frank 0461988884 scripts: unified deploy.sh; units + bench point at the merged checkout
deploy.sh now deploys both workloads per target in one run:
push -> pull on /home/dank/serverAppConfigs -> podman stack steps
(dankclaw) -> LLM unit relink + daemon-reload + service restart ->
homepage health + LLM health (up to 20 min model-load wait).
Sudo: plain sudo on the fleet hosts (passwordless verified
2026-08-22); FRANK_SUDO_PW covers headless runs elsewhere. The
vllmScripts deploy.sh is superseded.

Units: frank-default-{small,large,npu}.service now reference
/home/dank/serverAppConfigs/scripts/... (the merged checkout).
bench-spec-stack.sh: env file path updated to the merged checkout.
2026-08-22 15:50:39 -07:00
Frank 0951fc165c deploy.sh: restart caddy when the reload cannot be applied
A 'caddy reload' delivered over the admin API cannot apply a config
that disables the admin API -- the fix cannot ride the API it is meant
to enable (2026-08-22: admin off sat in the running config across
deploys). Track the reload result; on failure, restart the container
so it reads the rendered file on start.
2026-08-22 13:35:31 -07:00
Frank bb1ef9bfb3 caddy: keep admin on for hot reload; deploy.sh reloads after render
The Caddyfile had 'admin off', which disables Caddy's config file
watcher. Rendered Caddyfile changes never loaded until the container
was recreated, so the double-hash basicauth fix sat on disk unapplied
across deploys (2026-08-22). Admin binds to 127.0.0.1:2019 inside the
container and the port is unpublished, so nothing outside reaches it.

deploy.sh now issues an explicit 'caddy reload' (retry loop) after
'up -d', with the file watcher as backstop.
2026-08-22 13:22:01 -07:00
Frank 5c9ade4e7a deploy.sh: fix quoting of the cfhash label template 2026-08-22 04:16:18 -07:00
Frank e0a25415a1 deploy.sh: rebuild local images when their Containerfile hash changes
buildah can serve a stale COPY --from layer (cache keyed on stage
name), so a Containerfile edit silently produced an unchanged image.
The image now carries a cfhash label; a mismatch forces a --no-cache
rebuild.
2026-08-22 04:14:43 -07:00
Frank dc421efea4 deploy.sh: escape the image-build loop vars for the remote string 2026-08-22 04:04:24 -07:00
Frank 1f4766f233 observability: fix alloy ui component, busybox healthchecks for loki/alloy
- alloy 1.x renamed the server component to ui; the old block failed the
  initial load and the container crash-looped
- grafana/loki is distroless and grafana/alloy lacks wget, so their
  healthchecks could never run; local images add a static busybox from
  alpine for that one command
- deploy.sh builds all three local images if missing
- tests: 3 new seams
2026-08-22 03:59:14 -07:00
Frank d5e9c63802 deploy.sh: drop --pull=missing (podman-compose rejects it) 2026-08-22 02:51:36 -07:00
Frank 5caa2178c3 deploy.sh: extract CADDY_BASICAUTH_HASH via grep/cut, not by sourcing .env
Sourcing the .env in a set -u shell dies on the bcrypt hash's $2
expansion. grep/cut pulls the value without any shell interpretation.
2026-08-22 02:51:13 -07:00
Frank 7f4ce06927 Caddy phase 2: TLS via Porkbun DNS-01 wildcard certs
- Caddyfile: drop http:// site prefixes, global acme_dns porkbun block,
  HTTPS redirects on the bare-IP and legacy vhosts
- new dankclaw/caddy/Containerfile: xcaddy build with caddy-dns/porkbun
- compose: caddy uses franksdank/caddy-porkbun:latest + Porkbun env vars
- deploy.sh: build the image on first deploy if missing
- .env.example: rename PORKBUN_API_SECRET -> PORKBUN_API_SECRET_KEY
  (the name the caddy-dns/porkbun module expects)
- tests: 7 new caddyfile/compose seams for the TLS config
2026-08-22 02:39:41 -07:00