Files

T

Frank 327e78b950 docs(vault): destroy the dead weight — 53 pages and the change diary

The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.

Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
  the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
  campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
  "Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
  and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
  owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
  self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
  page that owns the thing it governs, never to a diary.

Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.

2026-10-19 10:56:37 -07:00

9.6 KiB

Raw Permalink Blame History

type, description, tags, timestamp

type

description

ComfyUI

ComfyUI is a node-based AI image generation engine. Workflows are built from nodes that load models, sample latents, and post-process images; everything is scriptable through a web UI and an API.

Host: Dankclaw — Port: 0.0.0.0:8188 published, no auth (ss -ltnp 2026-10-18) — no vhost and no guard chain: the Caddyfile routes no name to it (no comfyui or 8188 line anywhere in dankclaw/Caddyfile), so LAN access rides the Pi-hole A record comfyui.franksdank.cloud (dankclaw/pihole/addn-hosts:37) plus the raw port — the homepage card links http://comfyui.franksdank.cloud:8188 and probes 127.0.0.1:8188 (shared/homepage/services.json:519-521) — Backend: ROCm GPU (gfx1151) — Docs: https://github.com/Comfy-Org/ComfyUI

Facts

Item	Value
Image	`localhost/comfyui-rocm:gfx1151` — built from `dankclaw/Containerfile.comfyui`, `Pull=never` (`dankclaw/comfyui/quadlet/comfyui.container:41-42`; `podman inspect comfyui` 2026-10-18)
Base image	`docker.io/rocm/vllm:rocm7.12.0_gfx1151_ubuntu24.04_py3.12_pytorch_2.9.1_vllm_0.16.0` (`dankclaw/Containerfile.comfyui:9`)
Upstream latest	0.37.0 — the version the running container reports (`GET http://127.0.0.1:8188/system_stats`, 2026-10-18). The release date 2026-09-21 and its native Qwen-Image 2.1 support (CORE-423, `comfy/ldm/qwen_image21/`) (unverified 2026-10-18) — the image's own startup patch still rewrites detection and VAE-decode code at every boot (`dankclaw/apply_qwen21_patch.py:4,23,37`, run by `dankclaw/comfyui-start.sh:3`), so check before re-applying on the next rebuild.
Port	8188 (host) → 8188 (container), published `0.0.0.0` (`dankclaw/comfyui/quadlet/comfyui.container:44`)
Backend	ROCm GPU (gfx1151 / Radeon 8060S). No `--cpu` flag — the unit runs `python main.py --listen 0.0.0.0 --port 8188` (`dankclaw/comfyui-start.sh:4`). `rocm-smi --showproductname` reports `GFX Version: gfx1151` (2026-10-18).
Data	`/home/dank/comfyui-data/{models,input,output,custom_nodes}` → `/opt/ComfyUI/...` (`dankclaw/comfyui/quadlet/comfyui.container:48-51`)
Run	`./scripts/deploy.sh dankclaw --service comfyui` (Quadlet unit `comfyui.service`; handler `deploy_comfyui`, `scripts/deploy-remote.sh:649`)
Boot-persistent	Quadlet unit `dankclaw/comfyui/quadlet/comfyui.container` → `~/.config/containers/systemd/`; live 2026-10-18 `systemctl --user is-enabled comfyui` → `generated`, `is-active` → `active`

Why the GPU works here

Dankclaw is an AMD Strix Halo APU (gfx1151). Standard PyTorch ROCm wheels do not ship gfx1151 kernels. So torch.cuda.is_available() returns False and ComfyUI falls back to CPU. AMD publishes a gfx1151-tagged build, and the image above is based on it. On this APU it reports torch.cuda.is_available() = True, device Radeon 8060S Graphics (GET http://127.0.0.1:8188/system_stats 2026-10-18: one device, cuda:0 Radeon 8060S Graphics : native, type cuda), capability (11, 5) (dankclaw/Containerfile.comfyui:4-5). See ComfyUI ROCm and Vulkan for the full verification.

How the image is built

dankclaw/Containerfile.comfyui starts from the AMD gfx1151 base image and bakes the ComfyUI code in. It installs ComfyUI's dependencies but skips the torch family (torch, torchvision, torchaudio). The base image already ships the matching ROCm builds. Installing an unpinned torch from requirements.txt clobbers the gfx1151 kernels with a default build, so the Containerfile filters those lines out.

GPU device access (rootless), and why the unit exists

The unit passes /dev/kfd and /dev/dri (AddDevice= dankclaw/comfyui/quadlet/comfyui.container:45-46) and uses GroupAdd=keep-groups. A literal render entry cannot work: rootless Podman cannot map the group name into the user namespace. The companion udev rule dankclaw/udev/99-comfyui-gpu-access.rules keeps those nodes MODE 0666, which is what lets a user-unit spawn — carrying no supplementary groups — open them anyway. The rule is installed on the host (/etc/udev/rules.d/99-comfyui-gpu-access.rules, ls 2026-10-18) and the nodes are world-writable today: crw-rw-rw- /dev/kfd, crw-rw-rw- /dev/dri/renderD128 (ls -l 2026-10-18). Verified through the running service: /system_stats reports the CUDA-type device, i.e. torch.cuda.is_available() is True and the device is Radeon 8060S Graphics.

Incident, 2026-09-16 (retired era) — why this is a Quadlet now. ComfyUI ran from dankclaw/compose.yaml (no longer in the repo) through a hand-written ~/.config/systemd/user/ comfyui.service that called podman compose up -d. Its container was created before the GPU declarations were added to the compose service, and restart: unless-stopped then kept restarting that stale container: no /dev/kfd, no /dev/dri, no keep-groups, dead on RuntimeError: No HIP GPUs are available — 9,146 restarts and about seven hours down. The service was fine; the container was old. A --force-recreate would have fixed it that day, but the class of bug needed the unit: one owner per container name, repo-tracked, and recreated from the source of truth on every deploy. The compose service is deleted; deploy_comfyui retires any compose-era unit into ~/.AGENTTRASH/ (scripts/deploy-remote.sh:649,660) and removes a stale container before installing the unit.

Volumes

Container path	Host path
`/opt/ComfyUI/models`	`/home/dank/comfyui-data/models`
`/opt/ComfyUI/input`	`/home/dank/comfyui-data/input`
`/opt/ComfyUI/output`	`/home/dank/comfyui-data/output`
`/opt/ComfyUI/custom_nodes`	`/home/dank/comfyui-data/custom_nodes`

The ComfyUI code is baked into the image (not mounted). Rebuild the image to update the code.

Custom nodes

Custom nodes go in /home/dank/comfyui-data/custom_nodes (mounted at /opt/ComfyUI/custom_nodes). ComfyUI loads every subdirectory there on start. Because this path is a bind mount, it shadows anything baked into the image — so custom nodes are installed host-side, not in the Containerfile.

ComfyUI-Manager is installed here, alongside ComfyUI-GGUF (ls /home/dank/comfyui-data/custom_nodes 2026-10-18 — the GGUF node's Python deps are baked into the image, dankclaw/Containerfile.comfyui:29). Install Manager with a one-time clone, then restart the container:

git clone https://github.com/ltdrdata/ComfyUI-Manager \
  /home/dank/comfyui-data/custom_nodes/ComfyUI-Manager
systemctl --user restart comfyui

ComfyUI-Manager self-installs its own pip dependencies on start (GitPython, PyGithub, matrix-nio, transformers, and more). To confirm it loaded, query GET /manager/version — it answers V3.41 today (curl http://127.0.0.1:8188/manager/version 2026-10-18). Its pip installs live in the container's writable layer, so they survive a restart and a reboot but not an image rebuild; after a rebuild, Manager re-installs them on the next start.

Boot persistence

The Quadlet unit carries [Install] WantedBy=default.target with Restart=always and RestartSec=10 (dankclaw/comfyui/quadlet/comfyui.container:66-70), so the generator wires it into default.target and it comes back on its own (user linger is on: loginctl show-user dank -p Linger → yes). Inspect it with systemctl --user is-enabled comfyui.service → generated, is-active → active (both 2026-10-18); an explicit systemctl --user enable fails on generated units.

Healthcheck

The web UI responds on / (200 text/html, curl http://127.0.0.1:8188/ 2026-10-18). ComfyUI also exposes /system_stats. On this APU, /system_stats reports device=cuda:0 Radeon 8060S Graphics (type cuda, the ROCm device). The health probe is the repo file dankclaw/comfyui/comfyui-healthcheck.py bind-mounted to /usr/local/bin/comfyui-healthcheck.py and run as HealthCmd (dankclaw/comfyui/quadlet/comfyui.container:52,57); it GETs http://127.0.0.1:8188/ inside the container, and the container reports healthy (podman inspect comfyui 2026-10-18).

Gotchas

The GPU path needs the device passthrough. Do not remove AddDevice=/dev/kfd or AddDevice=/dev/dri from the unit, and keep GroupAdd=keep-groups.
Do not add the render group by name. Rootless podman cannot map it; the udev rule plus keep-groups is the mechanism that works.
A stale container outlives a compose edit. If the GPU declarations change, recreate the container (deploy.sh dankclaw --service comfyui removes the old one) — restart: unless-stopped will happily keep an old config alive forever.
Do not let pip reinstall the torch family. The Containerfile filters it out on purpose.
The base image is about 32 GiB (podman image inspect localhost/comfyui-rocm:gfx1151 → 33,765,227,471 B ≈ 31.4 GiB, 2026-10-18). A code update means a rebuild, because the code is baked in.
The empty-models note from the 2026-09-16 era is retired: /home/dank/comfyui-data/models now holds the Qwen-Image 2.1 payload — three diffusion_models, three text_encoders, two VAEs (~35 GB, ls -l 2026-10-18), listed in ComfyUI Qwen-Image Workflows. New model classes (LoRA, ControlNet) still need their directories created first: models/lora and models/controlnet do not exist yet.

9.6 KiB Raw Permalink Blame History