docs: record quadlet standardization of dankserver media + dankclaw data roots
Phase 1/2 of the podman standardization plan: service pages updated to the quadlet reality (units, Containers data roots, env-file secret convention, the v1-generator findings each caught live), Storage Policy documents the two rootless ownership regimes and the merge-move migration, Deployment Flow's service-manager table no longer claims dankserver media is bare inventory-only. Bundle change log entry added.
This commit is contained in:
1 parent
34ab6b4d0c
commit
272ef6f0b0
7 files changed
+150
-40
No files matched your search
@@ -63,7 +63,7 @@ service on the other host.
|
|||||||
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
|
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
|
||||||
| `frank-default-large` | systemd + `systemctl` | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
|
| `frank-default-large` | systemd + `systemctl` | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
|
||||||
| `frank-default-small` | systemd + `systemctl` | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
|
| `frank-default-small` | systemd + `systemctl` | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
|
||||||
| Dankserver media inventory | host-managed bare containers | Compose is inventory only; the scoped deploy does not create duplicate containers. |
|
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
|
||||||
|
|
||||||
The shared chat template maps to the inference service on the selected target.
|
The shared chat template maps to the inference service on the selected target.
|
||||||
The inactive `frank-default-npu` unit is relink-only when explicitly selected;
|
The inactive `frank-default-npu` unit is relink-only when explicitly selected;
|
||||||
|
|||||||
@@ -11,30 +11,50 @@ All persistent data uses **bind mounts**: a host directory mapped directly into
|
|||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
| Stack | Host root |
|
| Stack | Host data root |
|
||||||
|-------|-----------|
|
|-------|----------------|
|
||||||
| Dankserver | `/opt/<service>/...` |
|
| Dankserver | `/home/dank/Containers/<svc>/` (romm, jellyfin, calibre-web, copyparty) |
|
||||||
| Dankclaw | `/opt/<service>/...` (Gitea keeps `/opt/gitea/data/...`) |
|
| Dankclaw | `/home/dank/Containers/<svc>/` (caddy, gitea, authentik, hermes, …) |
|
||||||
|
|
||||||
|
This layout was standardized on 2026-09-01. The old `/opt/<service>` roots were
|
||||||
|
migrated by the deployers' `migrate_data_root` (merge-move, numeric ownership
|
||||||
|
preserved); `/opt` holds no live container data on either host.
|
||||||
|
|
||||||
## Ownership
|
## Ownership
|
||||||
|
|
||||||
Containers run as `PUID=1000` / `PGID=1000`. Host directories must be owned by `dank`:
|
Containers are rootless. Two regimes exist and the deployer must respect both:
|
||||||
|
|
||||||
|
- **PUID/PGID images** (linuxserver calibre-web, jellyfin): data is owned by
|
||||||
|
host uid 1000 (`dank`) — plain `chown` is safe.
|
||||||
|
- **Mapped-uid images** (official mariadb, redis, authentik, postgres): files
|
||||||
|
land in the subuid range (e.g. 100998 for container uid 999). Renaming such
|
||||||
|
a directory requires root (rename needs write on the directory itself) and
|
||||||
|
**re-chowning to 1000 breaks the container's internal user**. Migrations
|
||||||
|
therefore use `sudo mv` (numeric-preserving) and skip the chown pass
|
||||||
|
(`migrate_data_root … keep`).
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo chown -R 1000:1000 /opt/<service>
|
stat -c '%u %n' /home/dank/Containers/romm/mariadb # 100998 = mapped, leave it
|
||||||
```
|
```
|
||||||
|
|
||||||
## Secrets
|
## Secrets
|
||||||
|
|
||||||
Secrets never live in bind-mounted configs that get committed. They are:
|
Secrets never live in bind-mounted configs that get committed. The quadlet
|
||||||
|
convention on both hosts:
|
||||||
|
|
||||||
1. Set in `.env` files on the host (git-ignored, `.env.example` committed as template), or
|
1. One host-local env file per stack at `/home/dank/Containers/<svc>/env`
|
||||||
2. Injected as environment variables from the deploy script.
|
(mode 0600), referenced from the unit's `[Service] EnvironmentFile=` so
|
||||||
|
`${VAR}` expansion works at `ExecStart` time; never committed.
|
||||||
|
2. `.env` files in the checkout (git-ignored, `.env.example` committed as
|
||||||
|
template) for compose-era and Caddy-rendered secrets.
|
||||||
|
|
||||||
The live Gitea database password exists only in `.env` on Dankclaw. A test enforces that it never appears in this repo.
|
Deployed units hold only variable expansions (`${COPAY_ACCOUNT}` …), never
|
||||||
|
literals — enforced by the `no secret material` seams. The live Gitea
|
||||||
|
database password exists only in `/home/dank/Containers/gitea/env` on
|
||||||
|
Dankclaw. A test enforces that it never appears in this repo.
|
||||||
|
|
||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
- [[Deployment Flow]]
|
- [[Deployment Flow]]
|
||||||
- [[Gitea]] — the existing bind-mount stack being migrated
|
- [[Gitea]] — the original bind-mount stack, now on the Containers layout
|
||||||
- [[Backblaze Integration]] — future backup target for these mounts
|
- [[Backblaze Integration]] — future backup target for these mounts
|
||||||
@@ -49,12 +49,20 @@ Additional packages added: `libxfixes3` and `libasound2t64` to support epub-to-p
|
|||||||
|
|
||||||
## Volumes
|
## Volumes
|
||||||
|
|
||||||
| Container path | Host path (convention) |
|
| Container path | Host path |
|
||||||
|----------------|------------------------|
|
|----------------|-----------|
|
||||||
| `/config` | `/opt/calibre-web/config` |
|
| `/config` | `/home/dank/Containers/calibre-web/config` |
|
||||||
| `/books` | The existing Calibre library directory (user-entered during setup) |
|
| `/books` | `/home/dank/Containers/calibre-web/books` |
|
||||||
|
|
||||||
## Docker Compose
|
Deployment since 2026-09-01 is a quadlet unit
|
||||||
|
(`dankserver/calibre-web/quadlet/calibre-web.container` →
|
||||||
|
`calibre-web.service`, deploy with
|
||||||
|
`./scripts/deploy.sh dankserver --service calibre-web`), image pinned to
|
||||||
|
`0.6.27-ls397`, healthcheck on `/`, `PublishPort=8083:8083`. This service was
|
||||||
|
already on the Containers data-root layout before the standardization, so the
|
||||||
|
cutover was a bare-container retire + unit start with no data movement.
|
||||||
|
|
||||||
|
## Docker Compose (reference — retired 2026-09-01; live mounts are under `/home/dank/Containers/calibre-web`)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
services:
|
services:
|
||||||
|
|||||||
@@ -80,7 +80,7 @@ takes its volume roots from argv plus `chdir: /w` in `/z/initcfg`, not from a
|
|||||||
rw on disk — copyparty enforces read-only at the application layer, so local
|
rw on disk — copyparty enforces read-only at the application layer, so local
|
||||||
services (Jellyfin, Calibre) keep writing into the array directly.
|
services (Jellyfin, Calibre) keep writing into the array directly.
|
||||||
|
|
||||||
## Docker Compose
|
## Docker Compose (reference — retired 2026-09-01; live deployment is the quadlet unit below)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
copyparty:
|
copyparty:
|
||||||
@@ -100,20 +100,28 @@ services (Jellyfin, Calibre) keep writing into the array directly.
|
|||||||
retries: 3
|
retries: 3
|
||||||
```
|
```
|
||||||
|
|
||||||
`${COPAY_PASS}` is compose-time interpolation from `dankserver/.env` — the
|
`${COPAY_ACCOUNT}`/`${COPAY_PASSWORD}` are **never** written into the unit:
|
||||||
password lands in the container's **argv**, visible to anyone who can run
|
they live in the host-local `/home/dank/Containers/copyparty/env` (0600),
|
||||||
|
loaded via `[Service] EnvironmentFile=` so systemd expands them into
|
||||||
|
`Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD}` at start. The password still
|
||||||
|
lands in the container argv — visible to anyone who can run
|
||||||
`podman inspect copyparty` on the host.
|
`podman inspect copyparty` on the host.
|
||||||
|
|
||||||
**Applying changes:** `scripts/deploy.sh` deliberately does not touch the
|
## Deployment (Quadlet)
|
||||||
dankserver compose stack (host containers are not quadlet-managed there), so
|
|
||||||
`command:` edits need a manual recreate on Dankserver:
|
|
||||||
|
|
||||||
```
|
Since 2026-09-01: `dankserver/copyparty/quadlet/copyparty.container` →
|
||||||
cd /home/dank/serverAppConfigs/dankserver && podman-compose up -d --force-recreate copyparty
|
`copyparty.service`, deployed with
|
||||||
```
|
`./scripts/deploy.sh dankserver --service copyparty`. Cutover details:
|
||||||
|
|
||||||
Same project (`dankserver`), only copyparty is recreated; a fresh container
|
- v1 generator rejects `Command=` → the argv lives in `Exec=`.
|
||||||
answers `000` for the first seconds while it mounts.
|
- No `HealthCmd=` — the v1 generator splits unquoted spaces in that key;
|
||||||
|
self-healing is `Restart=always`.
|
||||||
|
- `PublishPort=3923:3923` — the first cutover dropped it (unit joined
|
||||||
|
`dankserver_appnet`), copyparty answered inside the netns only; the
|
||||||
|
published-port seam test now pins 3923 alongside 8085/8096/8083.
|
||||||
|
- Config root moved `/opt/copyparty` → `/home/dank/Containers/copyparty/cfg`.
|
||||||
|
- The deployer refuses to start the unit before the env file exists, so a
|
||||||
|
missing seed cannot create an admin account with a garbage password.
|
||||||
|
|
||||||
## Healthcheck
|
## Healthcheck
|
||||||
|
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ graph LR
|
|||||||
J --> Cache[Jellyfin cache]
|
J --> Cache[Jellyfin cache]
|
||||||
```
|
```
|
||||||
|
|
||||||
## Docker Compose
|
## Docker Compose (reference — retired 2026-09-01; live mounts use `/home/dank/Containers/jellyfin`, see Deployment below)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
services:
|
services:
|
||||||
@@ -82,11 +82,23 @@ Set **Base URL** to `/` (or the path you configure) in Jellyfin Dashboard → Ne
|
|||||||
|
|
||||||
| Mount | Purpose | Size Estimate |
|
| Mount | Purpose | Size Estimate |
|
||||||
|-------|---------|---------------|
|
|-------|---------|---------------|
|
||||||
| `/opt/jellyfin/config` | SQLite database, settings, cached images | 10–100 GB |
|
| `/home/dank/Containers/jellyfin/config` | SQLite database, settings, cached images | 10–100 GB |
|
||||||
| `/opt/jellyfin/cache` | Transcoding segments, extracted posters | Self-cleaning per session |
|
| `/home/dank/Containers/jellyfin/cache` | Transcoding segments, extracted posters | Self-cleaning per session |
|
||||||
|
| `/mnt/entropy/Movies`, `/mnt/entropy/TVShows` | Media, **read-only binds** (`:ro`) — no automated write path | — |
|
||||||
|
|
||||||
The database must be on local storage (SSD/NVMe preferred). Media libraries can be on NFS/mergerfs. If media is on NFS, set `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`.
|
The database must be on local storage (SSD/NVMe preferred). Media libraries can be on NFS/mergerfs. If media is on NFS, set `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`.
|
||||||
|
|
||||||
|
## Deployment (Quadlet)
|
||||||
|
|
||||||
|
Since 2026-09-01: `dankserver/jellyfin/quadlet/jellyfin.container` →
|
||||||
|
`jellyfin.service` (generator-enabled), deployed with
|
||||||
|
`./scripts/deploy.sh dankserver --service jellyfin`. GPU passthrough uses
|
||||||
|
`AddDevice=/dev/dri` — plain `Device=` is **not** a quadlet key and makes the
|
||||||
|
v1 generator reject the whole unit (learned live: "unsupported key 'Device'
|
||||||
|
in group 'Container'"). Entropy mounts stay `:ro`; the container runs as
|
||||||
|
root inside the user namespace, so the render-node ACLs apply via the device
|
||||||
|
cgroup regardless of `GroupAdd=keep-groups`.
|
||||||
|
|
||||||
## Authentication
|
## Authentication
|
||||||
|
|
||||||
- **Built-in auth (default):** Per-user libraries, parental controls, max bitrate limits. Manage via Dashboard → Users.
|
- **Built-in auth (default):** Per-user libraries, parental controls, max bitrate limits. Manage via Dashboard → Users.
|
||||||
@@ -147,7 +159,7 @@ Run Jellyfin alongside Plex for an incremental migration. Point Jellyfin at the
|
|||||||
|
|
||||||
1. Verify GPU on host: `ls -l /dev/dri/`, `lspci | grep -i vga`, `getent group render`.
|
1. Verify GPU on host: `ls -l /dev/dri/`, `lspci | grep -i vga`, `getent group render`.
|
||||||
2. Install media drivers: `sudo apt install intel-media-va-driver-non-free vainfo intel-gpu-tools`.
|
2. Install media drivers: `sudo apt install intel-media-va-driver-non-free vainfo intel-gpu-tools`.
|
||||||
3. Create directories: `/opt/jellyfin/config`, `/opt/jellyfin/cache`.
|
3. Create directories: `/home/dank/Containers/jellyfin/config`, `/home/dank/Containers/jellyfin/cache`.
|
||||||
4. Write compose file, verify device and GID.
|
4. Write compose file, verify device and GID.
|
||||||
5. Deploy: `podman compose up -d`.
|
5. Deploy: `podman compose up -d`.
|
||||||
6. Test GPU passthrough: `podman exec jellyfin /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128`.
|
6. Test GPU passthrough: `podman exec jellyfin /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128`.
|
||||||
|
|||||||
@@ -73,7 +73,7 @@ These are set via environment variables or in `config.yml`:
|
|||||||
|
|
||||||
### Config File
|
### Config File
|
||||||
|
|
||||||
RomM uses `config.yml` for scan settings, available through the admin UI (Scan Settings). Place the file at `/opt/romm/config/config.yml` on the host.
|
RomM uses `config.yml` for scan settings, available through the admin UI (Scan Settings). Place the file at `/home/dank/Containers/romm/config/config.yml` on the host.
|
||||||
|
|
||||||
Key scan config options:
|
Key scan config options:
|
||||||
|
|
||||||
@@ -99,17 +99,40 @@ scan:
|
|||||||
export: true # Export ES-DE compatible gamelist.xml
|
export: true # Export ES-DE compatible gamelist.xml
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Deployment (Quadlet)
|
||||||
|
|
||||||
|
Since 2026-09-01 the trio runs as repo-tracked quadlet units, deployed with
|
||||||
|
`./scripts/deploy.sh dankserver --service romm` (any of `romm`,
|
||||||
|
`romm-mariadb`, `romm-redis` triggers the full ordered cutover):
|
||||||
|
|
||||||
|
- Units: `dankserver/romm/quadlet/{romm,romm-mariadb,romm-redis}.container`
|
||||||
|
→ generated `romm.service` etc., enabled via the quadlet generator's
|
||||||
|
`default.target.wants` (`is-enabled` reports `generated`).
|
||||||
|
- Secrets: host-local `/home/dank/Containers/romm/env` (0600, 15 app vars —
|
||||||
|
compose-provided keys only; image-baked vars like `HOME`/`PATH` must never
|
||||||
|
enter it, they poison the podman client's service environment).
|
||||||
|
- Network: `dankserver_appnet` with aliases `mariadb`/`redis` (replaces
|
||||||
|
compose service discovery; `DB_HOST`/`REDIS_HOST` still resolve).
|
||||||
|
- Publish: `8085:8080` (`PublishPort=`, see the seam test on published ports).
|
||||||
|
|
||||||
## Volumes
|
## Volumes
|
||||||
|
|
||||||
| Container path | Host path (convention) |
|
| Container path | Host path |
|
||||||
|----------------|------------------------|
|
|----------------|-----------|
|
||||||
| `/romm/library` | `/opt/romm/library` |
|
| `/romm/library` | `/home/dank/Containers/romm/library` |
|
||||||
| `/romm/resources` | `/opt/romm/resources` |
|
| `/romm/resources` | `/home/dank/Containers/romm/resources` |
|
||||||
| `/romm/config` | `/opt/romm/config` |
|
| `/romm/config` | `/home/dank/Containers/romm/config` |
|
||||||
|
| `/var/lib/mysql` | `/home/dank/Containers/romm/mariadb` (mapped subuid owner — never chown) |
|
||||||
|
| `/data` (redis) | `/home/dank/Containers/romm/redis` (mapped subuid owner) |
|
||||||
|
|
||||||
The library volume contains ROM files and subdirectories per platform. The resources volume stores cached artwork, metadata, and the SQLite/SQLite-backed asset cache. The config volume holds `config.yml` and other configuration.
|
The library volume contains ROM files and subdirectories per platform. The resources volume stores cached artwork, metadata, and the asset cache. The config volume holds `config.yml` and other configuration.
|
||||||
|
|
||||||
## Docker Compose
|
The cutover rescued RomM's anonymous `/romm` volume (assets, plus empty
|
||||||
|
mountpoint stubs that shadowed nothing) into the data root before first unit
|
||||||
|
start — the mechanism behind the original "RomM data wiped" incident is now
|
||||||
|
structurally impossible: the units declare bind mounts only.
|
||||||
|
|
||||||
|
## Docker Compose (reference — retired 2026-09-01, superseded by the quadlet units above)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
services:
|
services:
|
||||||
|
|||||||
@@ -11,6 +11,45 @@ timestamp: 2026-09-14
|
|||||||
|
|
||||||
# Bundle Change Log
|
# Bundle Change Log
|
||||||
|
|
||||||
|
## 2026-09-01 — dankserver media stack standardized to quadlet; dankclaw data roots migrated
|
||||||
|
- **Incident context**: Dankserver hard-crashed and rebooted 2026-09-01; every
|
||||||
|
bare compose-era media container (RomM trio, Jellyfin, Calibre-Web,
|
||||||
|
Copyparty) stayed dead at boot — the failure mode the Boot Persistence rule
|
||||||
|
exists for. Recovery doubled as the Phase 1 cutover.
|
||||||
|
- **Phase 1 (dankserver)**: six quadlet units shipped
|
||||||
|
(`dankserver/{romm,jellyfin,calibre-web,copyparty}/quadlet/`) + first-cutover
|
||||||
|
deploy handlers (retire bare containers → rescue RomM's anonymous `/romm`
|
||||||
|
volume → merge-move `/opt/<svc>` into `/home/dank/Containers/<svc>` →
|
||||||
|
ordered start with health gates). All six verified: units `active`,
|
||||||
|
`is-enabled=generated` (wants in the generator dir), ports 8085→8080,
|
||||||
|
8096, 8083, 3923 answering, jellyfin holds `/dev/dri` render nodes,
|
||||||
|
entropy binds provably ro except copyparty's `/w`. MariaDB continuity
|
||||||
|
proven via the real `dank` admin row (created 2026-08-31), not a fresh
|
||||||
|
bootstrap. Stale compose-era `container-*.service` generate-systemd units
|
||||||
|
retired to `~/.AGENTTRASH/20260901-stale-generate-systemd`.
|
||||||
|
- **Deployer findings fixed** (each caught live, each with a new seam):
|
||||||
|
v1 generator rejects `Command=` (→ `Exec=`), `Device=` (→ `AddDevice=`,
|
||||||
|
whole-unit rejection is silent), and splits spaces in `HealthCmd=`
|
||||||
|
(→ omitted for copyparty); `migrate_data_root` must merge into pre-seeded
|
||||||
|
destinations and rename mapped-subuid roots (100998) with `sudo mv`
|
||||||
|
(numeric-preserving, `keep` mode skips the chown — chowning mapped data
|
||||||
|
breaks the in-container users); quadlet unit dir must be created on a host
|
||||||
|
that never had quadlet; missing `PublishPort` = app answers inside the
|
||||||
|
netns only (copyparty 3923); seeded env files must carry **compose-provided
|
||||||
|
vars only** — the image-baked `HOME=/root` in a `[Service]
|
||||||
|
EnvironmentFile=` kills the podman client itself.
|
||||||
|
- **Phase 2 (dankclaw)**: data roots of gitea, authentik, hermes trios moved
|
||||||
|
to `/home/dank/Containers/<svc>/` while running (rename is inode-bound —
|
||||||
|
zero downtime), units repointed, redeployed through `deploy.sh --service`;
|
||||||
|
auth chain verified end-to-end (outpost ping → 302 to flow).
|
||||||
|
`deploy_hermes` ordering bug fixed: generated units must be reloaded
|
||||||
|
**after** unit files are copied, not before (stale ExecStart crash-looped
|
||||||
|
the trio on the old path).
|
||||||
|
- **Docs**: [[RomM]], [[Jellyfin]], [[Calibre-Web]], [[Copyparty]] updated to
|
||||||
|
the quadlet reality; [[Storage Policy]] now documents the Containers layout
|
||||||
|
+ the two ownership regimes; [[Deployment Flow]] service-manager table
|
||||||
|
corrected (dankserver media is no longer "bare containers, inventory only").
|
||||||
|
|
||||||
## 2026-08-30 (night II) — frank-default-sightsound deployed; NPU SVA/IOMMU root cause
|
## 2026-08-30 (night II) — frank-default-sightsound deployed; NPU SVA/IOMMU root cause
|
||||||
- **Deployed** `frank-default-sightsound` on Dankclaw: FLM v1.0.1 (`/usr/bin/flm`,
|
- **Deployed** `frank-default-sightsound` on Dankclaw: FLM v1.0.1 (`/usr/bin/flm`,
|
||||||
dpkg `fastflowlm`) serving the whole local registry (36 tags) on port 52626;
|
dpkg `fastflowlm`) serving the whole local registry (36 tags) on port 52626;
|
||||||
|
|||||||
Reference in new issue
Block a user