Files
serverAppConfigs/dankclaw/observability/quadlet/alloy.container
Frank 634fef242b feat(alloy): run it as a root host-network service so it reads the journal
A rootless Alloy shipped zero lines: the journal files are
root:systemd-journal 0640 (other::---) and a rootless container carries no
supplementary groups, so sd_journal_open sees an empty view silently; the
rootless quadlet container logs also live in user-1000.journal, not the
ctr.log paths the old config tailed.

As root, one loki.source.journal reads system.journal + user-*.journal
and carries both. Network=host because Loki publishes 127.0.0.1:3100 and
no bridge container can reach the host loopback; the UI binds loopback
directly (no PublishPort). The /etc/machine-id mount is load-bearing —
sd_journal finds the journal dir by the local machine id; without it the
source reads 0 lines, health green.

deploy_alloy routes around the rootless loop: build rootless, mirror the
image into the root store (a root build fails on DNS in the build
container), retire the old rootless alloy unit, install to
/etc/containers/systemd. Validated with a throwaway root probe: 31,825
lines read, 26,966 pushed to Loki in 20s, zero drops.
2026-10-20 19:50:15 -07:00

60 lines
2.6 KiB
INI

# Quadlet unit: alloy (ROOT system service, boot-persistent).
#
# Install path on dankclaw: /etc/containers/systemd/alloy.container
# Unit name: alloy.service (container name: alloy) — SYSTEM manager, root.
#
# WHY ROOT (not rootless, unlike its observability siblings). Alloy reads the
# host journal. The journal files are root:systemd-journal 0640 (other::---);
# a rootless container carries no supplementary groups and cannot open them —
# a rootless read of system.journal returns DENIED (see the Alloy page in the
# vault). Running as root, one loki.source.journal reads system.journal AND
# user-1000.journal, so it carries both host logs and the rootless Quadlet
# containers' logs. Validated 2026-10-20: a root probe read 31,825 journal
# lines and pushed 26,966 into Loki in 20 s, zero drops.
#
# NETWORK=host. Loki publishes 127.0.0.1:3100 (loopback only) on the host, and
# no bridge container — rootless or root — can reach the host's loopback, so
# this unit runs on the host network and pushes to 127.0.0.1:3100. The side
# effect: the UI must bind loopback too, so Exec sets the listen-addr to
# 127.0.0.1 (the old rootless unit used 0.0.0.0 only because container loopback
# was not host loopback; under Network=host they are the same address, so the
# unauthenticated UI stays loopback-only — nothing off-host reaches it).
#
# THE /etc/machine-id MOUNT IS LOAD-BEARING. sd_journal locates the journal
# directory by the local machine id; without it the container opens nothing and
# the journal source reads exactly 0 lines — silently, no error, health green.
# This mount is the entire reason the pipeline carries data. Never remove it.
#
# IMAGE. Built rootless by build_compose_image alloy, then mirrored into the
# root image store by the deployer (a root build runs in the build container's
# own resolver and fails on DNS here). Pull=never: boot must not reach for the
# local tag at a registry.
[Unit]
Description=Alloy log shipper (Dankclaw)
After=network-online.target
[Container]
ContainerName=alloy
Image=localhost/franksdank/alloy:latest
Pull=never
Network=host
Exec=run --server.http.listen-addr=127.0.0.1:12345 /etc/alloy/config.alloy
Volume=/opt/alloy/config:/etc/alloy:ro
Volume=/etc/machine-id:/etc/machine-id:ro
Volume=/run/systemd/journal/socket:/run/systemd/journal/socket:ro
Volume=/var/log/journal:/var/log/journal:ro
Volume=/run/log/journal:/run/log/journal:ro
HealthCmd=/usr/local/bin/busybox wget --spider -q http://127.0.0.1:12345/-/healthy
HealthInterval=30s
HealthTimeout=10s
HealthRetries=3
HealthStartPeriod=30s
[Service]
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target