command-center/agent/managed-skills/swapfile-resize-safe-ordering/SKILL.md

name, description

name	description
swapfile-resize-safe-ordering	Resize a swapfile on a host whose MemAvailable is inflated by pinned model weights or huge page cache: put the replacement swap online BEFORE swapoff so live pages migrate to disk instead of into RAM that does not exist.

Resizing a swapfile without triggering the OOM killer

You cannot grow an active swapfile — it must be swapped off first. On a host that is already tight, swapoff must move every live swapped page somewhere, and if the replacement is not online yet, that somewhere is RAM.

The headline number lies. MemAvailable counts reclaimable page cache, and pinned/locked model weights plus ordinary cache make it read tens of GiB while only a few GiB is genuinely free. Never size a swapoff against free or MemAvailable.

The ordering

Create and enable the replacement while the old swap is still live, so pages migrate to disk:

# 1. Replacement online first — old swap untouched
sudo fallocate -l 64G /swap2.img
sudo chmod 600 /swap2.img
sudo mkswap /swap2.img        # also the hole check; refuses sparse files
sudo swapon /swap2.img        # total swap = old + new

# 2. Now drain the old one — pages go into the new swap, not RAM
sudo swapoff /swap.img        # minutes; watch `free` for used-RAM creep

# 3. Consolidate onto the conventional path so fstab needs no edit
sudo swapoff /swap2.img       # only the residual is left
sudo mkdir -p ~/.AGENTTRASH/<date>-swapfile-<oldsize>
sudo mv /swap.img ~/.AGENTTRASH/<date>-swapfile-<oldsize>/swap.img
sudo mv /swap2.img /swap.img
sudo swapon /swap.img

Chain step 3 as one tight command — it leaves the host with zero swap for a few seconds. Do not leave that window open.

Why each step

• mkswap is a free correctness gate. It rejects swapfiles containing holes, so a sparse fallocate fails loudly at format time instead of at 3 a.m. under memory pressure. Verify with filefrag if unsure; unwritten extents are allocated, not holes, and are fine.

• Consolidating to the original path means fstab never changes, so boot persistence carries over with no edit and no risk of a stale swap entry dropping boot to emergency. Renaming an active swapfile works (inode-based) but leaves a stale path in /proc/swaps — swap it off first, then rename, then swapon.

• Migration is observable. In the reference run, swapoff of 4 GiB took 65 s and used-RAM rose only ~3 GiB (pages touched during migration), the rest relocated on disk. If used-RAM climbs toward the whole swapped amount, abort reasoning and let it finish or re-enable.

Verify

swapon --show=NAME,TYPE,SIZE,USED,PRIO
systemctl is-active <file>.swap     # e.g. swap.img.swap — generated from fstab
grep swap /etc/fstab                # entry intact -> boot persistence

Then prove the workload survived with a real completion, not /health.

Filesystem caveat

ext4 and xfs take swapfiles directly. btrfs needs chattr +C (NOCOW), no compression, and no snapshots in the path — check df -T before planning, or the swapfile will be rejected.

After the change

More swap does not relieve pinned memory. Check vm.swappiness: at the default 60 the kernel will page anonymous memory under pressure, and if a large service's hot working set is anonymous (an LLM KV pool, for instance), paging it is worse than the OOM kill it prevents. The knob for that is lower swappiness or a cgroup cap on the service — not a bigger swapfile.

Stage the old file in trash rather than deleting it, and get explicit approval before purging.