Files

T

Frank 35c44c4123 docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash)

Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index,
Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream
llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0
draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16).
deploy.sh now maps dankserver to frank-default-small and symlinks its unit.
models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.

2026-08-11 01:18:29 -07:00

2.8 KiB

Raw Blame History

AGENTS.md — vllmScripts Agent Rules

Overview

This project manages deployment scripts for two LLM inference servers. Dankclaw runs a vLLM-ROCm service for Ling-3.0-flash (CIRU INT4). Dankserver runs a llama.cpp service for Qwen3.6-35B-A3B (HauhauCS uncensored Q6_K_P) with a DFlash draft and CPU-offloaded MoE experts. See docs/deployment.md for the full deployment and update process.

Git Workflow

All script edits must flow through git. Do not edit files on the remote machine directly.

Protocol

Source the script from the target machine — fetch it from the server where it runs
Place it in this repo — commit the fetched version
Edit here — make all changes locally in the working tree
Commit and push — push to origin (Gitea on dankclaw)
Pull on the target — git pull on the remote machine before restarting
Restart the service — use the updated script on the target

Deployment & Update Process

Use scripts/deploy.sh to deploy committed changes to a target. It pushes main, pulls on the target, restarts the service, and checks the health endpoint.

./scripts/deploy.sh dankclaw        # vLLM (frank-default-large, port 7770)
./scripts/deploy.sh dankserver      # llama.cpp (frank-default-small, port 7770)
./scripts/deploy.sh dankclaw --dry-run   # preview without changes

Deployment workflow:

graph LR
    A[Edit locally] --> B[Commit]
    B --> C[Push to origin]
    C --> D[deploy.sh: pull on target]
    D --> E[restart service]
    E --> F[health check]

Target map:

Target	Host	Runtime	Service	Port
Dankclaw	dank@dankclaw	vLLM-ROCm (CIRU INT4)	frank-default-large	7770
Dankserver	dank@dankserver	llama.cpp (Q6_K_P + DFlash, MoE offload)	frank-default-small	7770

Why

Direct edits on the remote drift from version control. Future agents will not know what changed. Git is the single source of truth.

User request → Fetch script from remote → Commit here → Edit → Commit → Push → Pull on remote → Restart

Summary

Context	Action
Editing a deployment script	Work through git — fetch, commit, edit, push, pull, restart. Never edit on the remote directly.
Script exists only on remote	Pull it into the repo first — commit the fetched version before making changes.
Unstaged local changes	Commit or discard — check `git status` before starting work. Ask the user if unsure.
Deploying a committed change	Run `scripts/deploy.sh <target>` — it pushes, pulls on the target, restarts the service, and checks health. See docs/deployment.md.
Tests exist	Run them — verify tests pass after every change. Do not commit failing tests.

2.8 KiB Raw Blame History