Frank
33eae71d33
All ~173 managed skills read in full and merged, one polished reference page per skill-cluster: - LLM cross-cutting (17 pages): GGUF Compatibility, Llama Cpp Serving Sizing, Llama Cpp Multi-Device Splits, PCIe Link and Atomics, RDNA4 Container Serving, RDNA4 MoE Expert Offload VRAM Budget, vLLM Boot Forensics, vLLM Porting and Compile Triage, LLM Bench Windows, LLM Performance Diagnosis, Model Swap and Rollback, Tcclaviger Fork Ops, Heterogeneous Split Triage, FreeToken Builds + deltas folded into the retired runbooks (GLM DFlash2 port laws, Lucebox dual-R9700, R9V qualification pipeline, FLM pack registration, ik hybrid envelope, frank-medium split rebalance, llama.cpp GGUF preflight, batch-ubatch split section, bench-window D-state/stale-read notes, second-endpoint bring-up). - Architecture (3): Stale Signal Audit, Quadlet Lifecycle, Deploy State Triage. - Operations (6 + index): Remote Job Runner, Payload and Disk Reclaim, Container Host Reachability, Fleet Ops Playbook, Fleet Audit and Posture, SSH and Remote Access. - Infrastructure: Observability Provisioning + Homepage Status Probes; updates to Authentik, Anubis, LAN-DNS (per-client DoH bypass, Porkbun zone ops), Swap Configuration, OpenWebUI, Hermes (pre-existing page preserved and extended). - FrankBook-Pro: ARR-Stack filled (API split, schema-first probing, auth-posture, auth-config trap); qBittorrent E2E verification. - Desktop/Agent/Research: Noctalia and Niri, OMP Harness, Agent Docs and Edits, Command Center Dotfiles, PICO-8 Asset Standards. - Indexes: LLM Cross-Cutting References section, Operations/ and Agent/ index pages, root-index Skill Distillation section, AGENTS.md pointer. - log.md carries the wave plan. check-vault-links.py 0 unresolved (185 pages); ./tests/run.sh 366/366.
98 lines
4.4 KiB
Markdown
98 lines
4.4 KiB
Markdown
---
|
|
type: reference
|
|
title: FreeToken Builds
|
|
description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit.
|
|
tags: [freetoken, build, dankclaw, vllm, mtp]
|
|
timestamp: 2026-10-05T00:00:00-07:00
|
|
---
|
|
|
|
# FreeToken Builds
|
|
|
|
FreeToken = the official franksdank fork of FreeTok Tokenworks' freetoken
|
|
vLLM-MTP runtime. Two build paths exist, both on **Dankclaw**, with
|
|
different dependency traps. Both were verified end-to-end 2026-09-11/12.
|
|
|
|
## Common verification (do not skip)
|
|
|
|
**Verify the build by real completion, not by version string.** A version
|
|
bump alone does not move data through the model — a build can print a new
|
|
version and still not have completed a real token. Before declaring a
|
|
FreeToken build deployable, run an actual completion (any tiny prompt) and
|
|
read the returned content:
|
|
|
|
```python
|
|
import json, urllib.request
|
|
req = urllib.request.Request("http://<host>:<port>/v1/chat/completions",
|
|
json.dumps({"model": "frank-small", "messages": [{"role": "user",
|
|
"content": "Hi"}]}).encode(), {"Content-Type": "application/json"})
|
|
print(json.load(urllib.request.urlopen(req, timeout=600)))
|
|
```
|
|
|
|
A 200 with a real `choices[0].message.content` is the only build green.
|
|
|
|
## CUDA build (the torch/vLLM dependency trap)
|
|
|
|
Both paths build on **Dankclaw** with the same preinstall state: Python
|
|
3.12.10 in the system venv and **Torch 2.10.0.dev** (nightly, repo
|
|
download.pytorch.org/whl/nightly/cu130, installed with
|
|
`uv pip install --index-strategy unsafe-best-match "torch==2.10.0.dev<date>+cu130"
|
|
--index-url https://download.pytorch.org/whl/nightly/cu130` — use
|
|
`--index-strategy unsafe-best-match`, `uv` otherwise refuses the dev
|
|
version). Verify the exact pair first; mismatched preinstalled deps are the
|
|
#1 build-failure cause.
|
|
|
|
Then install the **matching vllm dev wheel**:
|
|
|
|
```bash
|
|
uv pip install --index-strategy unsafe-best-match \
|
|
--index-url https://wheels.vllm.ai/<commit-sha>/vllm-0.6.6.dev0+g<shortsha>.cu130-cp312-cp312-linux_x86_64.whl \
|
|
--extra-index-url https://wheels.vllm.ai/nightly/cu130
|
|
```
|
|
|
|
The wheel URL pattern is fixed:
|
|
`https://wheels.vllm.ai/<full-commit>/vllm-0.6.6.dev0+g<8-char>.cu130-cp312-cp312-linux_x86_64.whl`
|
|
— read it back from vLLM's CI-run page (`run details →
|
|
build_and_install_package` job, "install the vllm wheel" step). **The trap:
|
|
vLLM refuses versions < its install_requires floor at runtime, so the wheel
|
|
version must be ≥ the runtime's floor.** A wheel built at a lower base
|
|
(0.5.x) downloads but `import vllm` fails with a version constraint error —
|
|
that failure looks like a build failure and is not one; upgrade the wheel,
|
|
not the code.
|
|
|
|
If `uv pip install -e . --no-deps` refuses (some repos pin the
|
|
`--no-deps` branch off), fall back to `python3 setup.py` inside the venv,
|
|
after `python -m venv --copies .venv && source .venv/bin/activate` and
|
|
`uv pip install --no-build-isolation "torch==…"` — the venv must NOT
|
|
re-resolve torch (a second torch from PyPI d1 breaks the CUDA extension ABI).
|
|
|
|
## ROCm build (gfx1151 via TheRock nightly)
|
|
|
|
Same venv discipline, ROCm wheels from TheRock's nightly index:
|
|
|
|
```bash
|
|
uv pip install --index-strategy unsafe-best-match \
|
|
--index-url https://rocm.nightlies.amd.com/v2/gfx1151-dcgpu-supports_fbsgens_split_0717-rc1/ \
|
|
"torch==2.10.0.dev<date>+rocm" "triton==3.6.0" "pytorch-triton==3.6.0"
|
|
```
|
|
|
|
Then `vllm` itself (`uv pip install -e . --no-build-isolation --no-deps`,
|
|
repo branch `specs-mtp-dev`, commit `2ee440d`), after copying the two
|
|
prebuilt extension wheels into the tree (the repo's `setup.py` skips
|
|
compiling them when found). The ROCm build also needs
|
|
`install-fastrak-gfx1151.sh` from the repo's scripts and
|
|
`FASTRACK_HOME=/home/dank/FastTrack-runtime` in the unit env. FreeToken's
|
|
x86_64-gfx1151 prebuilt extension (`freetoken-ops`) ships on HF hub
|
|
`franksdank/freetoken-ops-gfx1151` — fetch it once and point
|
|
`FREETOKEN_OPS_HOME` at it; never rebuild the extension from source unless
|
|
the extension API changed.
|
|
|
|
`--index-strategy unsafe-best-match` is load-bearing on both paths: the
|
|
nightly indexes carry version pairs that `uv`'s strict resolver otherwise
|
|
rejects, and the error (`No candidate found for pytorch-triton`) reads like
|
|
a missing package.
|
|
|
|
## Related
|
|
|
|
- [[Model Swap and Rollback]] — the swap/deploy story these builds serve
|
|
- [[Llama Cpp Serving Sizing]] — the harness measurement discipline
|