Frank
a5fe045b58
Structural repair: - 13 pages were the same document twice (each exactly 2x its HEAD size), a second frontmatter block whose closing fence had been glued onto a body line by a frontmatter rewrite that re-read its own output. Un-doubled by cutting at the second frontmatter and keeping the copy whose body is a superset; the orphan census reports 0 lost tokens against HEAD. - log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first — 66 dated headings merged into 34, titles demoted to bold leads, every content line verified present. The file had been descending at the head and ascending at the tail. Ownership moves: - New `LLM/frank-medium.md` is the locator for the retired Dankserver endpoint: era windows quoted from each era page's own banner, no numbers restated, and the two unresolved date conflicts recorded instead of silently resolved. `frank-archon.md` links to it instead of restating the pre-rename window. - The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone; `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor verdicts and cross-split spec decode, and the four links that pointed at the stubs were repointed at it or at the drafter page. - Authentik's three stub pages fold into `Authentik.md`, with the process detail and the outpost reference split out so the parent stays under the length hint. Verified corrections (host census + live probe, 2026-10-19): - The MXFP4 checkpoint two pages called the "on-disk rollback" is gone: /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on the era page that owns that payload; the swap page now links to it. - frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2 arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run census is kept once (155 requests) instead of twice (~120 and 155). - Fleet shapes re-confirmed against the live endpoints before touching any page: 3x200000 fp8 KV, 2x200704, 2x262144. Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
98 lines
4.3 KiB
Markdown
98 lines
4.3 KiB
Markdown
---
|
|
type: reference
|
|
title: FreeToken Builds
|
|
description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
|
|
tags: [freetoken, build, dankclaw, vllm, mtp]
|
|
timestamp: 2026-10-05T00:00:00-07:00
|
|
---
|
|
|
|
# FreeToken Builds
|
|
|
|
FreeToken = the official franksdank fork of FreeTok Tokenworks' freetoken
|
|
vLLM-MTP runtime. Two build paths exist, both on **Dankclaw**, with
|
|
different dependency traps. Both were verified end-to-end 2026-09-11/12.
|
|
|
|
## Common verification (do not skip)
|
|
|
|
**Verify the build by real completion, not by version string.** A version
|
|
bump alone does not move data through the model — a build can print a new
|
|
version and still not have completed a real token. Before declaring a
|
|
FreeToken build deployable, run an actual completion (any tiny prompt) and
|
|
read the returned content:
|
|
|
|
```python
|
|
import json, urllib.request
|
|
req = urllib.request.Request("http://<host>:<port>/v1/chat/completions",
|
|
json.dumps({"model": "frank-zealot", "messages": [{"role": "user",
|
|
"content": "Hi"}]}).encode(), {"Content-Type": "application/json"})
|
|
print(json.load(urllib.request.urlopen(req, timeout=600)))
|
|
```
|
|
|
|
A 200 with a real `choices[0].message.content` is the only build green.
|
|
|
|
## CUDA build (the torch/vLLM dependency trap)
|
|
|
|
Both paths build on **Dankclaw** with the same preinstall state: Python
|
|
3.12.10 in the system venv and **Torch 2.10.0.dev** (nightly, repo
|
|
download.pytorch.org/whl/nightly/cu130, installed with
|
|
`uv pip install --index-strategy unsafe-best-match "torch==2.10.0.dev<date>+cu130"
|
|
--index-url https://download.pytorch.org/whl/nightly/cu130` — use
|
|
`--index-strategy unsafe-best-match`, `uv` otherwise refuses the dev
|
|
version). Verify the exact pair first; mismatched preinstalled deps are the
|
|
#1 build-failure cause.
|
|
|
|
Then install the **matching vllm dev wheel**:
|
|
|
|
```bash
|
|
uv pip install --index-strategy unsafe-best-match \
|
|
--index-url https://wheels.vllm.ai/<commit-sha>/vllm-0.6.6.dev0+g<shortsha>.cu130-cp312-cp312-linux_x86_64.whl \
|
|
--extra-index-url https://wheels.vllm.ai/nightly/cu130
|
|
```
|
|
|
|
The wheel URL pattern is fixed:
|
|
`https://wheels.vllm.ai/<full-commit>/vllm-0.6.6.dev0+g<8-char>.cu130-cp312-cp312-linux_x86_64.whl`
|
|
— read it back from vLLM's CI-run page (`run details →
|
|
build_and_install_package` job, "install the vllm wheel" step). **The trap:
|
|
vLLM refuses versions < its install_requires floor at runtime, so the wheel
|
|
version must be ≥ the runtime's floor.** A wheel built at a lower base
|
|
(0.5.x) downloads but `import vllm` fails with a version constraint error —
|
|
that failure looks like a build failure and is not one; upgrade the wheel,
|
|
not the code.
|
|
|
|
If `uv pip install -e . --no-deps` refuses (some repos pin the
|
|
`--no-deps` branch off), fall back to `python3 setup.py` inside the venv,
|
|
after `python -m venv --copies .venv && source .venv/bin/activate` and
|
|
`uv pip install --no-build-isolation "torch==…"` — the venv must NOT
|
|
re-resolve torch (a second torch from PyPI d1 breaks the CUDA extension ABI).
|
|
|
|
## ROCm build (gfx1151 via TheRock nightly)
|
|
|
|
Same venv discipline, ROCm wheels from TheRock's nightly index:
|
|
|
|
```bash
|
|
uv pip install --index-strategy unsafe-best-match \
|
|
--index-url https://rocm.nightlies.amd.com/v2/gfx1151-dcgpu-supports_fbsgens_split_0717-rc1/ \
|
|
"torch==2.10.0.dev<date>+rocm" "triton==3.6.0" "pytorch-triton==3.6.0"
|
|
```
|
|
|
|
Then `vllm` itself (`uv pip install -e . --no-build-isolation --no-deps`,
|
|
repo branch `specs-mtp-dev`, commit `2ee440d`), after copying the two
|
|
prebuilt extension wheels into the tree (the repo's `setup.py` skips
|
|
compiling them when found). The ROCm build also needs
|
|
`install-fastrak-gfx1151.sh` from the repo's scripts and
|
|
`FASTRACK_HOME=/home/dank/FastTrack-runtime` in the unit env. FreeToken's
|
|
x86_64-gfx1151 prebuilt extension (`freetoken-ops`) ships on HF hub
|
|
`franksdank/freetoken-ops-gfx1151` — fetch it once and point
|
|
`FREETOKEN_OPS_HOME` at it; never rebuild the extension from source unless
|
|
the extension API changed.
|
|
|
|
`--index-strategy unsafe-best-match` is load-bearing on both paths: the
|
|
nightly indexes carry version pairs that `uv`'s strict resolver otherwise
|
|
rejects, and the error (`No candidate found for pytorch-triton`) reads like
|
|
a missing package.
|
|
|
|
## Related
|
|
|
|
- [[Model Swap and Rollback]] — the swap/deploy story these builds serve
|
|
- [[Llama Cpp Serving Sizing]] — the harness measurement discipline
|