frank-primary: never write the stage index through the raw symlink

Run 3 rewrote the staged index THROUGH the relative symlink into the raw
checkpoint: the raw index.json was restamped with the stacked layout
(605 KB smaller), the raw verify then failed 143/144 and the overnight
run died before stacking. Two fixes:

- stack-qwen38-official-fp8.py: index/config writes are now temp+os.replace
  (an atomic swap replaces the symlink itself; the raw tree stays pristine).
- build-frank-primary-rad.sh: the stage's index.json is COPIED like
  config.json; only shards stay relative symlinks.
This commit is contained in:
Frank committed 2026-10-06 00:03:58 -07:00
1 parent 90a08ab4b5
commit fb1a4fd150
2 files changed
+18 -8

No files matched your search

+10 -5
View File
@@ -424,12 +424,17 @@ def main():
index["weight_map"] = new_wm
if "metadata" in index and isinstance(index["metadata"], dict):
index["metadata"]["total_size"] = total
with open(index_path, "w") as fh:
json.dump(index, fh, indent=2)
# ATOMIC REPLACE, never open(path, "w"): the stage's index is a symlink
# to the RAW checkpoint (staging symlinks everything but config.json) —
# open("w") would rewrite the raw file through the link. os.replace
# swaps the link itself for a real file; the raw tree stays pristine.
for path, payload in ((index_path, json.dumps(index, indent=2)),
(cfg_path, json.dumps(cfg, indent=2))):
tmp = path + ".tmp"
with open(tmp, "w") as fh:
fh.write(payload)
os.replace(tmp, path)
n = set_split(cfg, 1)
with open(cfg_path, "w") as fh:
json.dump(cfg, fh, indent=2)
log(f"config: split_ngram_parts=1 written at {n} location(s)")
with open(os.path.join(stage, "ple_scale.txt"), "w") as fh: