78 lines
4.8 KiB
Markdown
78 lines
4.8 KiB
Markdown
---
|
|
name: ryzenai-venv-dependency-triangle
|
|
description: "Debug the torchao/torchtune/transformers import triangle inside an AMD Ryzen AI (ryzen_ai-*.tgz) venv under torch 2.6, including the masked \"cannot import name\" failures and the port-the-one-symbol escape."
|
|
---
|
|
|
|
# Ryzen AI venv dependency triangle (torch 2.6 era)
|
|
|
|
Symptom that starts it all — and it never names the culprit:
|
|
|
|
```
|
|
ImportError: cannot import name 'HubertModel' from 'transformers'
|
|
```
|
|
|
|
The named symbol is healthy. `import torchao` alone reproduces the failure; `import transformers` alone does not. transformers imports its **torchao integration** whenever the real modelling path loads, and the failure is re-raised two frames deep by the lazy loader as a missing name.
|
|
|
|
## Instruments
|
|
|
|
Never conclude from `grep -c Symbol transformers/__init__.py` — 4.5x generates its lazy `__init__`, so an absent literal proves nothing. Get the real exception:
|
|
|
|
```python
|
|
import traceback, torch, transformers
|
|
try:
|
|
__import__("transformers.models.hubert.modeling_hubert")
|
|
except Exception:
|
|
traceback.print_exc()
|
|
```
|
|
|
|
That prints the truth, e.g. `AttributeError: module 'torchao.quantization' has no attribute 'Float8WeightOnlyConfig'` from `transformers/modeling_layers.py`.
|
|
|
|
Also probe the order matrix (`import X; from transformers import Y` for each suspect): only the poisoned order fails, which isolates the culprit without reading vendor code.
|
|
|
|
## The triangle (measured 2026-10-19, ryzen_ai 1.8.0, torch 2.6.0+cpu, transformers 4.57.6)
|
|
|
|
| Option | Result |
|
|
|---|---|
|
|
| torchao absent | `from transformers import HubertModel` -> OK |
|
|
| torchao 0.7.0+cpu | `ImportError: Float8WeightOnlyConfig` (masked as missing symbol) |
|
|
| torchao 0.8.0+cpu | imports (warns `No module named 'triton'`), still lacks `Float8WeightOnlyConfig` |
|
|
| torchao >= 0.9 | has the symbol, requires torch >= 2.7 -> impossible in this venv |
|
|
| torchtune, ANY version (0.6.1 … 0.1.1) | guard `torchao not installed`, or `pyarrow requires NumPy 2.0 or newer, found 1.26.4` |
|
|
|
|
Two independent dead ends: torchtune's package init pulls `datasets` -> **the pyarrow AMD ships in its own venv refuses that venv's own numpy 1.26.4**. So `import torchtune` is structurally dead, and torchao (the only thing that satisfies its guard) breaks transformers. **Never "just upgrade" torch/numpy: the vendor's ONNX runtime and torch build are what the NPU EP is loaded against.**
|
|
|
|
## Escape: port the one symbol, pin both absent
|
|
|
|
Ask what the third-party package is actually used for before fighting the resolver. In `neucodec` it was one symbol (`RotaryPositionalEmbeddings`, ~40 lines, copied verbatim from `torchtune/modules/position_embeddings.py`, BSD-3) used by two files. Apply it in the image with a transform that **asserts** each import line exists, so a vendor update fails the build instead of shipping a half-patched package:
|
|
|
|
```dockerfile
|
|
COPY patches/rope_local.py /opt/patch/rope_local.py
|
|
RUN python - <<'PY'
|
|
import pathlib
|
|
from importlib.util import find_spec
|
|
pkg = pathlib.Path(find_spec("neucodec").origin).parent # find_spec: importing it is what is impossible yet
|
|
(pkg / "rope_local.py").write_text(pathlib.Path("/opt/patch/rope_local.py").read_text())
|
|
old, new = "from torchtune.modules import RotaryPositionalEmbeddings", "from .rope_local import RotaryPositionalEmbeddings"
|
|
for name in ("bs_roformer5.py", "codec_decoder_vocos.py"):
|
|
p = pkg / name; s = p.read_text()
|
|
assert old in s, name + ": import line moved - re-read the vendor source"
|
|
p.write_text(s.replace(old, new))
|
|
PY
|
|
```
|
|
|
|
Gate both absences at build time so nobody re-adds them and inherits a three-week-later mystery:
|
|
|
|
```dockerfile
|
|
RUN ... import-check \
|
|
&& python -c "import importlib.util as u; assert u.find_spec('torchao') is None; assert u.find_spec('torchtune') is None"
|
|
```
|
|
|
|
Prove a ported numeric kernel with the vendor's own fixtures, not with faith: `neucodec` ships `samples/jo.wav` *and* its own encoding `samples/jo.pt`, so re-encoding the WAV and comparing integer codes is a pass/fail correctness test of the RoPE port.
|
|
|
|
## Rules of thumb
|
|
|
|
- `--no-deps` for anything adjacent to torch/onnxruntime/numpy; check the vendor pins by reading them out of the venv (`python -c "import torch; print(torch.__version__)"`), never from docs.
|
|
- A `--dry-run` install that resolves without touching torch/numpy/onnxruntime is the licence to install *with* deps.
|
|
- `uv pip install --target DIR` / a nested venv does **not** layer over a parent venv: a venv created from `/opt/rai/bin/python` resolves "system site-packages" to the base interpreter, and uv's resolver ignores `.pth` paths and will pull a second numpy that breaks the vendor C extensions.
|
|
- Ephemeral `podman run --rm` probes discard everything you installed in them — build a throwaway tag (truncate the Containerfile before the failing step) instead of re-hypothesising.
|