---
name: pcie-link-degradation-vs-idle-triage
description: "Decide whether a GPU/NVMe PCIe link is genuinely degraded (bad riser, bad slot, wrong connector) or just idle-downshifted, and whether moving the card to another slot could widen it — using kernel boot bandwidth lines, port LnkCap vs LnkSta, and NVML sampled under load."
---

# PCIe link: degraded or just idle?

Answer "is my GPU running at full speed?" without fooling yourself. The trap: an
idle NVIDIA GPU reports `LnkSta: Speed 2.5GT/s (downgraded)` because the driver
downshifts the link to Gen1 for power. That is not a fault, and it is not the
slot's fault either.

## Step 1 — the path ceiling, from the kernel (cheapest, most decisive)

```bash
sudo journalctl -k -b --no-pager | grep "available PCIe bandwidth"
```

Reads out per endpoint, e.g.:

```
pci 0000:c4:00.0: 63.012 Gb/s available PCIe bandwidth, limited by 16.0 GT/s
PCIe x4 link at 0000:00:03.1 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
```

- `63.012 Gb/s` = Gen4 **x4** (~7.88 GB/s/direction after 128b/130b encoding).
- "limited by ... link at <BDF>" names the **bottleneck device** (usually a root
  port) and its width — this is the number that decides whether a swap can help.
- "capable of ... x16 link" is the **endpoint's own** capability, not the path's.
  A big gap between the two is the whole story.

`dmesg` is often rotated; `journalctl -k -b` keeps this boot.

## Step 2 — the two sides separately

```bash
sudo lspci -vv -s <endpoint> | grep -E "LnkCap:|LnkSta:|LnkCtl2:|LnkCap2:"
sudo lspci -vv -s <root port> | grep -E "LnkCap:|LnkSta:|LnkCtl2:"
```

- `LnkSta` on a *bridge* is the trained link; on an endpoint it is that
  endpoint's view. Widths/`(downgraded)` compare against the **own** `LnkCap`.
- `LnkCtl2: Target Link Speed: 16GT/s` on both ends = the platform *intends*
  Gen4, so a Gen1 current speed is power management, not misconfiguration.
- A root port whose `LnkCap` width exceeds the trained width means the socket or
  adapter wires fewer lanes than the controller can drive. **M.2 connectors are
  4 lanes by pinout** — an M.2 riser can never deliver more, regardless of what
  the bridge advertises.

## Step 3 — prove the link under load (the step people skip)

Never conclude from an idle reading on NVIDIA. Sample NVML while the GPU works:

```bash
nvidia-smi --query-gpu=pcie.link.gen.current,pcie.link.gen.max,\
pcie.link.gen.hostmax,pcie.link.width.current,pcie.link.width.max --format=csv
```

Fire a small, legitimate request at whatever already serves on that GPU (e.g. a
`max_tokens` completion against the local LLM endpoint) and poll the query while
utilization is high:

- `gen.current` climbing 1 → `hostmax` = idle downshift, link healthy.
- `gen.current` stuck below `hostmax` under load = real degradation.
- `width.current < width.max` under load = **physical lane limit** (wiring,
  riser, or connector) — no software touches this.
- On a busy production GPU, this is a usage smoke test, not a benchmark: never
  stop/bench a live service to read its link.

## Step 4 — do not plan hardware from `dmidecode -t slot`

On consumer/mini-PC firmware this table is routinely fiction — it has been seen
labelling an Ethernet port and an SD-reader controller as "M.2 Socket 3" slots
and advertising a free slot in a **PCI domain that does not exist** (e.g.
`0200:00:03.2`). Cross-check with the live bridge list:

```bash
lspci -nn | grep "PCI bridge"    # every root port, whether populated or empty
lspci -t                          # topology; which endpoints hang off which port
```

Empty root ports still enumerate, so a genuinely absent port is genuinely
absent. Trust live PCI, not SMBIOS.

## Step 5 — verdict on "would moving the card to another slot help?"

- Compare the **bottleneck device named in step 1** with the target slot's port
  `LnkCap` width. If the bottleneck is the *endpoint side* (riser/adapter) and
  the target port is wider, a move can help.
- If both candidate connectors cap at the same width (two M.2 sockets = both
  x4), moving is a **no-op** — say so plainly instead of endorsing the swap.
- Error counters are a useful tiebreaker: clean `DevSta`/`CESta` with a low
  trained speed means "wired narrow", not "failing".

## Reporting

Give the ceiling in Gb/s, the card's own capability, what the idle reading
meant, and whether any physical change can move the number. If the answer is
"the link is already at the hardware maximum", state that and quantify whether
the workload actually cares (resident weights = PCIe is nearly idle per token;
CPU layer offload = it hurts).