VoltgridNews

Microsoft RDMA over USB4: 28-50 µs Latency Links Windows PCs for AI

Microsoft's RDMA over USB4 developer preview splits one AI model across 3 to 4 Windows PCs at 28-50 µs round-trip latency, about 4x lower than Ethernet over the same link. The Insider build has no date yet.

Chips & Power Desk

2 min read

AMD Ryzen AI Max Series processor package, the chip family in Microsoft's RDMA over USB4 mini-PC tests

AMD's Ryzen AI Max Series processor, the chip in Microsoft's three-PC test cluster. Product image; it does not show the USB4 setup.

Image: AMD

Source · Press image from the AMD product page, used for news reporting.

Microsoft announced a developer preview of RDMA over USB4 for Windows on Nov. 19, 2026, measuring 28-50 microseconds of round-trip latency for small messages between two PCs, roughly 4x lower than Ethernet (UDP) over the same USB4 link, according to Microsoft's USB blog.

RDMA, or Remote Direct Memory Access, moves data straight between the memory of two machines and skips most of the network stack. Microsoft says it exposes the feature through NetworkDirect, the API Windows already uses with datacenter RDMA adapters, so there is no RDMA network card, switch or third-party driver to add. TechPowerUp, which reported the preview on Nov. 20, cites 116-194 microseconds for the UDP baseline, which matches the roughly fourfold gap.

Microsoft's vendor-claimed small-message latency over USB4 is roughly 4x lower with RDMA than with Ethernet (UDP).

Graphic: Voltgrid News

The point for local AI is memory. Microsoft says a model split across PCs with tensor parallelism makes the machines exchange data many times per generated token, so latency matters more than bandwidth. It reports the same low latency at 40 Gbps and 80 Gbps, with 80 Gbps giving twice the bandwidth.

Tested hardware and speeds

In Microsoft's lab, three mini-PCs with AMD Ryzen AI Max processors, Radeon 8060S graphics and 128 GB of unified memory each ran Qwen3-235B-A22B, a 235-billion-parameter mixture-of-experts model, at about 18.5 tokens per second with 4-bit quantization. That is 384 GB of memory across the cluster, on a 40 Gbps full mesh. AMD's own three-PC run of the same model measured 21.98 tokens per second decode and 117.98 tokens per second prompt processing.

Intel measured GPT-OSS-120B, a roughly 120-billion-parameter model that does not fit in one 64 GB PC, on Core Ultra Series 3 machines with Thunderbolt 4 and 64 GB each. At a 1K-token context, three PCs reached 49.4 tokens per second and four reached 54.2, about 10% more for a fourth machine. At 2K context the four-PC figure fell to 51.7. These are vendor measurements on Microsoft's preview.

What you need, and what is missing

Each PC needs N-1 USB4 ports to join an N-PC cluster, so a four-PC mesh takes three ports per machine and six cables. Microsoft says not to daisy-chain: performance drops sharply. TechPowerUp's headline says daisy-chained PCs, but Microsoft's own post describes a direct full mesh and lists daisy chains as a known limitation.

The feature is off by default. Turning it on needs a registry key and test signing, which requires Secure Boot to be off, and Microsoft tells testers to use dedicated machines. The transport does not retry dropped connections. The NetworkDirect provider and a llama.cpp-based testbed are open source now, but the Windows Insider build in the Experimental channel (Future Platforms) is only listed as coming soon.

For power and capacity planning, this is a desk-scale way to pool unified memory, not a datacenter fabric. It raises the ceiling for models of 120B-235B parameters on mini-PCs without server-class networking, and it ties the cluster size to USB4 port counts.

What is not confirmed yet

Microsoft has not given a date for the Insider build or for a general Windows release, and says only that it is exploring more PCs and silicon partners. It says latency is a transport measurement, and results depend on model, framework build and each platform's memory and GPU. The 18.5 tokens per second figure is Microsoft's; the 21.98 and 49-54 figures come from AMD and Intel. No independent tests have been published.

Sources

Published
Sources
2 cited