PyTorch Accelerated on AMD MI350 GPUs with FlyDSL
In brief
- FlyDSL accelerates matrix operations on AMD MI350 GPUs via PyTorch, with speedups from 1.10x to 4.82x depending on workloads.
- Attention and MXFP8/MXFP4 GEMM kernels use optimized pipelines to overlap computations and manage scales.
- Performance gains depend on data shapes and hardware setups, validated in vLLM end-to-end tests.
PyTorch now supports FlyDSL for AMD MI350-series GPUs, accelerating key matrix operations through existing APIs. According to the vendor, performance gains range from 1.10x to 4.82x depending on workloads, with up to 3.01x speedups in end-to-end vLLM tests.
FlyDSL introduces a *layout algebra* programming model to optimize hardware-specific kernels, including dense and grouped GEMM, fused multi-head attention, MXFP8/MXFP4 operations, and RMSNorm. Attention kernels use a dual-wave software pipeline to overlap computations, while MXFP GEMM kernels handle scales and accumulations explicitly.
This optimization targets AMD MI350 GPUs (gfx950 architecture) and complements existing implementations. Performance gains depend on data shapes and hardware configurations, with measurements conducted in isolated operator mode or end-to-end tests.
This summary gives you the essentials. The full article, its figures and quotes are on PyTorch Blog.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error