The Swizzle Operator

GPU shared memory is divided into 32 banks. When multiple threads in a warp access different addresses in the same bank, the accesses are serialized (bank conflict).

The swizzle operator (XOR-based index transformation) remaps memory addresses so that threads in a warp access different banks, even for column-wise access patterns. This is particularly important for matrix transpose and GEMM operations.

The traditional workaround of padding (e.g., NxN+1 instead of NxN) wastes shared memory. Swizzling achieves conflict-free access without wasting memory.

References:

• NVIDIA - Shared Memory Bank Conflicts
• CUTLASS - Swizzle