PyTorch has introduced a new API called FlexAttention, designed to simplify the implementation of various attention mechanisms in machine learning models. This API allows users to implement diverse attention variants using a few lines of idiomatic PyTorch code. FlexAttention supports arbitrary masks and biases, exploits blockwise sparsity for speed improvements, and can express complex attention mechanisms such as Gemma2 soft-capping and neighborhood attention. Additionally, it enables the packing of mixed-size images or text into a single context. The API also accepts a user-defined function, score_mod, which processes the attention score and position of two tokens, enhancing flexibility for developers. This development aims to overcome the limitations of previous fused attention implementations, providing a more versatile tool for machine learning practitioners.