nystrom attention:Nyströmformer论文中的Nyström自我注意的实现 源码

Nyström注意 论文中的Nyström自我注意的。 安装 $ pip install nystrom-attention 用法 import torch from nystrom_attention import NystromAttention attn = NystromAttention ( dim = 512 , dim_head = 64 , heads = 8 , num_landmarks = 256 , # number of landmarks pinv_iterations = 6 , # number of moore-penrose iterations for approximating pinverse. 6 was recommended by the paper residual = True