Attention

On the weak link between importance and prunability of attention heads
On the Importance of Local Information in Transformer Based Models
Towards Transparent and Explainable Attention Models
Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources