Attention

publications

Publication Image
On the weak link between importance and prunability of attention heads

Aakriti Budhraja, Madhura Pande, Preksha Nema, Pratyush Kumar, Mitesh M. Khapra,

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics.

http://dx.doi.org/10.18653/v1/2020.emnlp-main.260
Publication Image
On the Importance of Local Information in Transformer Based Models

Madhura Pande, Aakriti Budhraja, Preksha Nema, Pratyush Kumar, Mitesh M. Khapra,

arXiv:2008.05828

https://arxiv.org/pdf/2008.05828.pdf
Publication Image
Towards Transparent and Explainable Attention Models

Akash Kumar Mohankumar, Preksha Nema, Sharan Narasimhan, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran,

In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.

https://doi.org/10.18653/v1/2020.acl-main.387
Publication Image
Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources

Janarthanan Rajendran, Aravind Lakshminarayanan, Mitesh M Khapra, B Ravindran, others,

arXiv preprint arXiv:1510.02879