BERT

blogs

blog-image
The Heads Hypothesis: A Unifying Statistical Approach Towards Understanding Multi-Headed Attention in BERT
BERT has been a top contender in the space of NLP models. With its sucess, a parallel stream of research, named BERTology, has emerged, that tries to understand how does BERT work so well. With the similar objective, this blog explains a novel technique for analysing the behaviour of BERT’s attention heads. Read this for interesting insights which unravel the inner working of BERT!
By Madhura Pande
• Sep 3, 2021
Know More

publications

Publication Image
Perturbation CheckLists for Evaluating NLG Evaluation Metrics

Ananya B. Sai , Tanay Dixit , Dev Yashpal Sheth & more

In Proceedings of Empirical Methods in Natural Language Processing (EMNLP 2021)

https://arxiv.org/abs/2109.05771
Publication Image
The heads hypothesis: A unifying statistical approach towards understanding multi-headed attention in BERT

Madhura Pande , Aakriti Budhraja , Preksha Nema & more

In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, No. 15, pp. 13613-13621)

https://ojs.aaai.org/index.php/AAAI/article/view/17605
Publication Image
Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining

Ananya B. Sai , Akash Kumar Mohankumar , Siddhartha Arora and Mitesh M. Khapra

Transactions of the Association for Computational Linguistics

https://doi.org/10.1162/tacl_a_00347
Publication Image
On the weak link between importance and prunability of attention heads

Aakriti Budhraja , Madhura Pande , Preksha Nema & more

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics.

http://dx.doi.org/10.18653/v1/2020.emnlp-main.260
Publication Image
Towards Interpreting BERT for Reading Comprehension Based QA

Sahana Ramnath , Preksha Nema , Deep Sahni & more

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.

http://dx.doi.org/10.18653/v1/2020.emnlp-main.261
Publication Image
On Incorporating Structural Information to improve Dialogue Response Generation

Nikita Moghe , Priyesh Vijayan , Balaraman Ravindran & more

arXiv:2005.14315

https://arxiv.org/pdf/2005.14315.pdf