NLP

blogs

The Heads Hypothesis: A Unifying Statistical Approach Towards Understanding Multi-Headed Attention in BERT

BERT has been a top contender in the space of NLP models. With its sucess, a parallel stream of research, named BERTology, has emerged, that tries to understand how does BERT work so well. With the similar objective, this blog explains a novel technique for analysing the behaviour of BERT’s attention heads. Read this for interesting insights which unravel the inner working of BERT!

By Madhura Pande

• Sep 3, 2021

Know More

Levelling up NLP for Indian Languages

We are working towards building a better ecosystem for Indian languages while also keeping up with the recent advancements in NLP. To this end, we are releasing IndicNLPSuite, which is a collection of various resources and models for Indian languages

By Divyanshu Kakwani , By Anoop Kunchukuttan

• Feb 10, 2021

Know More

people

Post Baccalaureate Fellow

Specialization:

Know More

PhD Scholar

Specialization:

Deep learning, NLP, Artificial metabolic pathway design

Know More

PhD scholar

Specialization:

ML/DL, NLP, BioInformatics

Know More

PhD Scholar

Specialization:

NLP, Deep learning

Know More

PhD Scholar

Specialization:

Theoretical Machine Learning, Deep Learning, Gaussian Processes, Computer Vision, NLP

Know More

publications

A Survey of Adversarial Defences and Robustness in NLP

Shreya Goyal, Sumanth Doddapaneni, Mitesh M. Khapra, Balaraman Ravindran,

ACM Computing Surveys

Towards Building ASR Systems for the Next Billion Users

Tahir Javed, Sumanth Doddapaneni, Abhigyan Raman, Kaushal Santosh Bhogale, Gowtham Ramesh, Anoop Kunchukuttan, Pratyush Kumar, Mitesh M. Khapra,

To appear in the Proceedings of the Thirty-Sixth Annual Conference of the Association for Advancement of Artificial Intelligence (AAAI 2022)

https://arxiv.org/abs/2111.03945

Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages

Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Mahalakshmi J, Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan, Pratyush Kumar, Mitesh Shantadevi Khapra,

Transactions of the Association for Computational Linguistics

https://doi.org/10.1162/tacl_a_00452

Perturbation CheckLists for Evaluating NLG Evaluation Metrics

Ananya B. Sai, Tanay Dixit, Dev Yashpal Sheth, Sreyas Mohan, Mitesh M. Khapra,

In Proceedings of Empirical Methods in Natural Language Processing (EMNLP 2021)

https://arxiv.org/abs/2109.05771

Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining

Ananya B. Sai, Akash Kumar Mohankumar, Siddhartha Arora and Mitesh M. Khapra,

Transactions of the Association for Computational Linguistics

https://doi.org/10.1162/tacl_a_00347

IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages

Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, Pratyush Kumar,

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings.

https://doi.org/10.18653/v1/2020.findings-emnlp.445

On the weak link between importance and prunability of attention heads

Aakriti Budhraja, Madhura Pande, Preksha Nema, Pratyush Kumar, Mitesh M. Khapra,

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics.

http://dx.doi.org/10.18653/v1/2020.emnlp-main.260

Towards Interpreting BERT for Reading Comprehension Based QA

Sahana Ramnath, Preksha Nema, Deep Sahni, Mitesh M. Khapra,

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.

http://dx.doi.org/10.18653/v1/2020.emnlp-main.261

A Survey of Evaluation Metrics Used for NLG Systems

Ananya B. Sai, Akash Kumar Mohankumar, Mitesh M. Khapra,

ACM Computing Surveys

https://arxiv.org/pdf/2008.12009

On the Importance of Local Information in Transformer Based Models

Madhura Pande, Aakriti Budhraja, Preksha Nema, Pratyush Kumar, Mitesh M. Khapra,

arXiv:2008.05828

https://arxiv.org/pdf/2008.05828.pdf

Incorporating prior knowledge about structural constraints in model identification

Deepak Maurya, Sivadurgaprasad Chinta, Abhishek Sivaram, Raghunathan Rengaswamy,

arXiv:2007.04030

https://arxiv.org/pdf/2007.04030.pdf

On Incorporating Structural Information to improve Dialogue Response Generation

Nikita Moghe, Priyesh Vijayan, Balaraman Ravindran, Mitesh M. Khapra,

arXiv:2005.14315

https://arxiv.org/pdf/2005.14315.pdf

Towards Transparent and Explainable Attention Models

Akash Kumar Mohankumar, Preksha Nema, Sharan Narasimhan, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran,

In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.

https://doi.org/10.18653/v1/2020.acl-main.387

Let's Ask Again: Refine Network for Automatic Question Generation

Preksha Nema, Akash Kumar Mohankumar, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran,

In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing

https://doi.org/10.18653/v1/D19-1326

NLP

blogs

The Heads Hypothesis: A Unifying Statistical Approach Towards Understanding Multi-Headed Attention in BERT

Levelling up NLP for Indian Languages

people

Adhil Ahmed P M Shums

Post Baccalaureate Fellow

Roshan M S B

PhD Scholar

Nency Bansal

PhD scholar

Ananya Sai

PhD Scholar

Rahul Vashisht

PhD Scholar

publications