ARES: A Reading Comprehension Ensembling Service

Anthony Ferritto, Lin Pan, Rishav Chakravarti, Salim Roukos, Radu Florian, J William Murdock, Avi Sil

Demo Paper

Gather-3I: Nov 17, Gather-3I: Nov 17 (18:00-20:00 UTC) [Join Gather Meeting]

Abstract: We introduce ARES (A Reading Comprehension Ensembling Service): a novel Machine Reading Comprehension (MRC) demonstration system which utilizes an ensemble of models to increase F1 by 2.3 points. While many of the top leaderboard submissions in popular MRC benchmarks such as the Stanford Question Answering Dataset (SQuAD) and Natural Questions (NQ) use model ensembles, the accompanying papers do not publish their ensembling strategies. In this work, we detail and evaluate various ensembling strategies using the NQ dataset. ARES leverages the CFO (Chakravarti et al., 2019) and ReactJS distributed frameworks to provide a scalable interactive Question Answering experience that capitalizes on the agreement (or lack thereof) between models to improve the answer visualization experience.

Similar Papers

MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics
Anthony Chen, Gabriel Stanovsky, Sameer Singh, Matt Gardner,
A Simple and Effective Model for Answering Multi-span Questions
Elad Segal, Avia Efrat, Mor Shoham, Amir Globerson, Jonathan Berant,
NwQM: A neural quality assessment framework for Wikipedia
Bhanu Prakash Reddy Guda, Sasi Bhushan Seelaboyina, Soumya Sarkar, Animesh Mukherjee,