Reasoning about Goals, Steps, and Temporal Ordering with WikiHow

Li Zhang, Qing Lyu, Chris Callison-Burch

Semantics: Sentence-level Semantics, Textual Inference and Other areas Short Paper

Gather-3H: Nov 17, Gather-3H: Nov 17 (18:00-20:00 UTC) [Join Gather Meeting]

You can open the pre-recorded video in a separate window.

Abstract: We propose a suite of reasoning tasks on two types of relations between procedural events: goal-step relations (“learn poses” is a step in the larger goal of “doing yoga”) and step-step temporal relations (“buy a yoga mat” typically precedes “learn poses”). We introduce a dataset targeting these two relations based on wikiHow, a website of instructional how-to articles. Our human-validated test set serves as a reliable benchmark for common-sense inference, with a gap of about 10% to 20% between the performance of state-of-the-art transformer models and human performance. Our automatically-generated training set allows models to effectively transfer to out-of-domain tasks requiring knowledge of procedural events, with greatly improved performances on SWAG, Snips, and Story Cloze Test in zero- and few-shot settings.
NOTE: Video may display a random order of authors. Correct author list is at the top of this page.

Connected Papers in EMNLP2020

Similar Papers

Learning from Task Descriptions
Orion Weller, Nicholas Lourie, Matt Gardner, Matthew Peters,
Towards Interpretable Reasoning over Paragraph Effects in Situation
Mucheng Ren, Xiubo Geng, Tao Qin, Heyan Huang, Daxin Jiang,
PRover: Proof Generation for Interpretable Reasoning over Rules
Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava, Mohit Bansal,