COIN: COmmonsense INference in Natural Language Processing

Workshop to be held in conjunction with EMNLP-IJCNLP in Hong Kong

November 3, 2019

Research in natural language understanding and textual inference has advanced considerably in recent years, resulting in powerful models that are able to read and understand texts, even outperforming humans in some cases. However, it remains challenging to answer questions that go beyond the texts themselves, requiring the use of additional commonsense knowledge. Previous work has explored using both explicit representations of background knowledge (e.g., ConceptNet or NELL), and latent representations that capture some aspects of commonsense (e.g., OpenAI GPT). These and any other methods for representing and using commonsense in NLP are of interest to this workshop.

The COIN workshop aims at bringing together researchers that are interested in modeling commonsense knowledge, developing computational models thereof, and applying commonsense inference methods in NLP tasks. We are interested in any type of commonsense knowledge representation, and explicitly encourage work that makes use of knowledge bases and approaches developed to mine or learn commonsense from other sources. The workshop is also open for evaluation proposals that explore new ways of evaluating methods of commonsense inference, going beyond established natural language processing tasks.

The workshop will also include two shared tasks on common-sense machine reading comprehension in English, one based on everyday scenarios and one based on news events. See Shared Tasks for more details.

If you are participating or interested in participating in COIN, we welcome you to join the COIN mailing list on Google Groups. Follow the link and click "Join Group" to join.

This workshop includes two shared tasks on English reading comprehension using commonsense knowledge. The first task is a multiple choice reading comprehension task on everyday narrations. The second task is a cloze task on news texts.

In contrast to other machine comprehension tasks and workshops, our focus will be on the inferences over commonsense knowledge about events and participants that are required for text understanding. Participants are encouraged to use any external resources that could improve their systems. Below we give a list of external resources that we expect to be helpful for the tasks.

If you make submissions to one of the shared tasks, they will be added to the development data leaderboard. The test data for both tasks will not be public, but you will have to submit your models so that we can run them on the test data. During the evaluation pahse (first 3 weeks of June), your submissions will count towards the final ranking on the test data. The final leaderboard will be made public only after the evaluation phase ends.

The development set leaderboard will be updated approx. once a week with all current submissions.

If you want to participate or have any questions, please join the google group for participants. We'll post updates in the group, and answer questions on the shared task and workshop.

Shared Task 1: Commonsense inference in everyday narrations

Shared Task 2: Commonsense inference in news articles

Commonsense Knowledge Resources

Commonsense Knowledge Bases

Script Knowledge Bases

Script Knowledge Representations

All deadlines refer to 11:59pm GMT -12 hours ("anywhere in the world").

Program Committee