SemEval 2022 Task2: Data, Evaluation Scripts and Submission format

This repository contains Data and Evaluation scripts for SemEval 2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding

Please see that Task Description Website (https://sites.google.com/view/semeval2022task2-idiomaticity) for a detailed description of each Subtask.

Training Data

This repository contains the Training Data, Evaluation Scripts and Submission format for:

1. Subtask A
2. Subtask B

Data for each Subtask consists of three data splits: "Training" (except in the pre-train setting), "dev" and "eval". You are provided with the gold labels associated with the dev set and the evaluation script. You are NOT provided with the gold labels associated with the evaluation set. You are required to submit these results to corresponding codalab task for evaluation.

IMPORTANT: While we provide an "eval" split, this must not be confused with the "test" split that will be released in January.

Trial Data

This repository also contains the Trial Data (aimed at providing participants with a clearer understanding of what to expect). It is a (very small) subset of the training data.