Welcome to VATEX Captioning Challenge!


Overview


We are pleased to announce the VATEX Captioning Challenge 2019! The challenge will be hosted at the 3rd Workshop on Closing the Loop Between Vision and Language, ICCV 2019.

Please stay tuned for more information!


Dates


  • Challenge Launch: Aug 6th, 2019.
  • Submission Deadline: Oct 1st, 2019.
  • The winners will be announced at the 3rd CLVL workshop, ICCV 2019 on Oct 28th, 2019.

Prize


We will have several prizes and rewards for the winning teams! Details will be released soon.


Guidelines


The VATEX dataset is a new large-scale multilingual video description dataset, which contains over 41,250 videos and 825,000 captions in both English and Chinese. Among the captions, there are over 206,000 English-Chinese parallel translation pairs. Compared to the widely-used MSRVTT dataset, VATEX is multilingual, larger, linguistically complex, and more diverse in terms of both video and natural language descriptions. Please refer to our ICCV paper for more details. This VATEX Captioning Challenge aims to benchmark progress towards models that can describe the videos in various languages such as English and Chinese.

Dataset Download

Please refer to the details at the Download page. You can download English/Chinese captions and video features from the page.


Submission

The challenge is hosted at the CodaLab. Please go to the challenge page to submit your models.


Challenge Phases
  • Dev (English/Chinese): This phase evaluates algorithms on the VATEX validation set. We recommend using this Dev phase for algorithm validation. This phase should not be used for reporting results in the paper. A submission needs to consist of results on the entire validation set to be considered as a valid submission.
  • Test (English/Chinese): This phase evaluates algorithms on VATEX public test set. We recommend using this phase for reporting comparison numbers in academic papers. A submission needs to consist of results on the entire test set to be considered as a valid submission. This phase is aimed at the final evaluation of the model and one is not allowed to create multiple submissions using multiple teams.

Leaderboard