Shared task participation

For organization of shared tasks, please read (/docs/shared-task-organize/))

Shared task participation

A user can participate in shared tasks in following way:

1. To find shared tasks

  • Up to 10 recently updated shared tasks are listed in the front page of PubAnnotation.
  • By clicking the heading Shared Tasks, the full list of shared tasks can be accessed.
  • Note that, in PubAnnotation, a shared task is a collection which includes multiple projects.

2. To obtain data sets

  • The data set in each project should be downloadable in the Annotations section. Otherwise contact the maintainer of the shared task.

3. To submit predicted annotation

  1. create a project (creating annotations),
  2. import texts from a project, either development or evaluation data set, of the shared task (importing documents), and
  3. add your predicted annotation to the texts (adding annotations).

4. To evaluate your prediction

  • compare the project with your predicted annotation against the corresponding project, either development or evaluation data set, of the shared task (comparing projects).

5. To make an entry to the task

TBD