Iris Eskhol(LLL) and Carole Etienne (ICAR) organised a workshop on Inter-Annotator agreement in Paris on June, 26th, 2018
Invited talks by: Karën Fort, Jean-Yves Antoine, Yann Mathet
SegCor: Inter-annotator agreement for annotations based on both categorization and boundaries
Our workshop program:
- Iris Eshkol-Taravella & Carole Etienne: « A short presentation of the specificities of SegCor annotations and our expectations as regard Inter-annotator agreement »
- Karën Fort & Yann Mathet: « The good, the bad and the metrics: understanding inter-annotator agreement »
- Jean-Yves Antoine: « An experimental approach to give a meaning to kappa and alpha inter-annotator agreement measures »
- Thomas Schmidt (IDS Mannheim): « A crowdsourcing annotation experiment to segment oral data using pauses »
- Discussions applied to SegCor data
As several annotators proceeded to manual segmentation tasks, we get interested in inter-annotator agreement even if most part of the time we only have two or three annotators due to the time allowed for manual annotations before segmentation