Video: Dr Sheila Castilho - Context-aware MT evaluation: what have we learned?
Watch the full video recording of the lecture Context-aware MT evaluation: what have we learned?, by Dr Sheila Castilho. The lecture was delivered on the 1st of March 2023 as part of the Convergence lecture series promoted by the Centre for Translation Studies, University of Surrey.
Title: Context-aware MT evaluation: what have we learned?
Abstract: The challenge of evaluating translations in context has been raising interest in the machine translation (MT) field. However, the definition of what constitutes a document-level (doc-level) MT evaluation, in terms of how much of the text needs to be shown, is still unclear (Castilho et al., 2020). Few works have taken into account doc-level human evaluation (Barrault et al., 2020), and one common practice is the usage of test suites with context-aware markers. But why do we need context in MT evaluation? Document-level evaluation of MT allows for a more thorough examination of the output quality with context, avoiding common cases of misevaluations. The main objective of the DELA Project is to define best practices for doc-level MT evaluation, and test the existing human and automatic sentence-level evaluation metrics to the doc-level. In this talk, we will look into the results of the project regarding methodologies for adding context in MT evaluation tasks, a document-level corpus annotated with context-related issues, and how much context span is enough to solve those issues.
Short bio: Sheila Castilho is an Assistant professor in SALIS at Dublin City University. She graduated in Linguistics and Education from the UNIOESTE University in Brazil, and holds a joint Master in Natural Language Processing from the University of Wolverhampton, UK and the University of Algarve, Portugal. She completed her PhD dissertation at Dublin City University in 2016. Previously, she was an Irish Research Council Research Fellow at the Adapt Centre working on the DELA Project, which aimed to test the existing human and automatic sentence-level metrics to the document-level and define best practices for document-level machine translation evaluation. Her research interests include machine translation, post-editing, machine and human translation evaluation, document-level machine translation, usability, and translation technologies.
This event took place online on Wednesday 1st March 2023, 3.30pm GMT.