Convergence lecture series: "Context-aware MT evaluation: what have we learned?", by Dr Sheila Castilho
Following the great success of the Convergence conference 2023, the Centre for Translation Studies (CTS) has the pleasure to announce the next event in the Convergence lecture series. Dr Sheila Castilho will deliver the online lecture Context-aware MT evaluation: what have we learned?. The event will take place online on Wednesday 1st March 2023, 3.30p
Title: Context-aware MT evaluation: what have we learned?
Abstract: The challenge of evaluating translations in context has been raising interest in the machine translation (MT) field. However, the definition of what constitutes a document-level (doc-level) MT evaluation, in terms of how much of the text needs to be shown, is still unclear (Castilho et al., 2020). Few works have taken into account doc-level human evaluation (Barrault et al., 2020), and one common practice is the usage of test suites with context-aware markers. But why do we need context in MT evaluation? Document-level evaluation of MT allows for a more thorough examination of the output quality with context, avoiding common cases of misevaluations. The main objective of the DELA Project is to define best practices for doc-level MT evaluation, and test the existing human and automatic sentence-level evaluation metrics to the doc-level. In this talk, we will look into the results of the project regarding methodologies for adding context in MT evaluation tasks, a document-level corpus annotated with context-related issues, and how much context span is enough to solve those issues.
Short bio: Sheila Castilho is an Assistant professor in SALIS at Dublin City University. She graduated in Linguistics and Education from the UNIOESTE University in Brazil, and holds a joint Master in Natural Language Processing from the University of Wolverhampton, UK and the University of Algarve, Portugal. She completed her PhD dissertation at Dublin City University in 2016. Previously, she was an Irish Research Council Research Fellow at the Adapt Centre working on the DELA Project, which aimed to test the existing human and automatic sentence-level metrics to the document-level and define best practices for document-level machine translation evaluation. Her research interests include machine translation, post-editing, machine and human translation evaluation, document-level machine translation, usability, and translation technologies.
Event details: The event will take place online on Wednesday 1st March 2023, 3.30pm GMT / UK time. Registration is free and can be done on this Eventbrite webpage.