Accueil Accueil

Tralogy 2011
Session 5-How can we measure machine translation quality ?, (3580) - Vendredi 04 mars 2011 11:50 - 12:10

Session 5-How can we measure machine translation quality ?,

In this opinion paper, we describe our research work on machine translation evaluation approaches that include mechanisms for human feedback and are designed to allow partial adaptation of the translation models which are being evaluated. While there exists a plethora of different automatic evaluation metrics for machine translation, their output in terms of scores, distances, etc. quite often is neither transparent to translators nor shows good correlation with manual evaluation by human experts.
Even worse, machine translation tuning efforts based on these automatic metrics to a certain extent move the research focus into a wrong direction; shifting it from « good » translations to those with a « higher scores ». This further widens the gap between machine translation research and translation producers or users.
We first describe several automatic metrics which are being used in current machine translation research. Afterwards we provide a brief overview on manual evaluation techniques which are used in our machine translation group. As minimum error rate training for tuning of (statistical) machine translation system is an important part of the workflow, we think that a (semi-) automatic implementation of such evaluation tasks would be a helpful extension of current state-of-the-art machine translation systems. We conclude by describing the need to shift from automated metrics to consumer-oriented, semi-automatic evaluation as this seems to be highly important to allow more advanced MT techniques to see wider acceptance and usage in real life applications.

C. Federmann, Deutsches Forschungszentrum fuer Kuenstliche Intelligenz

Sciences de l'Homme et de la Société Sciences de l'ingénieur Communication

Toutes les vidéos de l'évenement Tralogy 2011 (46 videos)

00:01:29.44

2011-03-03

09:00 - 09:35 : Mot de bienvenue, G. Sentise

00:09:30.80

2011-03-03

09:35 - 09:45 : Introduction, A. Kowalska

00:22:20.44

2011-03-03

09:45 - 10:10 : Machine Translation at the European Commission : language tools a...

00:20:06.12

2011-03-03

10:10 - 10:35 : Machine Translators

00:26:26.96

2011-03-03

10:35 - 11:00 : Language technologies in European research & innovation progr...

00:20:06.12

2011-03-03

11:00 - 11:15 : Session 1-Terminologie et traduction, une complémentarité oubli...

00:10:52.12

2011-03-03

11:15 - 11:30 : Session 1-User-centred Views on Terminology Extraction Tools : Us...

00:12:38.92

2011-03-03

11:30 - 11:45 : Session 1- Future Technologies for the Legal Translator : the Law...

00:12:44.32

2011-03-03

11:45 - 12:00 : Session 1-La Traduction assistée par ordinateur dans le contexte...

00:09:08.83

2011-03-03

12:00 - 12:15 : Discussion Session 1 tralogy 2011

00:20:13.28

2011-03-03

14:00 - 14:20 : Session 2-Aspects humains des technologies langagières dans l’...

00:12:55.48

2011-03-03

14:20 - 14:40 : Session 2-La technologie au service de la traduction

00:12:08.52

2011-03-03

15:00 - 15:20 : Session2-On the Systematicity of Human Translation Processes

00:13:58.44

2011-03-03

15:20 - 15:40 : Session 2-La post-édition à la portée du traducteur

00:13:08.88

2011-03-03

15:40 - 16:00 : Session 2-Face à la nouvelle donne : l’émergence d’un tradu...

00:20:29.23

2011-03-03

16:00 - 16:30 : Discussion session 2 tralogy 2011

00:22:19.20

2011-03-03

16:30 - 16:50 : Session 3-Language technologies and translators’ training - con...

00:19:46.40

2011-03-03

16:50 - 17:10 : Session 3-La formation du traducteur au Canada : espoirs et réal...

00:12:48.36

2011-03-03

17:10 - 17:30 : Session 3-Creating Blended Resources for Translator Training

00:10:34.64

2011-03-03

17:30 - 17:50 : Session 3 -The Internet, Google Translate and Google Translator T...

00:13:53.68

2011-03-03

17:50 - 18:10 : Session 3-Corpora, online resources and technology in training tr...

00:15:01.88

2011-03-03

18:10 - 18:30 : Session 3-Translation and technology in a project-based learning ...

00:30:48.80

2011-03-03

18:30 - 19:00 : Discussion session 3 Tralogy 2011

00:26:07.76

2011-03-04

09:00 - 09:20 : Session 4-Spoken Language Translation

00:18:40.76

2011-03-04

09:20 - 09:40 : Session 4-The Translator’s Workstations revisited : A new parad...

00:07:05.84

2011-03-04

09:40 - 10:00 : Session 4-What is web-based machine translation up to ?

00:10:53.32

2011-03-04

10:00 - 10:20 : Session 4-Translation 2.0 : facing the challenges of the global e...

00:10:24.32

2011-03-04

10:20 - 10:40 : Session 4-Traduction et informatique dématérialisée : une réa...

00:12:09.76

2011-03-04

10:40 - 11:00 : Session 4-Speech Recognition, Machine Translation and Gesture Loc...

00:28:16.30

2011-03-04

11:00 - 11:15 : Discussion session 4 Tralogy 2011

00:25:03.72

2011-03-04

11:30 - 11:50 : Session 5-What is a better translation ? Reflections on six years...

00:11:28.08

2011-03-04

11:50 - 12:10 : Session 5-How can we measure machine translation quality ?,

00:09:26.04

2011-03-04

12:10 - 12:30 : Session 5-Unobtrusive methods for low-cost manual evaluation of m...

00:13:19.73

2011-03-04

12:30 - 12:45 : Discussion session 5 Tralogy 2011

00:18:11.96

2011-03-04

14:00 - 14:20 : Session 6-Improving MT coherence through text-level processing of...

00:24:49.20

2011-03-04

14:00 - 14:40 : Session 6-Que peut apporter au traducteur la linguistique de corp...

00:11:45.80

2011-03-04

14:40 - 15:00 : Session 6-Using Alignment to detect associated multiword expressi...

00:08:06.72

2011-03-04

15:20 - 15:40 : Session 6-Repérage automatique des équivalences traductionnelle...

00:07:45.84

2011-03-04

15:40 - 16:00 : session 6-Traduction Automatique et multilinguisme : Bonnes prati...

00:37:57.00

2011-03-04

16:00 - 16:05 : discussion session 6 Tralogy 2011

00:20:57.84

2011-03-04

16:05 - 16:15 : session 7-Translation and the New Digital Commons

00:15:32.12

2011-03-04

16:15 - 16:30 : Session 7-TAUS Data Association (TDA) : a member-governed industr...

00:12:25.72

2011-03-04

16:30 - 16:45 : Session 7-Language Resources for Translation and Multilingual Tec...

00:11:03.90

2011-03-04

16:45 - 17:00 : discussion session 7 Tralogy 2011

00:19:06.88

2011-03-04

17:00 - 17:15 : Conclusion-Techniques futures pour le traducteur professionne

00:20:14.88

2011-03-04

17:15 - 17:30 : J’ai connu ce qu’ignorent les Grecs

Contact Webcast

La cellule webcast du CCIN2P3 vous propose de diffuser en direct et/ou en différé sur internet vos manifestations, colloques, conférences. Attention, ce service est réservé au domaine public dans le domaine de la Recherche Scientifique.


Cellule Webcast
Centre de Calcul IN2P3/CNRS
21 Avenue Pierre de Coubertin
CS70202
69627 VILLEURBANNE Cedex

Tél. :
+33(0) 4.78.93.08.80

Fax. :
+33(0) 4.72.69.41.70



Email :

Voir les mentions légales du site