A Graphical Pronoun Analysis Tool for the PROTEST Pronoun Evaluation Test Suite
2016 (English)In: Baltic Journal of Modern Computing, ISSN 2255-8942, E-ISSN 2255-8950, Vol. 4, no 2, 318-330 p.Article in journal (Refereed) PublishedText
We present a graphical pronoun analysis tool and a set of guidelines for manual evaluation to be used with the PROTEST pronoun test suite for machine translation (MT). The tool provides a means for researchers to evaluate the performance of their MT systems and browse individual pronoun translations. MT systems may be evaluated automatically by comparing the translation of the test suite pronoun tokens in the MT output with those in the reference translation. Those translations that do not match the reference are referred for manual evaluation, which is supported by the graphical pronoun analysis tool and its accompanying annotation guidelines. By encouraging the manual examination and evaluation of individual pronoun tokens, we hope to understand better how well MT systems perform when translating different categories of pronouns, and gain insights as to where MT systems perform poorly and why.
Place, publisher, year, edition, pages
2016. Vol. 4, no 2, 318-330 p.
Evaluation, machine translation, pronouns, graphical interface, manual annotation
Language Technology (Computational Linguistics)
IdentifiersURN: urn:nbn:se:uu:diva-299621ISI: 000377706600020OAI: oai:DiVA.org:uu-299621DiVA: diva2:949834