SPECIAL FEATURE: AI IN T&I, PART 2
In Part 1 of our feature on AI in T&I, published in the July issue of In Touch, Dr Anna Gadd, Director of Graduate Translation Studies at the University of Western Australia, described several of the AI models currently being used in translation. Here she goes on to look at the accuracy of AI in translation from the perspective of a T&I educator.
My description of some of the AI models currently in use by translators didn’t make any kind of qualitative statement, assessment or evaluation of their efficacy.
The number of models available on the market is large, and increasing. Educators have been updating their skills, and know that they need to prepare translation students to be ready to work not only on existing technologies, but also on those that haven’t been created yet, by using existing technologies training platforms for those of the future. Language students, interpreters and translators all have their preferred AI tools, depending on the language pair(s), sector and genre they work with. These factors can highlight vast gaps in the performance of the technology. Literature shows that AI, fishing from the belly of the internet, does better on topics that are the subject of widespread written communication in the online community, and less well on obscure or niche topics. There is, therefore, a wide gap between its accuracy on the majority languages (i.e., those most used on the internet, such as Spanish, English and Russian) and minority languages (such as Catalan).
AI … does not ‘understand’ the translations it creates …
Fundamental to AI’s issues with accuracy is the fact that it does not ‘understand’ the translations it creates; it merely gathers content from the internet and its own users, adopts a majority standard, and regurgitates this. This means it can – and does – make grammatical and contextual errors, especially if errors are common among its users – which is why, as existing literature shows, we still need the language and cultural expertise of trained and qualified translators.
Ehrensberger-Dow et al. explain how AI gives a false idea of correctness: ‘the raw output from some of the freely available systems is often convincing (and misleadingly fluent; cf. Martindale and Carpuat 2018), so might be seen as a quick, economical, accessible alternative to involving a third party who has the competence to provide professional translation.’1 In the same publication, the authors identify typical problems in raw machine translation and divide them into four categories: accuracy, adaptation, coherence and conventions.2
As most users of AI are not expert translators, they may accept less-than-accurate translations simply because they do not have the knowledge required to identify calques (loan translations), common errors and biases. And when AI has made an error, it replicates it ad infinitum. It is only with in-depth linguistic and cultural knowledge that these errors can be corrected; without this oversight, the errors are here to stay.
Ehrensberger-Dow et al. also warn us about the danger of perpetuating various types of bias.3
AI differs significantly from online corpora, which can be useful to students looking for more authoritative linguistic information. Linguistic and translational corpora are compiled by expert linguists and translators, who carefully select material according to specific criteria. Using both corpora and AI and comparing their output might be useful in teaching both L2 learners and future translators.
Analysis carried out across six languages (Mandarin, Japanese, Italian, Spanish, Korean and Bahasa Indonesia) at the University of Western Australia in October 2024 aimed to ascertain whether AI technologies, when used for translation, produce results that are accurate, reliable and consistent.
Results showed that the technologies analysed [those listed in Dr Gadd’s article ‘AI in translation: an overview,’ published in the last issue of In Touch] produce texts which are far from perfect translations; that their accuracy varies dramatically depending on languages, genre and sector: and that a human mind is still very much needed in order to achieve reliable and consistent results. Despite the obvious usefulness of these tools, and clear improvements on earlier machine technologies, their performance was often inaccurate: there were errors, calques, common technological issues such as overtranslation of names and addresses, and both language- and sector-specific issues. In the Asian languages involved, particularly those that don’t use the Roman alphabet, omissions were constant: terms or even entire phrases of the source text were often – and apparently arbitrarily – skipped. It certainly was a case of translations ‘looking good at first sight,’ as current literature on the topic already shows.
… AI performed most poorly [in] literary translation.
The genre in which AI performed most poorly was literary translation. In all six languages it was perceived that ‘AI does not feel and therefore it cannot translate literary passages well.’ Even in the case of informative texts – which, according to linguist and translation scholar Katharina Reiss, are only developed on the level of communication, serve no aesthetic purpose and have no operative end, and should be rather easy to translate, or at least less complex than expressive or operative texts – results presented a significant number of serious errors (in the translation of addresses and names, for instance). Considering the potential repercussions of errors in official documents, it is reassuring that NAATI certification is typically required for their translation in Australia.
Quality assessment of AI translation – evaluating the efficacy of these tools in specific language pairs, text types and genres – is, therefore, essential. We cannot take AI output as valid per se; rather we must carry out studies to assess the accuracy of AI in translation. As non-specialists tend to judge a translation’s efficacy on fluency, and lack sufficient language knowledge to question AI output and investigate further, incorrect AI ‘translations’ will be perceived as correct, setting theoretical translation studies back centuries. As theorists of translation we have superseded these concepts and have moved forward towards translations that no longer attempt to pretend they are originals, but rather are unapologetically that: translations.
When using AI, our task as contemporary translators is to check the target texts thoroughly for bias, to examine them carefully for errors, and to correct them by using both our traditional skills as translators and our expertise in culture and language.
There is also an ethical/legal issue in the context of AI which teachers should explicitly address. Intellectual property rights (IPR) laws proscribe the uploading of excerpts of original literary texts onto AI platforms without explicit permission from their authors, and this includes feeding excerpts into the cloud via free accounts. Non-specialists and students of languages and translation are often unaware of the laws and issues around copyright and IPRs, hence we should do everything we can to educate our students and the wider public on these matters.
I believe it is our task, as experts in translation, to educate non-specialists on how imperfect AI tools are, and also on the errors they make and the biases they perpetuate, and how to avoid them. We must show non-specialists that these tools often fail, and that their output should not be regarded as authoritative.
Dr Anna Gadd is an award-winning language lecturer and translator with experience spanning three countries and nearly three decades. Dr Gadd launched the Spanish Department at the University of Western Australia (UWA) in 2017, and is currently UWA’s Director of Graduate Translation Studies and Convenor of Spanish Studies. She has also published extensively on translation studies and second language acquisition.
The International Federation of Translators (FIT) published a Position Paper on the Use of AI in Interpreting (2024) and a Position Paper on Machine Translation in the Age of AI (2025), and AUSIT is currently preparing its own Position Statement on AI in Translation & Interpreting.
1 Ehrensberger-Dow M, Delorme Benites A & Lehr C (2023) A New Role for Translators and Trainers: MT literacy consultants. The Interpreter and Translator Trainer 17(3), 395.
2 Ehrensberger-Dow et al., 403.
3 Ehrensberger-Dow et al., 403.
4 Reiss K (1981) Type, Kind and Individuality of Text: Decision making in translation text typology. Poetics Today 2(4), 121–31.
5 Benjamin W (1923) The Translator’s Task. In Venuti L, The Translation Studies Reader, 75–83.
Further reading
Berner S (2019) Utilising technology ethically. In Touch 27(2), 1.
Berner S (2023) ChatGPT: Neither doomsday nor a panacea. In Touch 31(3), 3.
Chan S-W (2023) The Development of Translation Technology: 1967–2014. In Chan S-W, Routledge Encyclopedia of Translation Technology, 4.
Doherty S (2017) Let’s talk machine translation (Part 1). In Touch 25(3), 3.
Doherty S, Castilho S, Gaspari F & Moorkens J (2018) Let’s talk machine translation (Part 2). In Touch 26(1), 3.
Doherty S & Kenny D (2014) The design and evaluation of a statistical machine translation syllabus for translation students. The Interpreter and Translator Trainer 8(2), 295–315.
Ordorica S (2020) Why technology will not replace professional translators. Forbes.
Advertisement:
