Investigating the Influence of Context in the Use and Reception of Raw Machine Translation

This dissertation is concerned with how people use raw, unedited machine translation (MT) for “gisting,” or to access texts that are in languages they do not speak, with the aim of understanding as much of the text as is needed for a specific purpose. Technology improvements and the growth in popularity of free online MT tools over the past 15 years has led to an exponential expansion in the number of people using MT, arguably making it one of the most important recent innovations in human communication. However, despite the ubiquity and importance of the phenomenon, to date it has received little attention in research.

The aims of the dissertation were to analyze a limited number of contexts in which MT gisting takes place, to identify factors in those contexts that influence people’s use and reception of raw MT, and finally, to examine theoretical frameworks that can help to conceptualize the phenomenon of MT gisting. The exploratory approach of the dissertation eventually led to four different contexts being analyzed, each of which was unique and required distinct methods of study. Most of the studies relied on qualitative methods; one study was quantitative. Results were published in five articles, and the dissertation comprises those articles plus this dissertation summary.

The four contexts studied were online MT, the use of raw MT in a professional ecosystem, MT-mediated interviewing for research, and the use of MT to increase accessibility to information. The study on the online MT context involved a survey of users of one online MT tool, and the results revealed a very diverse user group who relied on MT for a variety of purposes, the most prominent of which was gisting. Study was the most popular area of life respondents reported using MT in, followed by work and leisure. A somewhat surprising finding was that a large majority (83%) reported having some level of understanding of the language they were translating texts from, a possible indication that they use raw MT in a different way than they use human translations.

In the second context, patent professionals working in the intellectual property rights (IPR) field used raw MT on a regular basis to understand patent documents that are in languages they do not speak. MT gisting in this context was shaped by the fact that it occurred inside an ecosystem that accommodated it. The general riskiness of IPR work provided affordances and a tolerance for the risk involved in using raw MT, and the practice of using raw MT was considered legitimate and supported. A second finding was that patent professionals employed a process of risk assessment and management in evaluating when and where to rely on raw MT. Finally, the use of raw MT in the context was analyzed through the concept of distributed cognition, and the concept was found to be an appropriate way to analyze and understand the practice.

The third context involved data-gathering interviews for research which were conducted via MT-mediated communication, meaning that the interviewer and participants communicated through a chat application with integrated MT. The interviewer and participants did not share a language; rather, each typed messages in their own language and those messages were machine-translated for the other participant. The study concluded that the method was promising and deserving of further study. It also uncovered seven considerations for using MT-mediated communications.

The final context concerned situations in which groups of people lack accessibility to the information they would need in order to participate fully in society, primarily due to a lack of competence in the languages in which information is published. The dissertation reviewed research and projects that aimed to increase accessibility to information and consequently, to improve access to civic life, health and safety information, and culture and media. It also explored promising new developments as well as challenges, including ethical ones, that are involved when using MT for increasing accessibility.

The second aim of the dissertation was to identify factors in the contexts studied that influenced people’s use and reception of raw MT. Eleven contextual influences were identified and classified as relating to users, their tasks and goals, or the technical and organizational environment. User qualities that influence MT gisting included users’ competences in the source or target languages, their familiarity with the textual context of the texts they were machine-translating, and their level of MT literacy. Factors that related to user tasks and goals included a tendency to access multimodal information in source texts to augment understanding of the raw MT, a tendency to verify raw MT output by using multiple MT tools, a tendency to adapt input to produce better MT output, and a tendency to negotiate the meaning of raw MT with others. Environmental aspects that affected MT gisting involved auxiliary technologies that augmented MT use, the status and legitimacy MT enjoys in the environment, and affordances provided by an environment that is accustomed to negotiating the meaning of texts and is tolerant of risk.

Finally, three different frameworks through which MT gisting can be conceptualized were proposed. The phenomenon can be viewed and analyzed through the framework of contextual influences, it can be conceptualized as an exercise in risk management, or it can be viewed through the framework of distributed cognition. It was proposed that the frameworks can be applied in both academic and industrial settings, for example, in future research, in evaluations of the suitability of raw MT for various use cases, in product or process development, or in the development of MT literacy programs.

The dissertation contributes new insights into some previously under-researched or unexplored contexts in which MT gisting is occurring. Its highlights and elevates the role of context in the use and reception of raw MT. Finally, it offers new alternatives for viewing and analyzing the phenomenon of MT gisting.