ICAME42, TU Dortmund University, August 18-21, 2021

Kaunisto, M. (Organiser)
Jukka Tyrkkö (Organiser)
Marco Schilk (Organiser)

Language Studies

Activity: Participating in or organising an event › Organization of a workshop, panel, session or tutorial

Description

Workshop: "Corpus pitfalls: dealing with messy data (and other
traps for the unwary)"

Period	18 Aug 2021
Event type	Conference
Conference number	42
Location	Dortmund, GermanyShow on map
Degree of Recognition	International

Country of activity

Germany

Nature of activity

Scientific

Documents & Links

In a short but important paper published thirty years ago in ICAME Journal, Rissanen (1989) identified and discussed three potential problems that the use of corpora may present to unwitting scholars not necessarily closely familiar with the contents, structure and overall character of the corpora that they set out to examine. As solutions to the issues raised, Rissanen promoted the compilation of larger and more representative corpora, as well as getting acquainted with the data itself. Today we have access to corpora which are massive, compared to the ones available thirty years ago, which may widely improve representativeness. At the same time, paradoxically, these increases in corpus size make it considerably less likely that corpus users will have a clear understanding of the characteristics of many of the texts (or text types) included in the corpora. In other words, the sound advice of “knowing your data” becomes increasingly hard to follow when working with corpora that consist of billions of words from a huge number of different sources. It may therefore be argued that larger corpora have also brought about new kinds of problems (as noted, e.g., by Hiltunen, McVeigh & Säily 2017). Finding unwanted items among search results is probably a very typical, almost everyday experience for many corpus linguists. Corpus compilers spend considerable effort regarding corpus annotation, markup, boilerplate removal, identification of duplicates or OCR errors. Similarly, scholars using corpora use increasingly refined methods when constructing elaborate query strings. Yet, achieving perfect precision and/or recall is still highly unlikely. We often need to exclude different kinds of search hits from further analyses, and the reasons for weeding out unwanted items can be varied. Occasionally the occurrence of false positives is mentioned in research articles, perhaps in a footnote, but it may also be the case that much of the clean-up of irrelevant items is done silently. For example, finding a search term within a quotation in a corpus text might justifiably give rise to exclusion of a token from further analysis. In fact, quotations can constitute a significant part of many corpora even in terms of their word count, yet their role overall in corpora has received little attention (see e.g. Rissanen 1992; Kaunisto 2017). Another related issue concerns the inclusion of various levels of linguistic annotation in corpora, which are often accepted as given especially by less experienced corpus linguists, but which may at times be less than helpful (see, e.g., Sinclair 2004; Archer 2012). Furthermore, the dispersion of tokens across the corpora can be a significant factor when assessing search results (see e.g. Gries 2008). There are undoubtedly many types of persistent problems and messiness in corpus data that seasoned scholars have encountered and know about, but which are seldom specifically addressed. Yet beginning corpus users might benefit from learning about what may be regarded as tacit knowledge in corpus linguistics, and even the more advanced scholar may encounter issues new to them that have been addressed earlier. This workshop intends to tap into this knowledge by inviting papers on the following topics: • false positives found in corpora; how to find them or assess their frequency in a corpus? • the significance of identifying different types of unwanted items; how to deal with them and what are the risks if they are not identified? • problems associated with categories built into corpus design and various types of linguistic annotation in corpora; to what extent can these seemingly helpful features encourage uncritical thinking or guide corpus users research? It deserves to be mentioned that problematic aspects may be detected in individual corpora, and observing such infelicities as well as dealing with them is without question necessary and useful as the aim of such observations is to advance corpus linguistic endeavours on the whole. However, instead of focusing on corpus-specific issues, this workshop welcomes papers that reflect on general issues or their own experiences of, and mistakes in, corpus compiling and corpus-based research. In the collegial spirit of ICAME, this workshop is not intended as a forum for highlighting mistakes or shortcomings in fellow scholars’ work.