OCR Quality Affects Perceived Usefulness of Historical Newspaper Clippings. A User Study

Kimmo Kettunen, Heikki Keskustalo, Sanna Kumpulainen, Tuula Pääkkönen, Juha Rautiainen

Tutkimustuotos: KonferenssiartikkeliScientificvertaisarvioitu

Abstrakti

Effects of Optical Character Recognition (OCR) quality on historical information retrieval have so far been studied in data-oriented scenarios regarding the effectiveness of retrieval results. Such studies have either focused on the effects of artificially degraded OCR quality (see, e.g., [1-2]) or utilized test collections containing texts based on authentic low quality OCR data (see, e.g., [3]). In this paper the effects of OCR quality are studied in a user-oriented information retrieval setting. Thirty-two users evaluated subjectively query results of six topics each (out of 30 topics) based on pre-formulated queries using a simulated work task setting. To the best of our knowledge our simulated work task experiment is the first one showing empirically that users' subjective relevance assessments of retrieved documents are affected by a change in the quality of optically read text. Users of historical newspaper collections have so far commented effects of OCR'ed data quality mainly in impressionistic ways, and controlled user environments for studying effects of OCR quality on users' relevance assessments of the retrieval results have so far been missing. To remedy this The National Library of Finland (NLF) set up an experimental query environment for the contents of one Finnish historical newspaper, Uusi Suometar 1869-1918, to be able to compare users' evaluation of search results of two different OCR qualities for digitized newspaper articles. The query interface was able to present the same underlying document for the user based on two alternatives: either based on the lower OCR quality, or based on the higher OCR quality, and the choice was randomized. The users did not know about quality differences in the article texts they evaluated. The main result of the study is that improved optical character recognition quality affects perceived usefulness of historical newspaper articles significantly. The mean average evaluation score for the improved OCR results was 7.94% higher than the mean average evaluation score of the old OCR results.

AlkuperäiskieliEnglanti
OtsikkoProceedings of the 18th Italian Research Conference on Digital Libraries
ToimittajatGiorgio Maria di Nunzio, Beatrice Portelli, Domenico Redavid, Gianmaria Silvello
KustantajaCEUR-WS.org
Sivumäärä11
TilaJulkaistu - 2022
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
Tapahtuma18th Italian Research Conference on Digital Libraries, IRCDL 2022 - Padua, Italia
Kesto: 24 helmik. 202225 helmik. 2022

Julkaisusarja

NimiCEUR Workshop Proceedings
KustantajaCEUR-WS
Vuosikerta3160
ISSN (painettu)1613-0073

Conference

Conference18th Italian Research Conference on Digital Libraries, IRCDL 2022
Maa/AlueItalia
KaupunkiPadua
Ajanjakso24/02/2225/02/22

Julkaisufoorumi-taso

  • Jufo-taso 1

!!ASJC Scopus subject areas

  • Computer Science(all)

Sormenjälki

Sukella tutkimusaiheisiin 'OCR Quality Affects Perceived Usefulness of Historical Newspaper Clippings. A User Study'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä