Detection of Typical Pronunciation Errors in Non-native English Speech Using Convolutional Recurrent Neural Networks

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

1 Citation (Scopus)


A machine learning method for the automatic detection of pronunciation errors made by non-native speakers of English is proposed. It consists of training word-specific binary classifiers on a collected dataset of isolated words with possible pronunciation errors, typical for Finnish native speakers. The classifiers predict whether the typical error is present in the given word utterance. They operate on sequences of acoustic features, extracted from consecutive frames of an audio recording of a word utterance. The proposed architecture includes a convolutional neural network, a recurrent neural network, or a combination of the two. The optimal topology and hyperpa-rameters are obtained in a Bayesian optimisation setting using a tree-structured Parzen estimator. A dataset of 80 words uttered naturally by 120 speakers is collected. The performance of the proposed system, evaluated on a well-represented subset of the dataset, shows that it is capable of detecting pronunciation errors in most of the words (46/49) with high accuracy (mean accuracy gain over the zero rule 12.21 percent points).

Original languageEnglish
Title of host publication2019 International Joint Conference on Neural Networks, IJCNN 2019
ISBN (Electronic)9781728119854
Publication statusPublished - 1 Jul 2019
Publication typeA4 Article in conference proceedings
EventInternational Joint Conference on Neural Networks - Budapest, Hungary
Duration: 14 Jul 201919 Jul 2019

Publication series

ISSN (Electronic)2161-4407


ConferenceInternational Joint Conference on Neural Networks


  • Computer-assisted language learning
  • computer-assisted pronunciation training CNN
  • CRNN
  • GRU
  • pronunciation learning

Publication forum classification

  • Publication forum level 1

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence


Dive into the research topics of 'Detection of Typical Pronunciation Errors in Non-native English Speech Using Convolutional Recurrent Neural Networks'. Together they form a unique fingerprint.

Cite this