Human and Machine Perception of Affect in Speech

This poster presents the motivation, methods and goals for a research project on modeling and studying expression of affect in Finnish language. The project combines machine learning and signal processing methods with linguistic knowledge on affect expression. The research aims at finding ways to recognize how affect is expressed by individual speakers in different speaking conditions (differentiating what is being said from how things are being said) and how affect expressed in spoken language relates to affect expressed in, and implied by, written language. The research is planned to span over a four year period, during which a corpus (dataset) of emotional Finnish speech will be collected, analyzed, and processed for modelling purposes. Based on this data, a speech emotion classifier for conversational Finnish will be developed and later further improved to enable the study and automatic recognition of idiolectical variation of affect expression. Results of the project can be used for improved affective AI systems that are able to understand the richness of human emotions in spoken communication in various circumstances by various speakers and listeners.