Dynamics of Collective Group Affect: Group-level Annotations and the Multimodal Modeling of Convergence and Divergence
IEEE Transactions on Affective Computing, vol. 17, no. 1, pp. 1014–1029
Navin Raj Prabhu
I research how machines can recognise and generate emotion in human speech.
PhD in Computer Science from the University of Hamburg, where I built probabilistic and generative models for emotion recognition and speech emotion conversion. I'm now a Machine Learning Researcher at ai-coustics in Berlin.
Dissertation
Probabilistic and Generative Modeling for Emotion Recognition and Synthesis
- Degree
- PhD in Computer Science, University of Hamburg
- Defended
- 12 February 2026
- Supervisors
- Prof. Timo Gerkmann
Prof. Nale Lehmann-Willenbrock - External reviewer
- Prof. Carlos Busso
- Committee
- Prof. Frank Steinicke
Prof. Stefan Wermter
Current AI systems are good at many things but still poor at social intelligence: perceiving, interpreting and expressing the social signals that make collaboration work. The dissertation takes the social signal processing view, which splits that ability into recognising social signals and synthesising them, and focuses on affect. Across seven research questions, the contributions fall into two lines of work.
Recognising affect
- Bayesian neural networks trained on the full distribution of annotations, so models capture annotator disagreement as well as the mean.Interspeech 2022, IEEE TAFFC 2023
- Student's t-distributions in place of Gaussians, linking how few annotations exist to how uncertain a label is.ACII 2022
- A psychology-grounded protocol for annotating group affect over time, and multimodal models of how groups converge and diverge.IEEE TAFFC 2026
Synthesising affect in speech
- Emotion conversion without parallel data, by disentangling speaker, lexical and emotion embeddings and resynthesising with a HiFiGAN vocoder.ITG Speech Communication 2023
- Diffusion decoders that widen the range of emotions that can be expressed, especially rare, extreme arousal.ICASSP 2024
- A duration predictor that lets speech rate follow arousal: slower when calm, faster when excited.IEEE ASRU 2025
News
Defended my PhD at the University of Hamburg: Probabilistic and Generative Modeling for Emotion Recognition and Synthesis. Thank you to Carlos Busso for serving as external reviewer, and to Frank Steinicke and Stefan Wermter on the committee. Read the department news.
Our work on collective group affect, with annotations and multimodal models of convergence and divergence, is published in IEEE Transactions on Affective Computing.
Presented duration modeling for in-the-wild speech emotion conversion at IEEE ASRU in Honolulu.
A chapter on subjectivity and uncertainty in behavioral labels appeared in Computational Group and Team Dynamics (Oxford University Press).
Presented EMOCONV-Diff, diffusion-based emotion conversion for in-the-wild speech, at ICASSP in Seoul.
Selected publications
Enhancing In-the-Wild Speech Emotion Conversion with Resynthesis-based Duration Modeling
IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Honolulu, USA, December 2025
EMOCONV-Diff: Diffusion-Based Speech Emotion Conversion for Non-Parallel and In-the-Wild Data
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Korea, April 2024, pp. 11651–11655
End-to-End Label Uncertainty Modeling in Speech Emotion Recognition using Bayesian Neural Networks and Label Distribution Learning
IEEE Transactions on Affective Computing, vol. 15, no. 2, pp. 579–592 (published online June 2023)