Emotion recognition plays a crucial role in human-computer interaction, and continuous emotion recognition has gained significant attention due to its ability to capture a broader range of emotions, including more subtle ones. In the field of multimodal continuous emotion recognition, this paper proposed a continuous emotion recognition method based on perceiver resampling and multimodal fusion for the problems that the temporal series information obtained by the existing methods contains more redundancy and the obtained multimodal interactive information is not comprehensive. Firstly, the perceiver resampling module removed redundant information, focused on key information, compressed the key features with temporal relationships into hidden vectors, and reduced the computational complexity of the later fusion. Secondly, the multimodal fusion module captured the interactive information between modalities through cross-attention mechanism, and used the self-attention mechanism to obtain the hidden information within each modality, so as to make the feature information richer and more comprehensive. The mean CCC values of arousal and valence on the Ulm-TSST and Aff-Wild2 datasets are 63.62% and 50.09%, respectively, which prove the effectiveness of the model.

[2023-08-10] Accepted Paper /doc/2023.04.0217/v1

[2023-12-05] Printed Article /doc/2023.04.0217/v2