Aiming at the problem of intra-modality feature representation and inter modality feature fusion in multimodal sentiment analysis, this paper proposed a multi-level hybrid fusion multi-modal sentiment analysis model based on attention mechanism and multi-task learning. Firstly, the model used convolution neural network and bi-directional gated unit to extract the single-modality internal feature. Secondly, it used the cross-modality attention mechanism to realize the pairwise feature fusion between modalities. Thirdly, it used the self-attention mechanism to select the modality contribution at different levels. Finally, combining with multi-task learning, the model obtained both sentiment and emotion classification results. The experimental results on CMU-MOSEI dataset show that this method can improve the accuracy and F1-score of sentiment and emotion classification.

[2021-11-29] Accepted Paper /doc/2021.08.0357/v1

[2022-03-05] Printed Article /doc/2021.08.0357/v2