|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'501'711
Articles rated: 2609
19 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
A cross-modal fusion network based on self-attention and residual structure for multimodal emotion recognition | | Ziwang Fu
; Feng Liu
; Hanyang Wang
; Jiayin Qi
; Xiangling Fu
; Aimin Zhou
; Zhibin Li
; | | Date: |
3 Nov 2021 | | Abstract: | The audio-video based multimodal emotion recognition has attracted a lot of
attention due to its robust performance. Most of the existing methods focus on
proposing different cross-modal fusion strategies. However, these strategies
introduce redundancy in the features of different modalities without fully
considering the complementary properties between modal information, and these
approaches do not guarantee the non-loss of original semantic information
during intra- and inter-modal interactions. In this paper, we propose a novel
cross-modal fusion network based on self-attention and residual structure
(CFN-SR) for multimodal emotion recognition. Firstly, we perform representation
learning for audio and video modalities to obtain the semantic features of the
two modalities by efficient ResNeXt and 1D CNN, respectively. Secondly, we feed
the features of the two modalities into the cross-modal blocks separately to
ensure efficient complementarity and completeness of information through the
self-attention mechanism and residual structure. Finally, we obtain the output
of emotions by splicing the obtained fused representation with the original
representation. To verify the effectiveness of the proposed method, we conduct
experiments on the RAVDESS dataset. The experimental results show that the
proposed CFN-SR achieves the state-of-the-art and obtains 75.76% accuracy with
26.30M parameters. Our code is available at
this https URL. | | Source: | arXiv, 2111.02172 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER