A Multi-term and Multi-task Analyzing Framework for Affective Analysis in-the-wi

A Multi-term and Multi-task Analyzing Framework for Affective Analysis in-the-wild

Human affective recognition is an important factor in human-computer interaction. However, the method development with in-the-wild data is not yet accurate enough for practical usage.In this paper, we introduce the affective recognition method focusing on valence-arousal (VA) and expression (EXP) that was submitted to the Affective Behavior Analysis in-the-wild (ABAW) 2020 Contest. Since we considered that affective behaviors have many observable features that have their own time frames, we introduced multiple optimized time windows (short-term, middle-term, and long-term) into our analyzing framework for extracting feature parameters from video data. Moreover, multiple modality data are used, including action units, head poses, gaze, posture, and ResNet 50 or Efficient NET features, and are optimized during the extraction of these features. Then, we generated affective recognition models for each time window and ensembled these models together. Also, we fussed the valence, arousal, and expression models together to enable the multi-task learning, considering the fact that the basic psychological states behind facial expressions are closely related to each another. In the validation set, our model achieved a valence-arousal score of 0.498 and a facial expression score of 0.471. These verification results reveal that our proposed framework can improve estimation accuracy and robustness effectively.

进行情感分析的多项多任务分析框架

人的情感识别是人机交互的重要因素。但是，使用野外数据进行方法开发对于实际使用而言还不够准确。.. 在本文中，我们介绍了针对价-主动（VA）和表达（EXP）的情感识别方法，该方法已提交至2020年ABAW情感行为分析竞赛。由于我们认为情感行为具有许多具有各自时间范围的可观察特征，因此我们在分析框架中引入了多个优化时间窗口（短期，中期和长期），以从视频数据中提取特征参数。而且，使用了多种模态数据，包括动作单位，头部姿势，注视，姿势和ResNet 50或Efficient NET功能，并在提取这些功能时对其进行了优化。然后，我们为每个时间窗口生成了情感识别模型，并将这些模型整合在一起。此外，我们大惊小怪的价，唤醒，考虑到面部表情背后的基本心理状态彼此密切相关这一事实，可以将表情模型与表情模型结合在一起以实现多任务学习。在验证集中，我们的模型的化合价分数为0.498，面部表情分数为0.471。这些验证结果表明，我们提出的框架可以有效地提高估计的准确性和鲁棒性。 （阅读更多）