|
[1] PLUTCHIK R. The nature of emotions: human emotions have deep evolutionary roots, a fact that may explain their com-plexity and provide tools for clinical practice[J]. American Scientist, 2001, 89(4): 344-350.
[2] EKMAN P, FRIESEN W V. Constants across cultures in the face and emotion[J]. Journal of Personality and Social Psycho-logy, 1971, 17(2): 124.
[3] PORIA S, CAMBRIA E, BAJPAI R, et al. A review of affective computing: from unimodal analysis to multimodal fusion[J]. Information Fusion, 2017, 37: 98-125.
[4] PENG X J. Multi-modal affective computing: a comprehensive survey[J]. Journal of Hengyang Normal University, 2018, 39(3): 31-36.
彭小江. 基于多模态信息的情感计算综述[J]. 衡阳师范学院学报, 2018, 39(3): 31-36.
[5] SOLEYMANI M, GARCIA D, JOU B, et al. A survey of mul-timodal sentiment analysis[J]. Image and Vision Computing, 2017, 65: 3-14.
[6] HUDDAR M G, SANNAKKI S S, RAJPUROHIT V S. A survey of computational approaches and challenges in multi-modal sentiment analysis[J]. International Journal of Com-puter Sciences and Engineering, 2019, 7(1): 876-883.
[7] GAO J, LI P, CHEN Z, et al. A survey on deep learning for multimodal data fusion[J]. Neural Computation, 2020, 32(5): 829-864.
[8] ZHENG W L, LU B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J]. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162-175.
[9] LI S, DENG W. Deep facial expression recognition: a survey[J]. IEEE Transactions on Affective Computing, 2020.
[10] ZHANG Y, LAI G, ZHANG M, et al. Explicit factor models for explainable recommendation based on phrase-level senti-ment analysis[C]//Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Informa-tion Retrieval, Queensland, Jul 6-11, 2014. New York: ACM, 2014: 83-92.
[11] XU N, MAO W J, CHEN G D. Multi-interactive memory network for aspect based multimodal sentiment analysis[C]//Proceedings of the 2019 AAAI Conference on Artificial Intelligence, Hawaii, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 371-378.
[12] KOELSTRA S, MUHL C, SOLEYMANI M, et al. DEAP: a database for emotion analysis; using physiological signals[J]. IEEE Transactions on Affective Computing, 2011, 3(1): 18-31.
[13] YU W M, XU H, MENG F Y, et al. CH-SIMS: a Chinese multimodal sentiment analysis dataset with fine-grained annotation of modality[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 3718-3727.
[14] MORENCY L P, MIHALCEA R, DOSHI P. Towards multi-modal sentiment analysis: harvesting opinions from the Web[C]//Proceedings of the 13th International Conference on Multimodal Interfaces, Alicante, Nov 14-18, 2011. New York: ACM, 2011: 169-176.
[15] W?LLMER M, WENINGER F, KNAUP T, et al. Youtube movie reviews: sentiment analysis in an audio-visual con-text[J]. IEEE Intelligent Systems, 2013, 28(3): 46-53.
[16] ZADEH A, ZELLERS R, PINCUS E, et al. MOSI: multi-modal corpus of sentiment intensity and subjectivity analysis in online opinion videos[J]. arXiv:1606.06259, 2016.
[17] ELLIS J G, JOU B, CHANG S F. Why we watch the news: a dataset for exploring sentiment in broadcast video news[C]//Proceedings of the 16th International Conference on Multimodal Interaction, Istanbul, Nov 12-16, 2014. New York: ACM, 2014: 104-111.
[18] BUSSO C, BULUT M, LEE C C, et al. IEMOCAP: inter-active emotional dyadic motion capture database[J]. Language Resources and Evaluation, 2008, 42(4): 335-359.
[19] FENG X Y, HUANG D, CUI S X, et al. Spatial-temporal attention network for facial expression recognition[J]. Journal of Northwest University (Natural Science Edition), 2020, 50(3): 319-327.
冯晓毅, 黄东, 崔少星, 等. 基于空时注意力网络的面部表情识别[J]. 西北大学学报(自然科学版), 2020, 50(3): 319-327.
[20] LU J H, ZHANG S M, ZHAO J L. Facial expression reco-gnition based on CNN ensemble[J]. Journal of Qingdao University (Engineering & Technology Edition), 2020, 35(2): 24-29.
陆嘉慧, 张树美, 赵俊莉. 基于CNN集成的面部表情识别[J]. 青岛大学学报(工程技术版), 2020, 35(2): 24-29.
[21] LI X L, NIU H T. Facial expression recognition using feature fusion based on VGG-NET[J]. Computer Engineering & Science, 2020, 42(3): 500-509.
李校林, 钮海涛. 基于VGG-NET的特征融合面部表情识别[J]. 计算机工程与科学, 2020, 42(3): 500-509.
[22] LUO Y, ZHU L Z, LU B L. A GAN-based data augmentation method for multimodal emotion recognition[C]//LNCS 11554: Proceedings of the 16th International Symposium on Neural Networks, Moscow, Jul 10-12, 2019. Cham: Springer, 2019: 141-150.
[23] WANG K, PENG X, YANG J, et al. Suppressing uncertainties for large-scale facial expression recognition[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 6897-6906.
[24] LI S, DENG W H. A deeper look at facial expression dataset bias[J]. IEEE Transactions on Affective Computing, 2020.
[25] DAI R. Facial recognition method based on facial physiological features and deep learning[J]. Journal of Chongqing Univer-sity of Technology (Natural Science), 2020, 34(6): 146-153.
戴蓉. 基于面部生理特征和深度学习的表情识别方法[J]. 重庆理工大学学报(自然科学), 2020, 34(6): 146-153.
[26] YADOLLAHI A, SHAHRAKI A G, ZAIANE O R. Current state of text sentiment analysis from opinion to emotion mining[J]. ACM Computing Surveys, 2017, 50(2): 1-33.
[27] ARAQUE O, ZHU G, IGLESIAS C A. A semantic similarity-based perspective of affect lexicons for sentiment analysis[J]. Knowledge-Based Systems, 2019, 165: 346-359.
[28] ZHAO Y Y, QIN B, LIU T. Sentiment analysis[J]. Journal of Software, 2010, 21(8): 1834-1848.
赵妍妍, 秦兵, 刘挺. 文本情感分析[J]. 软件学报, 2010, 21(8): 1834-1848.
[29] CHEN F, YUAN Z, HUANG Y. Multi-source data fusion for aspect-level sentiment classification[J]. Knowledge-Based Systems, 2020, 187: 104831.
[30] LI Z, FAN Y, JIANG B, et al. A survey on sentiment analysis and opinion mining for social multimedia[J]. Multimedia Tools and Applications, 2019, 78(6): 6939-6967.
[31] PANG B, LEE L, VAITHYANATHAN S. Thumbs up? Senti-ment classification using machine learning techniques[C]//Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, Philadelphia, Jul 6-7, 2002. New York: ACM, 2002: 79-86.
[32] LI T T, JI D H. Sentiment analysis of micro-blog based on SVM and CRF using various combinations of features[J]. Application Research of Computers, 2015, 32(4): 978-981.
李婷婷, 姬东鸿. 基于 SVM 和 CRF 多特征组合的微博情感分析[J]. 计算机应用研究, 2015, 32(4): 978-981.
[33] PRUSA J D, KHOSHGOFTAAR T M, DITTMAN D J. Using ensemble learners to improve classifier performance on Tweet sentiment data[C]//Proceedings of the 2015 IEEE International Conference on Information Reuse and Integration, San Francisco, Aug 13-15, 2015: 252-257.
[34] CHEN J Y, YAN S K, WONG K C. Verbal aggression detec-tion on Twitter comments: convolutional neural network for short-text sentiment analysis[J]. Neural Computing and App-lications, 2020, 32(15): 10809-10818.
[35] CHEN K, LIANG B, KE W D, et al. Chinese micro-blog sentiment analysis based on multi-channels convolutional neural networks[J]. Journal of Computer Research and Deve-lopment, 2018, 55(5): 945-957.
陈珂, 梁斌, 柯文德, 等. 基于多通道卷积神经网络的中文微博情感分析[J]. 计算机研究与发展, 2018, 55(5): 945-957.
[36] ZHU Y, CHEN S P. Commentary text sentiment analysis combining convolution neural network and attention[J]. Journal of Chinese Computer Systems, 2020, 41(3): 551-557.
朱烨, 陈世平. 融合卷积神经网络和注意力的评论文本情感分析[J]. 小型微型计算机系统, 2020, 41(3): 551-557.
[37] CAO Y, LI T R, JIA Z, et al. BGRU: new method of Chinese text sentiment analysis[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(6): 973-981.
曹宇, 李天瑞, 贾真, 等. BGRU: 中文文本情感分析的新方法[J]. 计算机科学与探索, 2019, 13(6): 973-981.
[38] WANG X, JIANG W, LUO Z Y. Combination of convolutional and recurrent neural network for sentiment analysis of short texts[C]//Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Dec 11-16, 2016. Strou-dsburg: ACL, 2016: 2428-2437.
[39] LI Y, PAN Q, WANG S, et al. A generative model for category text generation[J]. Information Sciences, 2018, 450: 301-315.
[40] PAO T L, CHEN Y T, YEH J H, et al. Detecting emotions in Mandarin speech[J]. International Journal of Computational Linguistics & Chinese Language Processing, 2005, 10(3): 347-362.
[41] LI Y C, ISHI C T, WARD N, et al. Emotion recognition by combining prosody and sentiment analysis for expressing reactive emotion by humanoid robot[C]//Proceedings of the 2017 Asia-Pacific Signal and Information Processing Asso-ciation Annual Summit and Conference, Kuala Lumpur, Dec 12-15, 2017. Piscataway: IEEE, 2017: 1356-1359.
[42] SEMWAL N, KUMAR A, NARAYANAN S. Automatic speech emotion detection system using multi-domain acoustic feature selection and classification models[C]//Proceedings of the 2017 IEEE International Conference on Identity, Security and Behavior Analysis, New Delhi, Feb 22-24, 2017. Piscataway: IEEE, 2017: 1-6.
[43] SAMANTARAY A K, MAHAPATRA K, KABI B, et al. A novel approach of speech emotion recognition with prosody, quality and derived features using SVM classifier for a class of North-Eastern Languages[C]//Proceedings of the 2nd IEEE International Conference on Recent Trends in Information Systems, Kolkata, Jul 9-11, 2015. Piscataway: IEEE, 2015: 372-377.
[44] HUANG Z W, DONG M, MAO Q R, et al. Speech emotion recognition using CNN[C]//Proceedings of the 2014 ACM International Conference on Multimedia, Orlando, Nov 3-7, 2014. New York: ACM, 2014: 801-804.
[45] REN Z, JIA J, GUO Q, et al. Acoustics, content and geo-information based sentiment prediction from large-scale networked voice data[C]//Proceedings of the 2014 IEEE International Conference on Multimedia and Expo, Chengdu, Jul 14-18, 2014. Piscataway: IEEE, 2014: 1-4.
[46] WU L, OVIATT S L, COHEN P R. Multimodal integration—a statistical view[J]. IEEE Transactions on Multimedia, 1999, 1(4): 334-341.
[47] ZHANG C, YANG Z C, HE X D, et al. Multimodal intel-ligence: representation learning, information fusion, and app-lications[J]. IEEE Journal of Selected Topics in Signal Pro-cessing, 2020, 14(3): 478-493.
[48] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Com-munications of the ACM, 2017, 60(6): 84-90.
[49] SUN Y Y, JIA Z T, ZHU H Y. Survey of multimodal deep learning[J]. Computer Engineering and Applications, 2020, 56(21): 1-10.
孙影影，贾振堂，朱昊宇. 多模态深度学习综述[J]. 计算机工程与应用, 2020, 56(21): 1-10.
[50] PéREZ-ROSAS V, MIHALCEA R, MORENCY L P. Utterance-level multimodal sentiment analysis[C]//Procee-dings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Aug 4-9, 2013. Stroudsburg: ACL, 2013: 973-982.
[51] PORIA S, CAMBRIA E, HAZARIKA D, et al. Context-dependent sentiment analysis in user-generated videos[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 873-883.
[52] DENG D, ZHOU Y, PI J, et al. Multimodal utterance-level affect analysis using visual, audio and text features[J]. arXiv:1805.00625, 2018.
[53] PORIA S, CHATURVEDI I, CAMBRIA E, et al. Convolu-tional MKL based multimodal emotion recognition and senti-ment analysis[C]//Proceedings of the 16th International Con-ference on Data Mining, Barcelona, Dec 12-15, 2016. Piscata-way: IEEE, 2016: 439-448.
[54] HU T T, SHEN L J, FENG Y Q, et al. Research on anger and happy misclassification in speech and text emotion recognition[J]. Computer Technology and Development, 2018, 28(11): 124-127.
胡婷婷, 沈凌洁, 冯亚琴, 等. 语音与文本情感识别中愤怒与开心误判分析[J]. 计算机技术与发展, 2018, 28(11): 124-127.
[55] CHEN F, LUO Z, XU Y, et al. Complementary fusion of multi-features and multi-modalities in sentiment analysis[J]. arXiv:1904.08138, 2019.
[56] KUMAR A, VEPA J. Gated mechanism for attention based multi modal sentiment analysis[C]//Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, May 4-8, 2020. Piscataway: IEEE, 2020: 4477-4481.
[57] XU N, MAO W J. MultiSentiNet: a deep semantic network for multimodal sentiment analysis[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, Nov 6-10, 2017. New York: ACM, 2017: 2399-2402.
[58] YU J, JIANG J, XIA R. Entity-sensitive attention and fusion network for entity-level multimodal sentiment classification[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, 28: 429-439.
[59] YU Y, LIN H, MENG J, et al. Visual and textual sentiment analysis of a microblog using deep convolutional neural networks[J]. Algorithms, 2016, 9(2): 41.
[60] PORIA S, CAMBRIA E, GELBUKH A. Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis[C]//Procee-dings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Sep 17-21, 2015. Stroudsburg: ACL, 2015: 2539-2544.
[61] WANG H H, MEGHAWAT A, MORENCY L P, et al. Select-additive learning: improving generalization in multimodal sentiment analysis[C]//Proceedings of the 2017 IEEE Inter-national Conference on Multimedia and Expo, Hong Kong, China, Jul 10-14, 2017. Washington: IEEE Computer Society, 2017: 949-954.
[62] YU H L, GUI L K, MADAIO M, et al. Temporally selective attention model for social and affective state recognition in multimedia content[C]//Proceedings of the 25th ACM Inter-national Conference on Multimedia, Mountain View, Oct 23-27, 2017. New York: ACM, 2017: 1743-1751.
[63] CHEN M H, WANG S, LIANG P P, et al. Multimodal sentiment analysis with word-level fusion and reinforce-ment learning[C]//Proceedings of the 19th ACM International Conference on Multimodal Interaction, Glasgow, Nov 13-17, 2011. New York: ACM, 2017: 163-171.
[64] SHENOY A, SARDANA A. Multilogue-Net: a context aware RNN for multi-modal emotion detection and sentiment analysis in conversation[J]. arXiv:2002.08267, 2020.
[65] CIMTAY Y, EKMEKCIOGLU E, CAGLAR-OZHAN S. Cross-subject multimodal emotion recognition based on hybrid fusion[J]. IEEE Access, 2020, 8: 168865-168878.
[66] GUNES H, PICCARDI M. Bi-modal emotion recognition from expressive face and body gestures[J]. Journal of Network and Computer Applications, 2007, 30(4): 1334-1345.
[67] FIéRREZ-AGUILAR J, ORTEGA-GARCIA J, GONZALEZ-RODRIGUEZ J. Fusion strategies in multimodal biometric verification[C]//Proceedings of the 2003 IEEE International Conference on Multimedia and Expo, Baltimore, Jul 6-9, 2003. Piscataway: IEEE, 2003: 5-8.
[68] JIANG T, WANG J H, LIU Z Y, et al. Fusion-extraction network for multimodal sentiment analysis[C]//LNCS 12085: Proceedings of the 24th Pacific-Asia Conference on Know-ledge Discovery and Data Mining, Singapore, May 11-14, 2020. Cham: Springer, 2020: 785-797.
[69] JIANG D, ZOU D, DENG Z, et al. Contextual multimodal sentiment analysis with information enhancement[J]. Journal of Physics: Conference Series, 2020, 1453(1): 012159.
[70] ZADEH A, CHEN M, PORIA S, et al. Tensor fusion network for multimodal sentiment analysis[J]. arXiv:1707.07250, 2017.
[71] VERMA S, WANG J, GE Z, et al. Deep-HOSeq: deep higher order sequence fusion for multimodal sentiment analysis[J]. arXiv:2010.08218, 2020.
[72] VIELZEUF V, LECHERVY A, PATEUX S, et al. CentralNet: a multilayer approach for multimodal fusion[C]//LNCS 11134: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 575-589.
[73] MAJUMDER N, HAZARIKA D, GELBUKH A F, et al. Multimodal sentiment analysis using hierarchical fusion with context modeling[J]. Knowledge Based Systems, 2018, 161: 124-133.
|