关键词: 时序关联性, 帧关联短视频事件, 卷积神经网络, 贝叶斯神经网络, 不确定度

Abstract: Aiming at the problem of how to combine the frame uncertainty and temporal correlation of short videos to enhance event detection, a frame-associated short video event detection method based on uncertainty perception was proposed. Firstly, 2D convolutional neural network was used to extract the features of each frame of short video, and then the proposed features were forward propagated several times to obtain the feature mean value and the uncertainty information corresponding to the features through Bayesian variational layering. Then, the uncertainty-aware module constructed by the model was used to fuse the feature mean value and the uncertainty information, and then the fused features of the frames were strengthened by the temporal correlation module to enhance the connection in time domain. Finally, the temporal correlation features were combined with the temporal correlation module to enhance the event detection in short video, the time-domain associated features were realized through the classification network for short video event detection. The short video event detection dataset crawled from Flickr platform was utilized to carry out experimental comparison, experimental results show that subspace learning methods such as Support Vector Machine (SVM) have the poorest classification performance and do not adequately explore high-level semantic representations, while deep learning has significantly better accuracy for event detection. Compared to SviTT method, the accuracy, average recall, and average precision of the proposed method are improved by 3.37%, 2.55%, and 2.09%, respectively. The effectiveness of the proposed method for the task of short video event detection was verified. 

Key words: temporal correlation, frame associated short video event, convolutional neural network, Bayesian neural network, uncertainty