《计算机应用》唯一官方网站

• •    下一篇

基于独立循环神经网络与变分自编码网络的视频帧异常检测

贾晴,王来花,王伟胜   

  1. 曲阜师范大学 网络空间安全学院
  • 收稿日期:2021-12-07 修回日期:2022-04-13 发布日期:2022-06-13 出版日期:2022-06-13
  • 通讯作者: 贾晴
  • 作者简介:贾晴(1998—),女,山东济宁人,硕士研究生,主要研究方向:视频异常检测、计算机视觉;王来花(1988—),女,山东聊城人,副教授,博士,主要研究方向:数字图像处理、视频异常检测;王伟胜(1997—),男,山东烟台人,硕士研究生,主要研究方向:视频异常检测、计算机视觉。
  • 基金资助:
    国家自然科学基金(61601261)

Anomaly detection in video via independently recurrent neural network and variational autoencoder network

JIA Qing, WANG Laihua, WANG Weisheng   

  1. School of Cyber Science and Engineering, Qufu Normal University
  • Received:2021-12-07 Revised:2022-04-13 Online:2022-06-13 Published:2022-06-13
  • About author:JIA Qing, born in 1998. Her research interests include abnormal behavior detection, computer vision. WANG Laihua, born in 1988, Ph. D., associate professor. Her research interests include image processing, abnormal behavior detection. WANG Weisheng, born in 1997. His research interests include computer vision, abnormal behavior detection.
  • Supported by:
     National Natural Science Foundation of China (61601261).

摘要: 为了有效提取连续视频帧间的时间信息,提出一种融合独立循环神经网络(IndRNN)与变分自编码(VAE)网络的预测网络IndRNN-VAE。该网络首先利用变分自编码网络提取视频帧的空间信息,并通过线性变换得到视频帧的潜在特征;然后将潜在特征作为独立循环神经网络的输入以得到视频帧序列的时间信息;最后通过残差块将获得的潜在变量与时间信息进行融合输入到解码网络生成预测帧。通过在UCSD Ped1、Ped2、Avenue公开数据集上进行测试,实验结果表明,与现有的异常检测方法相比,基于IndRNN-VAE网络的方法性能得到了显著提升,AUC值分别达到了84.3%、96.2%和86.6%,EER值分别达到了22.7%、8.8%和19.0%,平均异常得分的差值分别达到了0.263、0.497和0.293,且运行速度达到了28FPS。

关键词: 视频异常检测, 视频监控, 变分自编码器, 独立循环神经网络, 特征提取

Abstract: To effectively extract the temporal information between consecutive video frames, a prediction network IndRNN-VAE that fused Independently Recurrent Neural Network (IndRNN) and Variational AutoEncoder (VAE) network was proposed. Firstly, the spatial information of video frames was extracted with VAE, then the latent features were obtained by a linear transformation. Secondly, the latent features were used as the input of the IndRNN to get the temporal information between video frames. Finally, the latent features and temporal information were concatenated by residual block and input to the decoding network to generate the prediction frame. By testing on UCSD Ped1, Ped2 and Avenue public datasets, experimental results show that compared with the existing anomaly detection methods, the performance of the method based on the IndRNN-VAE is significantly improved, and the AUC values of this method reach 84.3%, 96.2%, and 86.6%, the EER values of it reach 22.7%, 8.8%, and 19.0%, the difference values in the mean anomaly scores of it reach 0.263, 0.497, and 0.293 respectively. Besides, the running speed of this method reaches 28 FPS.

Key words: video anomaly detection, video surveillance, Variational AutoEncoder (VAE), Independently Recurrent Neural Network (IndRNN), feature extraction

中图分类号: