Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (2): 583-589.DOI: 10.11772/j.issn.1001-9081.2020050705

Special Issue: 前沿与综合应用

• Frontier and comprehensive applications • Previous Articles     Next Articles

Fall detection algorithm integrating motion features and deep learning

CAO Jianrong1,2, LYU Junjie1, WU Xinying1, ZHANG Xu1, YANG Hongjuan1,2   

  1. 1. School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan Shandong 250101, China;
    2. Shandong Provincial Key Laboratory of Intelligent Building Technology(Shandong Jianzhu University), Jinan Shandong 250101, China
  • Received:2020-05-27 Revised:2020-07-04 Online:2021-02-10 Published:2020-08-14
  • Supported by:
    This work is partially supported by the Shandong Provincial Key Research and Development Program (2019GSF111054,2019GGX104095), the Shandong Provincial Major Scientific and Technological Innovation Project (2019JZZY010120).

融合运动特征和深度学习的跌倒检测算法

曹建荣1,2, 吕俊杰1, 武欣莹1, 张旭1, 杨红娟1,2   

  1. 1. 山东建筑大学 信息与电气工程学院, 济南 250101;
    2. 山东省智能建筑技术重点实验室(山东建筑大学), 济南 250101
  • 通讯作者: 吕俊杰
  • 作者简介:曹建荣(1965-),男,山东济南人,教授,博士,主要研究方向:模式识别与智能信息处理、视频分析、深度学习、数据挖掘;吕俊杰(1991-),男,山东济南人,硕士研究生,主要研究方向:模式识别与智能信息处理、视频分析、深度学习、数据挖掘;武欣莹(1996-),女,山东济南人,硕士研究生,主要研究方向:深度学习、行人重识别、图像处理;张旭(1994-),男,山东济南人,硕士研究生,主要研究方向:深度学习、视频内容分析。
  • 基金资助:
    山东省重点研发计划项目(2019GSF111054,2019GGX104095);山东省重大科技创新工程(2019JZZY010120)。

Abstract: In order to use computer vision technology to accurately detect the fall of the elderly, aiming at the incompleteness of existing fall detection algorithms caused by artificial designing of features and the problems in the fall detection process such as the difficulty of separating foreground and background, the confusion of objects, the loss of moving objects, and the low accuracy of fall detection, a deep learning fall detection algorithm with the fusion of human motion information was proposed to detect the fall state of human body. Firstly, foreground and background were separated by the improved YOLOv3 network, and human object was marked by minimum bounding rectangle according to the detection results of YOLOv3 network. Then, by analyzing the motion features in the process of human fall, the motion features of human body were vectorized and transformed into the motion weight information between 0 and 1 through the Sigmoid activation function. Finally, in order to classify human falls, the motion features and the features extracted by Convolutional Neural Network (CNN) were spliced and fused through the fully connected layer. The proposed fall detection algorithm was compared with human object detection algorithms such as background difference, Gaussian mixture, VIBE (VIsual Background Extractor), Histogram of Oriented Gradient (HOG) and human fall judgment schemes such as threshold method, grading method, Support Vector Machine (SVM) classification, CNN classification, and tested under different lighting conditions and the interference of mixed daily noise motion. The results show that the proposed algorithm is superior to traditional human fall detection algortihms in environmental adaptability and fall detection accuracy. The proposed algorithm can effectively detect the human body in the video and accurately detect the fall state of human body, which further verifies the feasibility and efficiency of the deep learning recognition method with the fusion of motion information in the video fall behavior analysis.

Key words: fall detection, deep learning, object detection, YOLO (You Only Look Once) network, motion feature

摘要: 为了利用计算机视觉技术准确检测老年人的跌倒状况,针对现有跌倒检测算法中人为设计特征造成的不完备性以及跌倒检测过程中前后景分离困难、目标混淆、运动目标丢失、跌倒检测准确率低等问题,提出了一种融合人体运动信息的深度学习跌倒检测算法对人体跌倒状态进行检测。首先,通过改进YOLOv3网络进行前景与背景的分离,并根据YOLOv3网络的检测结果对前景人体目标进行最小外接矩形标记;其次,分析人体跌倒过程中的运动特征,将人体运动特征向量化并通过Sigmoid激活函数转化为0到1之间的运动权重信息;最后,通过全连接层将将运动特征与卷积神经网络(CNN)提取的特征进行拼接和融合从而实现人体跌倒分类判别。将所提跌倒检测算法与背景差分、高斯混合、VIBE、方向梯度直方图(HOG)等人体目标检测算法及阈值法、分级法、支持向量机(SVM)分类和CNN分类等人体跌倒判断方案进行了对比实验,并将所提跌倒检测算法在不同光照条件下和混合日常噪声运动干扰下进行了实验,结果表明所提算法在环境适应性和跌倒检测准确率上都优于传统的人体跌倒检测方法。该算法能有效检测出视频中的人体并对人体跌倒状态进行准确检测,进一步验证了融合运动信息的深度学习识别方法在视频跌倒行为分析上的可行性与高效性。

关键词: 跌倒检测, 深度学习, 目标检测, YOLO网络, 运动特征

CLC Number: