融合运动特征和深度学习的跌倒检测算法

doi:10.11772/j.issn.1001-9081.2020050705

计算机应用 ›› 2021, Vol. 41 ›› Issue (2): 583-589.DOI: 10.11772/j.issn.1001-9081.2020050705

所属专题：前沿与综合应用

融合运动特征和深度学习的跌倒检测算法

曹建荣^1,2, 吕俊杰¹, 武欣莹¹, 张旭¹, 杨红娟^1,2

1. 山东建筑大学信息与电气工程学院, 济南 250101;
2. 山东省智能建筑技术重点实验室(山东建筑大学), 济南 250101

收稿日期:2020-05-27 修回日期:2020-07-04 出版日期:2021-02-10 发布日期:2020-08-14
通讯作者: 吕俊杰
作者简介:曹建荣(1965-),男,山东济南人,教授,博士,主要研究方向:模式识别与智能信息处理、视频分析、深度学习、数据挖掘;吕俊杰(1991-),男,山东济南人,硕士研究生,主要研究方向:模式识别与智能信息处理、视频分析、深度学习、数据挖掘;武欣莹(1996-),女,山东济南人,硕士研究生,主要研究方向:深度学习、行人重识别、图像处理;张旭(1994-),男,山东济南人,硕士研究生,主要研究方向:深度学习、视频内容分析。
基金资助:
山东省重点研发计划项目（2019GSF111054，2019GGX104095）；山东省重大科技创新工程（2019JZZY010120）。

Fall detection algorithm integrating motion features and deep learning

CAO Jianrong^1,2, LYU Junjie¹, WU Xinying¹, ZHANG Xu¹, YANG Hongjuan^1,2

1. School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan Shandong 250101, China;
2. Shandong Provincial Key Laboratory of Intelligent Building Technology(Shandong Jianzhu University), Jinan Shandong 250101, China

Received:2020-05-27 Revised:2020-07-04 Online:2021-02-10 Published:2020-08-14
Supported by:
This work is partially supported by the Shandong Provincial Key Research and Development Program (2019GSF111054,2019GGX104095), the Shandong Provincial Major Scientific and Technological Innovation Project (2019JZZY010120).

摘要/Abstract

摘要： 为了利用计算机视觉技术准确检测老年人的跌倒状况，针对现有跌倒检测算法中人为设计特征造成的不完备性以及跌倒检测过程中前后景分离困难、目标混淆、运动目标丢失、跌倒检测准确率低等问题，提出了一种融合人体运动信息的深度学习跌倒检测算法对人体跌倒状态进行检测。首先，通过改进YOLOv3网络进行前景与背景的分离，并根据YOLOv3网络的检测结果对前景人体目标进行最小外接矩形标记；其次，分析人体跌倒过程中的运动特征，将人体运动特征向量化并通过Sigmoid激活函数转化为0到1之间的运动权重信息；最后，通过全连接层将将运动特征与卷积神经网络（CNN）提取的特征进行拼接和融合从而实现人体跌倒分类判别。将所提跌倒检测算法与背景差分、高斯混合、VIBE、方向梯度直方图（HOG）等人体目标检测算法及阈值法、分级法、支持向量机（SVM）分类和CNN分类等人体跌倒判断方案进行了对比实验，并将所提跌倒检测算法在不同光照条件下和混合日常噪声运动干扰下进行了实验，结果表明所提算法在环境适应性和跌倒检测准确率上都优于传统的人体跌倒检测方法。该算法能有效检测出视频中的人体并对人体跌倒状态进行准确检测，进一步验证了融合运动信息的深度学习识别方法在视频跌倒行为分析上的可行性与高效性。

关键词: 跌倒检测, 深度学习, 目标检测, YOLO网络, 运动特征

Abstract: In order to use computer vision technology to accurately detect the fall of the elderly, aiming at the incompleteness of existing fall detection algorithms caused by artificial designing of features and the problems in the fall detection process such as the difficulty of separating foreground and background, the confusion of objects, the loss of moving objects, and the low accuracy of fall detection, a deep learning fall detection algorithm with the fusion of human motion information was proposed to detect the fall state of human body. Firstly, foreground and background were separated by the improved YOLOv3 network, and human object was marked by minimum bounding rectangle according to the detection results of YOLOv3 network. Then, by analyzing the motion features in the process of human fall, the motion features of human body were vectorized and transformed into the motion weight information between 0 and 1 through the Sigmoid activation function. Finally, in order to classify human falls, the motion features and the features extracted by Convolutional Neural Network (CNN) were spliced and fused through the fully connected layer. The proposed fall detection algorithm was compared with human object detection algorithms such as background difference, Gaussian mixture, VIBE (VIsual Background Extractor), Histogram of Oriented Gradient (HOG) and human fall judgment schemes such as threshold method, grading method, Support Vector Machine (SVM) classification, CNN classification, and tested under different lighting conditions and the interference of mixed daily noise motion. The results show that the proposed algorithm is superior to traditional human fall detection algortihms in environmental adaptability and fall detection accuracy. The proposed algorithm can effectively detect the human body in the video and accurately detect the fall state of human body, which further verifies the feasibility and efficiency of the deep learning recognition method with the fusion of motion information in the video fall behavior analysis.

Key words: fall detection, deep learning, object detection, YOLO (You Only Look Once) network, motion feature

中图分类号:

TP391.4

曹建荣, 吕俊杰, 武欣莹, 张旭, 杨红娟. 融合运动特征和深度学习的跌倒检测算法[J]. 计算机应用, 2021, 41(2): 583-589.

CAO Jianrong, LYU Junjie, WU Xinying, ZHANG Xu, YANG Hongjuan. Fall detection algorithm integrating motion features and deep learning[J]. Journal of Computer Applications, 2021, 41(2): 583-589.

参考文献

[1] 叶良, 戚永钊, 施俊侠, 等. 人体摔倒检测技术研究综述[J]. 电子测试,2020(2):50-51,65.(YE L,QI Y Z,SHI J X,et al. Review of fall detection technology[J]. Electronic Test,2020(2):50-51,65.)
[2] 刘伟丽. 基于特征分析的个性化摔倒检测[D]. 大连:大连理工大学,2013:1-5.(LIU W L. Personalized fall detection based on feature analysis[D]. Dalian:Dalian University of Technology, 2013:1-5.)
[3] 禹明娟. 基于视频的室内老人摔倒检测研究[D]. 杭州:杭州电子科技大学,2016:2-3.(YU M J. Research on fall detection of indoor elderly based on video[D]. Hangzhou:Hangzhou Dianzi University,2016:2-3.)
[4] 张小驰, 陈天华, 许继平, 等. 基于多传感器融合的摔倒检测算法的研究[J]. 计算机测量与控制,2015,23(6):2237-2240. (ZHANG X C,CHEN T H,XU J P,et al. Fall detection algorithm research based on multi-sensor fusion[J]. Computer Measurement and Control,2015,23(6):2237-2240.)
[5] FOROUGHI H, ASKI B S, POURREZA H. Intelligent video surveillance for monitoring fall detection of elderly in home environments[C]//Proceedings of the 11th International Conference on Computer and Information Technology. Piscataway:IEEE, 2008:219-224.
[6] MIAOU S G,SUNG P H,HUANG C Y. A customized human fall detection system using omni-camera images and personal information[C]//Proceedings of the 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare. Piscataway:IEEE, 2006:39-42.
[7] ROUGIER C,MEUNIER J,ST-ARNAUD A,et al. Robust video surveillance for fall detection based on human shape deformation[J]. IEEE Transactions on Circuits and Systems for Video Technology,2011,21(5):611-622.
[8] 沈秉乾, 武志勇, 贺前华, 等. 人体姿势状态判决的跌倒检测方法[J]. 计算机应用,2014,34(S1):223-227,264.(SHEN B Q, WU Z Y,HE Q H,et al. Falling detection method based on human body posture judgment[J]. Journal of Computer Applications, 2014,34(S1):223-227,264.)
[9] 马露, 裴伟, 朱永英, 等. 基于深度学习的跌倒行为识别[J]. 计算机科学,2019,46(9):106-112.(MA L,PEI W,ZHU Y Y,et al. Fall action recognition based on deep learning[J]. Computer Science,2019,46(9):106-112.)
[10] GAMMULLE H,DENMAN S,SRIDHARAN S,et al. Two stream LSTM:a deep fusion framework for human action recognition[C]//Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision. Piscataway:IEEE,2017:177-186.
[11] NÚÑEZ-MARCOS A,AZKUNE G,ARGANDA-CARRERAS I. Vision-based fall detection with convolutional neural networks[J]. Wireless Communications and Mobile Computing,2017,2017:No. 9474806.
[12] 申代友, 库洪安, 皮红英, 等. 基于深度相机的老年跌倒监护系统[J]. 中国医学物理学杂志,2019,36(2):223-228.(SHEN D Y,KU H A,PI H Y,et al. Depth camera-based fall detection system for the elderly[J]. Chinese Journal of Medical Physics, 2019,36(2):223-228.)
[13] 袁智, 胡辉. 一种基于双流卷积神经网络跌倒识别方法[J]. 河南师范大学学报(自然科学版),2017,45(3):96-101.(YUAN Z, HU H A. fall detection method based on two-stream convolutional neural network[J]. Journal of Henan Normal University(Natural Science Edition),2017,45(3):96-101.)
[14] REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:unified,real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:779-788.
[15] REDMON J,FARHADI A. YOLO9000:better,faster,stronger[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:6517-6525.
[16] REDMON J, FARHADI A. YOLOv3:an incremental improvement[EB/OL].[2019-09-08]. https://pjreddie.com/media/files/papers/YOLOv3.pdf.
[17] HUANG G,LIU Z,VAN DER MAATEN L,et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:4700-4708.

融合运动特征和深度学习的跌倒检测算法

Fall detection algorithm integrating motion features and deep learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[2]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[3]	刘晓光, 靳少康, 韦子辉, 梁铁, 王洪瑞, 刘秀玲. 基于阈值和极端随机树的实时跌倒检测方法[J]. 计算机应用, 2021, 41(9): 2761-2766.
[4]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[5]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[6]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[7]	马佳良, 陈斌, 孙晓飞. 基于改进的Faster R-CNN的通用目标检测框架[J]. 计算机应用, 2021, 41(9): 2712-2719.
[8]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[9]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[10]	樊玮, 李晨炫, 邢艳, 黄睿, 彭洪健. 航空发动机损伤图像的二分类到多分类递进式检测网络[J]. 计算机应用, 2021, 41(8): 2352-2357.
[11]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[12]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[13]	陈静, 毛莺池, 陈豪, 王龙宝, 王子成. 基于改进单点多盒检测器的大坝缺陷目标检测方法[J]. 计算机应用, 2021, 41(8): 2366-2372.
[14]	肖振远, 王逸涵, 罗建桥, 熊鹰, 李柏林. 基于部分加权损失函数的RefineDet[J]. 计算机应用, 2021, 41(7): 1928-1932.
[15]	侯笑晗, 金国栋, 谭力宁, 薛远亮. 基于自适应和最优特征的合成孔径雷达舰船检测方法[J]. 计算机应用, 2021, 41(7): 2150-2155.