Non-perception class attendance method based on student body detection

doi:10.11772/j.issn.1001-9081.2020010067

Abstract

Abstract: Concerning the missed detection and low recognition rate in the class attendance system based on face recognition, a method that combines student body detection and face angle filtering was proposed by applying the master and slave dual-camera device. First, the bodies of students were detected from the photograph of master camera by the Mask R-CNN algorithm. Then, the slave camera (PTZ (Pan/Tilt/Zoom) camera) was controlled to acquire high-quality magnified image of each student in turn. Next, the face poses were detected and recognized in the magnified images through MTCNN (Multi-Task Convolutional Neural Network) algorithm and FSA(Fine-grained Structure Aggregation)-Net algorithm in order to filter the frontal face image of every student. Finally, the FaceNet algorithm was used to extract the features of the filtered student frontal face images for training or recognition of Support Vector Machine (SVM) classifiers. Experimental results showed that, compared with the Tiny-face algorithm, when the Intersection over Union (IOU) was 0.75, the body detection algorithm had the Average Precision (AP) value increased by about 36% and the detection time reduced by 57%; compared with the method of establishing a multi-pose face database, the method of face angle filtering improved the recognition rate by 4%; and the accuracy of student recognition in the entire classroom was close to 100% in most cases. The proposed method simplifies the student registration process, improves the face recognition rate, and provides new ideas for solving the problem of face missed detection.

Key words: class attendance, object detection, face pose, face recognition, body detection

摘要： 针对基于人脸识别的课堂考勤系统漏检和低识别率的问题，采用主、从双摄像机设备，提出一种联合学生人体检测和人脸角度筛选的方法。首先通过Mask R-CNN算法检测主摄像机拍摄图中的学生人体位置；然后控制从摄像机（PTZ相机）依次获取每位学生的高质量放大图像；再通过MTCNN算法和FSA-Net算法从中检测并识别出人脸姿态，筛选出每位学生的正脸图像；最后对筛选出的学生正脸图像使用FaceNet算法提取人脸特征，用于支持向量机（SVM）分类器的训练或识别。实验结果表明，与Tiny-face算法相比，人体检测算法在重叠比（IOU）为0.75时平均精度（AP）值提高了约36%且检测耗时减少了57%；与建立多姿态人脸数据库的方法相比，采用人脸角度筛选的方法使识别率提高了4%；多数情况下整个课堂学生识别的准确率接近100%。所提方法简化了学生注册过程，提高了人脸识别率，为解决人脸漏检问题提供了新的思路。

关键词: 课堂考勤, 目标检测, 人脸姿态, 人脸识别, 人体检测

CLC Number:

TP183

FANG Shuya, LIU Shouyin. Non-perception class attendance method based on student body detection[J]. Journal of Computer Applications, 2020, 40(9): 2519-2524.

方书雅, 刘守印. 基于学生人体检测的无感知课堂考勤方法[J]. 计算机应用, 2020, 40(9): 2519-2524.

References

[1] NORDMANN E,CALDER C,BISHOP P,et al. Turn up,tune in, don't drop out:the relationship between lecture attendance,use of lecture recordings,and achievement at different levels of study[J]. Higher Education,2019,77(6):1065-1084.
[2] 万强, 王海晖, 周剑秋. 基于大数据的教学分析[J]. 中国科技信息,2017(5):40-43.(WAN Q,WANG H H,ZHOU J Q. Teaching analysis based on big data[J]. China Science and Technology Information,2017(5):40-43.)
[3] TEIXEIRA A A C. The impact of class absenteeism on undergraduates' academic performance:evidence from an elite Economics school in Portugal[J]. Innovations in Education and Teaching International,2016,53(2):230-242.
[4] BALCOH N K,YOUSAF M H,AHMAD W,et al. Algorithm for efficient attendance management:face recognition based approach[J]. International Journal of Computer Science Issues,2012,9(4):146-150.
[5] FU R,WANG D,LI D,et al. University classroom attendance based on deep learning[C]//Proceedings of the 10th International Conference on Intelligent Computation Technology and Automation. Piscataway:IEEE,2017:128-131.
[6] SARKAR P R, MISHRA D, SUBHRAMANYAM G R K S. Automatic attendance system using deep learning framework[M]//TANVEER M, PACHORI R. Machine Intelligence and Signal Analysis,AISC 748. Singapore:Springer,2019:335-346.
[7] MERY D,MACKENNEY I,VILLALOBOS E. Student attendance system in crowded classrooms using a smartphone camera[C]//Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision. Piscataway:IEEE,2019:857-866.
[8] SUREKHA B,NAZARE K J,VISWANADHA RAJU S,et al. Attendance recording system using partial face recognition algorithm[M]//DEY N, SANTHI V. Intelligent Techniques in Signal Processing for Multimedia Security,SCI 660. Cham:Springer, 2017:293-319.
[9] VIOLA P, JONES M J. Robust real-time face detection[J]. International Journal of Computer Vision,2004,57(2):137-154.
[10] ZHANG K,ZHANG Z,LI Z,et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters,2016,23(10):1499-1503.
[11] 刘宇明, 凌志祥, 吴强, 等. 基于多任务卷积网络的参会人员人数统计算法[J]. 计算机应用,2018,38(S2):51-54.(LIU Y M, LING Z X, WU Q, et al. People counting algorithm for participants based on multi-task convolutional network[J]. Journal of Computer Applications,2018,38(S2):51-54.)
[12] HU P,RAMANAN D. Finding tiny faces[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:1522-1530.
[13] 龚锐, 丁胜, 章超华, 等. 基于深度学习的轻量级和多姿态人脸识别方法[J]. 计算机应用, 2020,40(3):704-709.(GONG R, DING S,ZHANG C H,et al. Lightweight and multi-pose face recognition method based on deep learning[J]. Journal of Computer Applications,2020, 40(3):704-709.)
[14] 卢涛, 杨威, 万永静. 基于图像超分辨极限学习机的极低分辨率人脸识别[J]. 计算机应用,2016,36(2):580-585.(LU T, YANG W, WAN Y J. Face recognition with extremely low resolution based on image super-resolution extreme learning machine[J]. Journal of Computer Applications,2016,36(2):580-585.)
[15] ZANGENEH E, RAHMATI M, MOHSENZADEH Y. Low resolution face recognition using a two-branch deep convolutional neural network architecture[J]. Expert Systems with Applications,2020,139:No. 112854.
[16] 方冠男. 基于视频流人脸识别的课堂考勤系统的设计与实现[D]. 武汉:华中师范大学,2018:30-40.(FANG G N. Design and implementation of class attendance system based on face recognition with real-time video[D]. Wuhan:Central China Normal University,2018:30-40.)
[17] HE K,GKIOXARI G,DOLLÁR P,et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:2980-2988.
[18] REN S,HE K,GIRSHICK R,et al. Faster R-CNN:towards realtime object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2015:91-99.
[19] LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:3431-3440.
[20] HE K,ZHANG X,REN S,et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015, 37(9):1904-1916.
[21] XIANG J,ZHU G. Joint face detection and facial expression recognition with MTCNN[C]//Proceedings of the 4th International Conference on Information Science and Control Engineering. Piscataway:IEEE,2017:424-427.
[22] WANG Y, YUAN G, ZHENG D, et al. Research on face detection method based on improved MTCNN network[C]//Proceedings of the 11th International Conference on Digital Image Processing,11179. Bellingham,WA:SPIE,2019:No. 111791C.
[23] YANG T Y,CHEN Y T,LIN Y Y,et al. FSA-Net:learning finegrained structure aggregation for head pose estimation from a single image[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019:1087-1096.
[24] SCHROFF F,KALENICHENKO D,PHILBIN J. FaceNet:a unified embedding for face recognition and clustering[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:815-823.