Abstract:

In order to solve the occlusion problem of student expression recognition in complex classroom scenes， and give full play to the advantages of deep learning in the application of intelligent teaching evaluation，a student expression recognition model and an intelligent teaching evaluation algorithm based on deep attention network in classroom teaching videos were proposed. A video library， an expression library and a behavior library for classroom teaching were constructed， then， multi-channel facial images were generated by cropping and occlusion strategies. A multi-channel deep attention network was built and self-attention mechanism was used to assign different weights to multiple channel networks. The weight distribution of each channel was restricted by a constrained loss function， then the global feature of the facial image was expressed as the quotient of the sum of the product of the feature times its attention weight of each channel divided by the sum of the attention weights of all channels. Based on the learned global facial feature， the student expressions in classroom were classified， and the student facial expression recognition under occlusion was realized. An intelligent teaching evaluation algorithm that integrates the student facial expressions and behavior states in classroom was proposed， which realized the recognition of student facial expressions and intelligent teaching evaluation in classroom teaching videos. By making experimental comparison and analysis on the public dataset FERplus and self-built classroom teaching video datasets， it is verified that the student facial expressions recognition model in classroom teaching videos achieves high accuracy of 87.34%， and the intelligent teaching evaluation algorithm that integrates the student facial expressions and behavior states in classroom achieves excellent performance on the classroom teaching video dataset.

Key words: deep learning, deep attention network, facial expression recognition, intelligent teaching evaluation, classroom teaching video

摘要：

为了解决复杂课堂场景下学生表情识别的遮挡的问题，同时发挥深度学习在智能教学评估应用上的优势，提出了一种基于深度注意力网络的课堂教学视频中学生表情识别模型与智能教学评估算法。构建了课堂教学视频库、表情库和行为库，利用裁剪和遮挡策略生成多路人脸图像，在此基础上构建了多路深度注意力网络，并通过自注意力机制为多路网络分配不同权重。通过约束损失函数限制各路权重的分配，将人脸图像的全局特征表示为每个支路的特征乘上注意力权重的和除以所有支路的注意力权重之和，并基于学习到的人脸全局特征进行学生课堂表情分类，实现遮挡情况下学生人脸表情识别。提出了融合课堂学生表情和行为状态的智能教学评估算法，实现了课堂教学视频中学生表情识别与智能教学评估。在公开数据集FERplus与自建课堂教学视频数据集上进行实验对比与分析，验证了提出的课堂教学视频中学生表情识别模型能够达到87.34%的准确率，且提出的融合课堂学生表情和行为状态的智能教学评估算法在课堂教学视频数据集上也取得优秀的性能。

关键词: 深度学习, 深度注意力网络, 表情识别, 智能教学评估, 课堂教学视频