《计算机应用》唯一官方网站

• •    下一篇

基于骨架和3D热图的注意缺陷多动障碍患者动作识别算法

石超1,周昱昕1,扶倩1,唐万宇1,何凌1,李元媛2   

  1. 1.四川大学 生物医学工程学院 2. 四川大学 华西医院心理卫生中心
  • 收稿日期:2024-09-11 修回日期:2025-01-11 发布日期:2025-03-24 出版日期:2025-03-24
  • 通讯作者: 李元媛
  • 作者简介:石超(1997—),男,贵州铜仁人,硕士研究生,主要研究方向:图像处理算法;周昱昕(2003—),女,广西桂林人,主要研究方向:人工智能算法;扶倩(2001—),女,四川成都人,硕士研究生,主要研究方向:深度学习算法;唐万宇(2001—),男,四川自贡人,硕士研究生,主要研究方向:视频图像处理;何凌(1981—),女,四川成都人,副教授,博士,主要研究方向:医学信号处理、医学图像处理;李元媛(1984—),女,辽宁大连人,副主任医师,博士,主要研究方向:青少年精神卫生、医学数据挖掘。
  • 基金资助:
    四川省科技计划项目(2023YFS0290)

Action recognition algorithm for ADHD patients using skeleton and 3D heatmap#br#
#br#

SHI Chao1, ZHOU Yuxin1, FU Qian1, TANG Wanyu1, HE Ling1, LI Yuanyuan2   

  1. 1. School of Biomedical Engineering, Sichuan University 2. West China Mental Health Center, Sichuan University
  • Received:2024-09-11 Revised:2025-01-11 Online:2025-03-24 Published:2025-03-24
  • About author:SHI Chao, born in 1997, M. S. candidate. His research interests include image processing algorithm. ZHOU Yuxin, born in 2003. Her research interests include artificial intelligence algorithm. FU Qian, born in 2001, M. S. candidate. Her research interests include deep learning algorithm. TANG Wanyu, born in 2001, M.S. candidate. His research interests include video picture processing. HE Ling, born in 1981, Ph. D., associate professor. Her research interests include medical signal processing, medical image processing. LI Yuanyuan, born in 1984, Ph. D., associate chief physician. Her research interests include adolescent mental health, medical data mining.
  • Supported by:
    Science and Technology Program of Sichuan Province (2023YFS0290)

摘要: 注意缺陷多动障碍(ADHD)是一种常见于儿童期的神经发育障碍,以注意力不集中、多动和冲动为主要特征,常表现出特定的动作模式。传统的动作识别算法在处理这些特定动作时存在识别准确率低、响应速度慢等问题。为解决此类问题,将2D姿态数据转化为3D热图方法,通过高斯分布精确地表示关节点间的空间关系,以有效保留时空信息。针对单一模态数据的限制,引入基于骨架和3D热图的多模态集成方法。通过融合Short 3D-CNN和自适应图卷积网络(AGCN)的输出特征,充分利用两种模态数据的优势,提升动作识别性能。在四川大学华西医院心理卫生中心采集的ADHD患者数据集上的实验结果表明,对于8种不同类型的动作,所提算法的TOP-1识别正确率为0.8604,TOP-5识别正确率为0.9873。此外,提出基于动作类型的ADHD自动分型算法,将ADHD分型为头面部体动型、躯干体动型和四肢体动型,识别正确率为0.7500,响应速度为5s,与2s-AGCN和POSEC3D相比,所提算法在复杂动作场景下具有更高的识别精度,为ADHD的个性化干预提供了新的技术手段。

关键词: 注意缺陷多动障碍, 动作识别, 骨架数据, 图卷积网络;3D卷积神经网络

Abstract: Attention Deficit Hyperactivity Disorder (ADHD) is a common neurodevelopmental disorder in childhood, characterized by inattention, hyperactivity, and impulsivity, often exhibiting specific motion patterns. Traditional action recognition algorithms were found to have low recognition accuracy and slow response speed when handling these specific actions. To address such issues, a method was adopted where 2D pose data were transformed into 3D heatmaps, and spatial relationships between joints were precisely represented using Gaussian distribution, which effectively preserved spatiotemporal information. To overcome the limitations of single-modal data, a multimodal fusion method based on skeleton and 3D heatmaps was introduced. The output features of Short 3D-CNN and Adaptive Graph Convolutional Network (AGCN) were fused to fully exploit the advantages of both modalities, thereby improving action recognition performance. Experimental results on the ADHD patient dataset collected at the Mental Health Center of West China Hospital, Sichuan University, showed that the proposed algorithm achieved a TOP-1 recognition accuracy of 0.8604 and a TOP-5 recognition accuracy of 0.9873 for eight different types of actions. Additionally, an automatic ADHD classification algorithm based on action types was proposed, which classified ADHD into head and facial movement type, trunk movement type, and limb movement type, achieving a recognition accuracy of 0.75 and a response time of 5 seconds. Compared with 2s-AGCN and POSEC3D, the proposed algorithm demonstrated higher recognition accuracy in complex action scenarios, providing a new technical approach for personalized ADHD intervention.

Key words: Attention Deficit Hyperactivity Disorder (ADHD), action recognition, skeleton data, Graph Convolutional Network (GCN), 3D Convolutional Neural Network (3D-CNN)

中图分类号: