Lightweight pose estimation network based on non-globally dependent integral regression

doi:10.11772/j.issn.1001-9081.2024030369

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (3): 972-977.DOI: 10.11772/j.issn.1001-9081.2024030369

• Multimedia computing and computer simulation • Previous Articles Next Articles

Lightweight pose estimation network based on non-globally dependent integral regression

Benjie SHE¹, Shuzhi SU¹, Yanmin ZHU²(), Jian HUA¹, Chao WANG¹

^1.School of Computer Science and Engineering，Anhui University of Science and Technology，Huainan Anhui 232001，China
^2.School of Mechatronics Engineering，Anhui University of Science and Technology，Huainan Anhui 232001，China

Received:2024-04-02 Revised:2024-06-04 Accepted:2024-06-11 Online:2024-07-24 Published:2025-03-10
Contact: Yanmin ZHU
About author:SHE Benjie， born in 1999， M. S. candidate. His research interests include pose estimation， behavior recognition.
SU Shuzhi， born in 1987， Ph. D.， associate professor. His research interests include computer vision， fault diagnosis.
HUA Jian， born in 1998， M. S. candidate. His research interests include computer vision.
WANG Chao， born in 1998， M. S. candidate. His research interests include computer vision.
Supported by:
National Natural Science Foundation of China(52374155);Natural Science Research Project of Colleges and Universities in Anhui Province （Major Project）(2022AH040113);Anhui Provincial Natural Science Foundation(2308085MF218);Anhui University of Science and Technology Graduate Innovation Foundation(2023cx2132)

基于非全局依赖积分回归的轻量姿态估计网络

佘本杰¹, 苏树智¹, 朱彦敏²(), 华健¹, 王超¹

^1.安徽理工大学计算机科学与工程学院，安徽淮南 232001
^2.安徽理工大学机电工程学院，安徽淮南 232001

通讯作者: 朱彦敏
作者简介:佘本杰（1999—），男，安徽滁州人，硕士研究生，主要研究方向：姿态估计、行为识别
苏树智（1987—），男，山东泰安人，副教授，博士，主要研究方向：计算机视觉、故障诊断
华健（1998—），男，江苏南京人，硕士研究生，主要研究方向：计算机视觉
王超（1998—），男，江苏常州人，硕士研究生，主要研究方向：计算机视觉。
基金资助:
国家自然科学基金资助项目(52374155);安徽省高等学校自然科学研究项目（重大项目）(2022AH040113);安徽省自然科学基金资助项目(2308085MF218);安徽理工大学研究生创新基金资助项目(2023cx2132)

Abstract

Abstract:

Significant success has been achieved in human pose estimation networks based on heatmap detection. However， the methods based on heatmap detection has a large number of parameters due to redundant computations， quantization errors， and the requirement of heatmap decoding. To address these issues， a Lightweight pose estimation Network based on Non-globally dependent Integral Regression （Lite-NIRNet） was designed to reduce redundant computations in the network by employing Partial Convolution （PConv）， which made the network more lightweight. To respond to the information loss caused by PConv， a Coordinate Attention （CA） mechanism was introduced to fuse inter-channel features， thereby enhancing the network performance. Additionally， a Non-globally dependent Integral Regression （NIR） module was designed to incorporate coordinate supervision to the network， which reduced the influence of quantization errors on network performance. The proposed NIR was able to reduce the bias produced by traditional integral regression during expectation calculations effectively， balancing better learning gradients with lower bias. Experimental results show that compared with the advanced High-Resolution Network （HRNet）， Lite-NIRNet reduces the number of parameters and computational complexity by 73.0% and 63.4%， respectively， on COCO validation set， and achieves the mean Average Precision （mAP） of 72.8% without additional heatmap decoding. Furthermore， on MPII validation set， Lite-NIRNet can also achieve a good balance between network performance and complexity.

Key words: pose estimation, High-Resolution Network (HRNet), Partial Convolution (PConv), Coordinate Attention (CA), integral regression

摘要：

基于热图检测的人体姿态估计网络取得了巨大的成功，然而由于冗余计算、量化误差以及热图解码的需求，基于热图检测的方法参数量较大。针对上述问题，设计基于非全局依赖积分回归的轻量姿态估计网络（Lite-NIRNet）。Lite-NIRNet通过局部卷积（PConv）降低网络的冗余计算，从而使网络更加轻量。为缓解PConv导致的信息丢失问题，引入坐标注意力（CA）机制融合跨通道特征，以提升网络性能。此外，设计非全局依赖的积分回归（NIR）模块，通过该模块，网络可以融入坐标进行监督，从而减少量化误差对网络性能的影响。所提的NIR可有效降低传统积分回归在期望计算时产生的偏差，从而兼顾更好的学习梯度和更低的偏差。实验结果表明，Lite-NIRNet与较先进的高分辨率网络（HRNet）相比，在COCO验证集上的参数量和计算量分别降低了73.0%和63.4%，平均精度均值（mAP）不需要额外的热图解码即可达到72.8%；在MPII验证集上，Lite-NIRNet在网络性能和复杂度之间也能实现良好的平衡。

关键词: 姿态估计, 高分辨率网络, 局部卷积, 坐标注意力, 积分回归

CLC Number:

TP391.4

Benjie SHE, Shuzhi SU, Yanmin ZHU, Jian HUA, Chao WANG. Lightweight pose estimation network based on non-globally dependent integral regression[J]. Journal of Computer Applications, 2025, 45(3): 972-977.

佘本杰, 苏树智, 朱彦敏, 华健, 王超. 基于非全局依赖积分回归的轻量姿态估计网络[J]. 《计算机应用》唯一官方网站, 2025, 45(3): 972-977.

Figures/Tables 9

Fig. 1 Structure of Lite-NIRNet

Fig. 2 Structure of PCA Block

Fig. 3 Comparison of Softmax results with different β coefficients

Tab. 1 Comparison of experimental results of different methods on COCO validation set

方法	输入尺寸	参数量/10⁶	计算量/GFLOPs	mAP/%	$A P 50$ /%	$A P 75$ /%	$A P M$ /%	$A P L$ /%	$A R$ /%
IPR^［8］	256×256	45.0	11.0	67.2	—	—	—	—	—
CPN^［19］	256×192	27.0	6.2	69.4	—	—	—	—	—
Hourglass^［20］	256×192	25.1	14.3	66.9	—	—	—	—	—
Simple Baseline^［21］	256×192	34.0	8.9	70.4	88.6	78.3	67.1	77.2	76.3
HRNet^［5］	256×192	28.5	7.1	73.4	89.5	80.7	70.2	80.1	78.9
Lite-HRNet^［7］	256×192	1.1	0.2	64.8	86.7	73.0	62.1	70.5	71.2
Lite-NIRNet	256×192	7.7	2.6	72.8	89.5	80.0	69.5	79.5	78.0

Tab. 1 Comparison of experimental results of different methods on COCO validation set

方法	输入尺寸	参数量/10⁶	计算量/GFLOPs	mAP/%	$A P 50$ /%	$A P 75$ /%	$A P M$ /%	$A P L$ /%	$A R$ /%
IPR^［8］	256×256	45.0	11.0	67.2	—	—	—	—	—
CPN^［19］	256×192	27.0	6.2	69.4	—	—	—	—	—
Hourglass^［20］	256×192	25.1	14.3	66.9	—	—	—	—	—
Simple Baseline^［21］	256×192	34.0	8.9	70.4	88.6	78.3	67.1	77.2	76.3
HRNet^［5］	256×192	28.5	7.1	73.4	89.5	80.7	70.2	80.1	78.9
Lite-HRNet^［7］	256×192	1.1	0.2	64.8	86.7	73.0	62.1	70.5	71.2
Lite-NIRNet	256×192	7.7	2.6	72.8	89.5	80.0	69.5	79.5	78.0

Tab. 2 Comparison of experimental results of different methods on MPII validation set

方法	输入尺寸	参数量/10⁶	计算量/GFLOPs	PCKh@0.5/%
方法	输入尺寸	参数量/10⁶	计算量/GFLOPs	头部	肘部	肩部	腕部	臀部	膝盖	踝关节	平均
Hourglass^［20］	256×256	25.1	19.10	96.5	88.4	95.3	82.5	87.1	83.5	78.3	87.5
Simple Baseline^［21］	256×256	68.6	20.90	96.7	88.6	95.4	82.9	87.5	83.8	79.0	87.9
HRNet^［5］	256×256	28.5	9.50	96.8	89.5	95.1	85.1	88.2	85.2	82.0	89.3
Lite-HRNet^［7］	256×256	1.8	0.42	—	—	—	—	—	—	—	87.0
FastNet^［22］	256×256	19.7	7.70	—	—	—	—	—	—	—	90.2
Lite-NIRNet	256×256	7.7	3.40	96.9	90.4	95.8	85.1	89.0	85.7	81.3	89.7

Fig. 4 Visualized results of Lite-NIRNet

Tab. 3 Ablation results comparison of Lite-NIRNet modules

模型	SCD	PConv	CA	NIR	参数量/ 10⁶	计算量/GFLOPs	mAP/%
模型1	×	×	×	×	28.5	7.1	71.4
模型2	√	×	×	×	28.5	7.1	73.4
模型3	√	√	×	×	7.6	2.6	71.3
模型4	√	√	√	×	7.7	2.6	72.1
模型5	×	√	√	√	7.7	2.6	72.8
模型6	×	×	×	√	28.5	7.1	74.2

Fig. 5 Influence of different β coefficient on IR and NIR Gaussian kernels

Tab. 4 Influence of different β coefficient on network accuracy

β	mAP/%	β	mAP/%
1	72.2	7	72.4
3	72.7	9	71.9
5	72.8

References 22

1	GU K， YANG L， MI M B， et al. Bias-compensated integral regression for human pose estimation ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2023， 45（9）： 10687-10702.
2	ZHENG C， WU W， CHEN C， et al. Deep learning-based human pose estimation： a survey ［J］. ACM Computing Surveys， 2024， 56（1）： No.11.
3	李坤，侯庆. 基于注意力机制的轻量型人体姿态估计［J］. 计算机应用， 2022， 42（8）： 2407-2414.
	LI K， HOU Q. Lightweight human pose estimation based on attention mechanism ［J］. Journal of Computer Applications， 2022， 42（8）： 2407-2414.
4	TOSHEV A， SZEGEDY C. DeepPose： human pose estimation via deep neural networks ［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014：1653-1660.
5	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5686-5696.
6	WANG Y， LI M， CAI H， et al. Lite Pose： efficient architecture design for 2D human pose estimation ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022：13126-13136.
7	YU C， XIAO B， GAO C， et al. Lite-HRNet： a lightweight high-resolution network ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 10435-10445.
8	SUN X， XIAO B， WEI F， et al. Integral human pose regression［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11210. Cham： Springer， 2018：536-553.
9	GU K， YANG L， YAO A. Dive deeper into integral pose regression［EB/OL］. ［2024-01-23］. .
10	GU K， YANG L， YAO A. Removing the bias of integral pose regression ［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 11047-11056.
11	CHEN J， KAO S H， HE H， et al. Run， don’t walk： chasing higher flops for faster neural networks ［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2023：12021-12031.
12	轩勃娜，李进，宋亚飞，等. 基于改进MobileNetV2的恶意代码分类方法［J］. 计算机应用， 2023， 43（7）：2217-2225.
	XUAN B N， LI J， SONG Y F， et al. Malicious code classification method based on improved MobileNetV2 ［J］. Journal of Computer Applications， 2023， 43（7）： 2217-2225.
13	张宸嘉，朱磊，俞璐. 卷积神经网络中的注意力机制综述［J］. 计算机工程与应用， 2021， 57（20）：64-72.
	ZHANG C J， ZHU L， YU L. Review of attention mechanism in convolutional neural networks ［J］. Computer Engineering and Applications， 2021， 57（20）： 64-72.
14	HOU Q， ZHOU D， FENG J. Coordinate attention for efficient mobile network design ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 13708-13717.
15	ZHANG F， ZHU X， DAI H， et al. Distribution-aware coordinate representation for human pose estimation ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 7091-7100.
16	LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： common objects in context ［C］// Proceedings of the 2014 European Conference on Computer Vision， LNCS 8693. Cham： Springer， 2014： 740-755.
17	ANDRILUKA M， PISHCHULIN L， GEHLER P， et al. 2D human pose estimation： new benchmark and state of the art analysis ［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014：3686-3693.
18	杨观赐，杨静，李少波，等. 基于Dropout与ADAM优化器的改进CNN算法［J］. 华中科技大学学报（自然科学版）， 2018， 46（7）：122-127.
	YANG G C， YANG J， LI S B， et al. Modified CNN algorithm based on Dropout and ADAM optimizer ［J］. Journal of Huazhong University of Science and Technology （Natural Science Edition）， 2018， 46（7）：122-127.
19	CHEN Y， WANG Z， PENG Y， et al. Cascaded pyramid network for multi-person pose estimation ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018：7103-7112.
20	NEWILL A， YANG K， DENG J. Stacked hourglass networks for human pose estimation ［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS 9912. Cham： Springer， 2016：483-499.
21	XIAO B， WU H， WEI Y. Simple baselines for human pose estimation and tracking ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11210. Cham： Springer， 2018： 472-487.
22	LUO Y， OU Z， WAN T， et al. FastNet： fast high-resolution network for human pose estimation ［J］. Image and Vision Computing， 2022， 119： No.104390.

[1]	Jiayang GUI, Shunji WANG, Zhengkang ZHOU, Jiashan TANG. Tunnel foreign object detection algorithm based on improved YOLOv8n [J]. Journal of Computer Applications, 2025, 45(2): 655-661.
[2]	Junying CHEN, Shijie GUO, Lingling CHEN. Lightweight human pose estimation based on decoupled attention and ghost convolution [J]. Journal of Computer Applications, 2025, 45(1): 223-233.
[3]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[4]	Yaxing BING, Yangping WANG, Jiu YONG, Haomou BAI. Six degrees of freedom object pose estimation algorithm based on filter learning network [J]. Journal of Computer Applications, 2024, 44(6): 1920-1926.
[5]	Yi WANG, Jie XIE, Jia CHENG, Liwei DOU. Review of object pose estimation in RGB images based on deep learning [J]. Journal of Computer Applications, 2023, 43(8): 2546-2555.
[6]	Bona XUAN, Jin LI, Yafei SONG, Zexuan MA. Malicious code classification method based on improved MobileNetV2 [J]. Journal of Computer Applications, 2023, 43(7): 2217-2225.
[7]	Guangyao ZHANG, Chunfeng SONG. Pedestrian head tracking model based on full-body appearance features [J]. Journal of Computer Applications, 2023, 43(5): 1372-1377.
[8]	Wenju LI, Gan ZHANG, Liu CUI, Wanghui CHU. Lightweight traffic sign recognition model based on coordinate attention [J]. Journal of Computer Applications, 2023, 43(2): 608-614.
[9]	Wei ZHAO, Yi LI. Kinect-based human pose estimation optimization and animation generation [J]. Journal of Computer Applications, 2022, 42(9): 2830-2837.
[10]	Kun LI, Qing HOU. Lightweight human pose estimation based on attention mechanism [J]. Journal of Computer Applications, 2022, 42(8): 2407-2414.
[11]	Feiyu YANG, Zhan SONG, Zhenzhong XIAO, Yaoyang MO, Yu CHEN, Zhe PAN, Min ZHANG, Yao ZHANG, Beibei QIAN, Chaowei TANG, Wu JIN. Rethinking errors in human pose estimation heatmap [J]. Journal of Computer Applications, 2022, 42(8): 2548-2555.
[12]	Qixiang SUN, Ning HE, Jingzun ZHANG, Chen HONG. Lightweight human pose estimation method based on non-local high-resolution network [J]. Journal of Computer Applications, 2022, 42(5): 1398-1406.
[13]	Dejian WEI, Wenming WANG, Quanyu WANG, Haopan REN, Yanyan GAO, Zhi WANG. Improved 3D hand pose estimation network based on anchor [J]. Journal of Computer Applications, 2022, 42(3): 953-959.
[14]	Kangzhe MA, Jiatian PI, Zhoubing XIONG, Jia LYU. 6D pose estimation incorporating attentional features for occluded objects [J]. Journal of Computer Applications, 2022, 42(12): 3715-3722.
[15]	Jingqi MA, Huan LEI, Minyi CHEN. Fall behavior detection algorithm for the elderly based on AlphaPose optimization model [J]. Journal of Computer Applications, 2022, 42(1): 294-301.

Lightweight pose estimation network based on non-globally dependent integral regression

基于非全局依赖积分回归的轻量姿态估计网络

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 22

Related Articles 15

Recommended Articles

Metrics