Robust joint modeling and optimization method for visual manipulators

doi:10.11772/j.issn.1001-9081.2022010037

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (3): 962-971.DOI: 10.11772/j.issn.1001-9081.2022010037

• Frontier and comprehensive applications • Previous Articles

Robust joint modeling and optimization method for visual manipulators

Xianbojun FAN¹, Lijia CHEN¹(), Shen LI², Chenlu WANG¹, Min WANG¹, Zan WANG¹, Mingguo LIU¹

^1.School of Physics and Electronics，Henan University，Kaifeng Henan 475004，China
^2.Kaifeng Pingmei New Carbon Materials Technology Company Limited，Kaifeng Henan 475002，China

Received:2022-01-13 Revised:2022-03-14 Accepted:2022-03-22 Online:2022-04-14 Published:2023-03-10
Contact: Lijia CHEN
About author:FAN Xianbojun， born in 1994， M. S. candidate. His research interests include swarm intelligence algorithm.
CHEN Lijia， born in 1979， Ph. D.， associate professor. His research interests include intelligent computing.
LI Shen， born in 1982， assistant engineer. His research interests include swarm intelligence algorithm.
WANG Chenlu， born in 1995， M. S. candidate. Her research interests include image processing.
WANG Min， born in 1997， M. S. candidate. Her research interests include neural network.
WANG Zan， born in 1984， Ph. D.， associate professor. His research interests include radar signal.
LIU Mingguo， born in 1984， Ph. D.， associate professor. His research interests include swarm intelligence algorithm.
Supported by:
National Natural Science Foundation of China(61901158);Key R & D and Promotion Special Project of Henan Provincal Department of Science and Technology(202102210121);Major Special Project of Kaifeng City(20ZD014)

鲁棒的视觉机械臂联合建模优化方法

范贤博俊¹, 陈立家¹(), 李珅², 王晨露¹, 王敏¹, 王赞¹, 刘名果¹

^1.河南大学物理与电子学院，河南开封 475004
^2.开封平煤新型炭材料科技有限公司，河南开封 475002

通讯作者: 陈立家
作者简介:范贤博俊（1994—），男，河南义马人，硕士研究生，主要研究方向：群智能算法
陈立家（1979—），男，河南开封人，副教授，博士，主要研究方向：智能计算
李珅（1982—），男，河南开封人，助理工程师，主要研究方向：群智能算法
王晨露（1995—），女，河南郑州人，硕士研究生，主要研究方向：图像处理
王敏（1997—），女，山东菏泽人，硕士研究生，主要研究方向：神经网络
王赞（1984—），男，河南开封人，副教授，博士，主要研究方向：雷达信号
刘名果（1984—），男，河南巩义人，副教授，博士，主要研究方向：群智能算法。
基金资助:
国家自然科学基金资助项目(61901158);河南省科技厅重点研发与推广专项(202102210121);开封市重大专项(20ZD014)

Abstract

Abstract:

To address the problems of low accuracy， difficult deployment and high calibration cost of visual manipulator in complex system environments， a robust joint modelling and optimization method for visual manipulators was proposed. Firstly， the subsystem models of the visual manipulator were integrated together， and the sample data such as servo motor rotation angles and manipulator end-effector coordinates were collected randomly in the workspace of the manipulator. Then， an Adaptive Multiple-Elites-guided Composite Differential Evolution algorithm with shift mechanism and Layered Optimization mechanism （AMECoDEs-LO） was proposed. Simultaneous optimization of the joint system parameters was completed by using the method of parameter identification. Principal Component Analysis （PCA） was performed by AMECoDEs-LO on stage data in the population， and with the idea of parameter dimensionality reduction， an implicit guidance for convergence accuracy and speed was realized. Experimental results show that under the cooperation of AMECoDEs-LO and the joint system model， the visual manipulator does not require additional instruments during calibration， achieving fast deployment and a 60% improvement in average accuracy compared to the conventional method. In the cases of broken manipulator linkages， reduced servo motor accuracy and increased camera positioning noise， the system still maintains high accuracy， which verifies the robustness of the proposed method.

Key words: visual manipulator, layered optimization, Principal Component Analysis (PCA), joint calibration, robustness

摘要：

针对视觉机械臂在复杂系统环境下整体精度不高、不易部署、校准成本高的问题，提出一种鲁棒的视觉机械臂联合建模优化方法。首先，对视觉机械臂的各个子系统模型进行集成，在机械臂的工作空间随机采集伺服电机转角、机械臂末端坐标等数据。其次，提出一种具有分层优化机制的自适应多精英引导复合差分进化算法（AMECoDEs-LO），使用参数辨识的方法同时优化联合系统参数。AMECoDEs-LO对种群中阶段性的数据进行主成分分析（PCA），以参数降维的思想实现对收敛精度和速度的隐式引导。实验结果表明，在AMECoDEs-LO和联合系统模型的作用下，视觉机械臂在校准过程中不需要额外的仪器，部署速度快，最终精度相较于传统方法提高60%；在机械臂连杆受损、伺服电机精度降低、相机定位噪声增大的情况下，系统仍然保持较高精度，验证了所提方法的鲁棒性。

关键词: 视觉机械臂, 分层优化, 主成分分析, 联合标定, 鲁棒性

CLC Number:

TP242

Xianbojun FAN, Lijia CHEN, Shen LI, Chenlu WANG, Min WANG, Zan WANG, Mingguo LIU. Robust joint modeling and optimization method for visual manipulators[J]. Journal of Computer Applications, 2023, 43(3): 962-971.

范贤博俊, 陈立家, 李珅, 王晨露, 王敏, 王赞, 刘名果. 鲁棒的视觉机械臂联合建模优化方法[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 962-971.

Figures/Tables 23

Fig.1 Grab process of visual manipulator system

Fig.2 Overall system optimization strategy

Fig.3 Joint coordinate system

Fig.4 Individual coding structure of AMECoDEs-LO

Fig.5 Switching between three optimization states

Fig.6 Flowchart of AMECoDEs-LO

Tab.1 Five individual configurations

编码类型	维度	配置
Type 0	20	$Φ$ ； T 前馈测量
Type 1	20	$Φ$ ； $ψ, T$ 前馈测量
Type 2	26	$Φ, Τ$ ； $ψ$ 前馈测量
Type 3	26	$Φ, T$
Type 4	41	$ψ, Φ, T$

Tab.1 Five individual configurations

编码类型	维度	配置
Type 0	20	$Φ$ ； T 前馈测量
Type 1	20	$Φ$ ； $ψ, T$ 前馈测量
Type 2	26	$Φ, Τ$ ； $ψ$ 前馈测量
Type 3	26	$Φ, T$
Type 4	41	$ψ, Φ, T$

Fig.7 Convergence of three individual configurations under normal manipulator environment

Tab.2 Individual configuration under normal manipulator environment

编码类型	迭代次数	收敛时间/s	Fitness/mm
Type 0	2 137	39	1.086 935
Type 3	4 948	92	0.718 967
Type 4	7 679	145	0.726 831

Fig.8 Convergence of four individual configurations under broken manipulator environment

Tab.3 Individual configurations under broken manipulator environment

编码类型	迭代次数	收敛时间/s	Fitness/mm
Type 1	1 944	37	1.886 338
Type 2	4 758	88	1.685 230
Type 3	4 988	93	1.458 970
Type 4	7 835	149	0.755 262

Tab.4 Different combinations of two parameters

趋平率阈值	切换率阈值	迭代次数	Fitness/mm
0.95	0.50	14 194	0.740 816
0.98	0.50	39 418	0.750 376
0.90	0.50	7 835	0.755 262
	0.55	6 547	0.812 778
	0.60	6 072	0.881 793
0.85	0.50	5 768	0.898 836
	0.55	5 373	0.916 278
	0.60	5 174	1.361 520
0.75	0.50	3 726	1.232 095
	0.55	3 169	1.391 849
	0.60	2 479	1.513 099

Fig.9 Comparison of fitness in populations with different scales

Fig.10 Performance of AMECoDEs-LO

Fig.11 Accuracy verification comparison between joint system and traditional method

Fig.12 Performance of servo motor soft compensation

Tab.5 Results of three schemes on different indexes

方案	RSS	MAE	RMSE	SD
f₁（x）	54.932	0.609	0.741	0.422
f₂（x）	14.248	0.281	0.377	0.252
f₃（x）	11.727	0.265	0.342	0.217

Fig.13 Four different data collection areas

Tab.6 Fitness comparison of two manipulators under seven algorithms in four collection areas

采集点数	机械臂	迭代次数	不同算法下的Fitness/mm
采集点数	机械臂	迭代次数	AMECoDEs-LO	AMECoDEs	PSO	GA	DE	IMPEDE	DESPS
25	R1	5 000	0.565 326	0.982 446	0.830 997	2.561 803	1.786 521	1.177 729	1.355 992
		10 000	0.532 048	0.719 843	0.770 335	2.057 273	1.237 504	0.969 473	0.765 316
		20 000	0.532 048	0.520 238	0.770 335	2.005 239	1.031 816	0.901 608	0.591 952
	R2	5 000	0.586 456	0.601 693	0.876 969	1.911 315	1.779 222	1.437 815	0.635 495
		10 000	0.585 898	0.573 826	0.741 698	1.852 563	1.092 410	1.135 834	0.599 992
		20 000	0.585 463	0.568 397	0.718 350	1.787 653	1.086 331	0.872 544	0.599 821
50	R1	5 000	0.726 315	1.148 595	1.870 231	3.041 135	1.593 779	0.744 047	2.842 765
		10 000	0.716 056	0.980 794	1.683 181	2.551 561	1.517 333	0.668 578	2.753 945
		20 000	0.677 934	0.744 180	1.644 061	2.550 204	1.329 803	0.665 560	0.933 366
	R2	5 000	0.664 622	1.983 563	1.975 506	3.164 919	2.273 540	0.665 486	2.731 112
		10 000	0.656 657	1.433 057	1.700 505	2.823 198	2.117 706	0.659 641	1.134 591
		20 000	0.650 878	0.767 033	1.698 549	2.432 036	1.367 968	0.630 471	0.942 106
75	R1	5 000	0.689 986	0.733 063	1.823 818	5.371 781	1.091 436	0.826 387	1.079 996
		10 000	0.689 852	0.733 061	1.800 903	3.579 188	1.086 935	0.794 721	1.042 488
		20 000	0.689 852	0.733 061	1.710 061	2.822 724	1.086 787	0.786 419	0.947 404
	R2	5 000	0.715 865	0.791 578	2.052 678	3.245 265	1.121 884	1.249 125	1.657 453
		10 000	0.690 582	0.707 336	1.832 811	3.087 309	1.121 884	1.161 167	1.158 240
		20 000	0.689 395	0.707 065	1.690 035	2.948 902	1.121 884	0.799 485	0.983 625
100	R1	5 000	0.774 702	1.172 915	3.391 321	2.591 554	1.719 753	1.893 676	2.185 051
		10 000	0.755 262	0.798 311	2.135 919	2.423 959	1.594 458	1.797 107	1.613 516
		20 000	0.755 262	0.798 311	1.484 260	2.423 633	1.511 633	1.162 780	1.336 374
	R2	5 000	0.767 045	0.850 048	1.590 780	2.787 817	2.201 815	2.319 529	2.590 174
		10 000	0.738 244	0.806 822	1.470 112	2.363 048	1.686 311	1.276 485	1.880 933
		20 000	0.738 244	0.806 822	1.467 441	2.360 980	1.576 970	1.159 064	1.347 907

Tab.7 Convergence and verification results of system

采集点数	系统平均误差		校验点平均误差
采集点数	R1	R2	R1	R2
25	0.532 048	0.585 463	1.345 582	1.317 775
50	0.677 934	0.650 878	0.901 385	0.913 546
75	0.689 852	0.689 395	0.820 336	0.811 455
100	0.755 262	0.738 244	0.761 395	0.750 017

Fig.14 Three Gaussian white noise with different intensities

Tab.8 Fitness of system model after adding noise with different intensities

评价对象	σ=0.0	σ=0.5		σ=1.0		σ=1.5
评价对象	σ=0.0	优化前	优化后	优化前	优化后	优化前	优化后
系统整体	0.755	0.749	0.807	0.716	0.823	0.724	0.840
x轴	0.366	0.367	0.398	0.352	0.414	0.373	0.410
y轴	0.371	0.372	0.444	0.367	0.424	0.352	0.412
z轴	0.385	0.382	0.412	0.393	0.366	0.397	0.464

Fig.15 Convergence of AMECoDEs-LO under noise with different intensities

References 21

1	WEN B， TAO B， LI G F. Research status and tendency of intelligent industrial robot ［J］. Journal of Physics： Conference Series， 2019， 1302（3）： No.032050. 10.1088/1742-6596/1302/3/032050
2	LEVINE S， PASTOR P， KRIZHEVSKY A， et al. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection ［J］. The International Journal of Robotics Research， 2018， 37（4/5）： 421-436. 10.1177/0278364917710318
3	LI Q W， WANG Z Q， SHEN C W， et al. Method for horizontal alignment deviation measurement using binocular camera without common target ［J］. IEEE Access， 2020， 8： 184540-184549. 10.1109/access.2020.3029593
4	HAN Y， CHU Z N， ZHAO K. Target positioning method in binocular vision manipulator control based on improved Canny operator ［J］. Multimeia Tools and Application， 2020， 79： 9599-9614. 10.1007/s11042-019-08140-9
5	XIAO P F， JU H H， LI Q D， et al. A new fixed axis-invariant based calibration approach to improve absolute positioning accuracy of manipulators ［J］. IEEE Access， 2020， 8： 134224-134232. 10.1109/ACCESS.2020.3011328
6	LV Y F， NA J， YANG Q M， et al. Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics ［J］. International Journal of Control， 2016， 89（1）： 99-112. 10.1080/00207179.2015.1060362
7	WANG K S. Application of genetic algorithms to robot kinematics calibration ［J］. International Journal of Systems Science， 2009， 40（2）： 147-153. 10.1080/00207720802630644
8	XU X H， ZHU D H， ZHANG H Y， et al. TCP-based calibration in robot-assisted belt grinding of aero-engine blades using scanner measurements ［J］. The International Journal of Advanced Manufacturing Technology， 2017， 90（1/2/3/4）： 635-647. 10.1007/s00170-016-9331-8
9	吴庆华，万偲，李子奇，等. 机器人线结构光视觉引导系统的手眼标定方法［J］. 制造业自动化， 2021， 43（11）：29-34，40. 10.3969/j.issn.1009-0134.2021.11.008
	WU Q H， WAN C， LI Z Q， et al. Hand eye calibration method of robot line structured light vision guidance system ［J］. Manufacturing Automation，2021， 43（11）：29-34，40. 10.3969/j.issn.1009-0134.2021.11.008
10	HUA J， ZENG L C. Hand-eye calibration algorithm based on an optimized neural network ［J］. Actuators， 2021， 10（4）： No.85. 10.3390/act10040085
11	BAI J， ZHANG Y C， QIN X S， et al. Hybrid calibration and detection approach for mobile robotic manufacturing systems ［J］. Industrial Robot， 2020， 47（4）： 511-519. 10.1108/ir-09-2019-0194
12	WANG H X， LU X， HU Z Y， et al. A vision-based fully-automatic calibration method for hand-eye serial robot ［J］. Industrial Robot， 2015， 42（1）： 64-73. 10.1108/ir-06-2014-0352
13	MISEIKIS J， GLETTE K， ELLE O J， et al. Automatic calibration of a robot manipulator and multi 3D camera system ［C］// Proceedings of the 2016 IEEE/SICE International Symposium on System Integration. Piscataway： IEEE， 2016： 735-741. 10.1109/sii.2016.7844087
14	YIN S B， REN Y J， ZHU J G， et al. A vision-based self-calibration method for robotic visual inspection systems ［J］. Sensors， 2013， 13（12）： 16565-16582. 10.3390/s131216565
15	YANG C G， JIANG Y M， HE W， et al. Adaptive parameter estimation and control design for robot manipulators with finite-time convergence ［J］. IEEE Transactions on Industrial Electronics， 2018， 65（10）： 8112-8123. 10.1109/tie.2018.2803773
16	CUI L Z， LI G H， ZHU Z X， et al. Adaptive multiple-elites-guided composite differential evolution algorithm with a shift mechanism ［J］. Information Sciences， 2018， 422： 122-143. 10.1016/j.ins.2017.09.002
17	DENG W， SHANG S F， CAI X， et al. Quantum differential evolution with cooperative coevolution framework and hybrid mutation strategy for large scale optimization ［J］. Knowledge-Based Systems， 2021， 224： No.107080. 10.1016/j.knosys.2021.107080
18	AHN M S， CHAE H， NOH D， et al. Analysis and noise modeling of the Intel RealSense D435 for mobile robots ［C］// Proceedings of the 2019 16th International Conference on Ubiquitous Robots . Piscataway： IEEE， 2019： 707-711. 10.1109/urai.2019.8768489
19	ZHANG Z Y. A flexible new technique for camera calibration ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2000， 22（11）： 1330-1334. 10.1109/34.888718
20	TONG L Y， DONG M G， JING C. An improved multi-population ensemble differential evolution ［J］. Neurocomputing， 2018， 290： 130-147. 10.1016/j.neucom.2018.02.038
21	GUO S M， YANG C C， HSU P H， et al. Improving differential evolution with a successful-parent-selecting framework ［J］. IEEE Transactions on Evolutionary Computation， 2015， 19（5）： 717-730. 10.1109/tevc.2014.2375933

[1]	Mengdi SUN, Zhonggui SUN, Xu KONG, Hongyan HAN. Design of guided adaptive mathematical morphology for multimodal images [J]. Journal of Computer Applications, 2023, 43(2): 560-566.
[2]	Yuyu MENG, Jing GUO. Link prediction algorithm based on information entropy improved PCA model [J]. Journal of Computer Applications, 2022, 42(9): 2823-2829.
[3]	GAO Gong, YANG Hongyu, LIU Hong. 3D point cloud face recognition based on deep learning [J]. Journal of Computer Applications, 2021, 41(9): 2736-2740.
[4]	LI Xingfeng, HUANG Yuqing, REN Zhenwen, LI Yihong. Robust multi-view clustering algorithm based on adaptive neighborhood [J]. Journal of Computer Applications, 2021, 41(4): 1093-1099.
[5]	Hua LI, Guifu LU, Qinru YU. Manifold regularized nonnegative matrix factorization based on clean data [J]. Journal of Computer Applications, 2021, 41(12): 3492-3498.
[6]	Yunpeng GONG, Zhiyong ZENG, Feng YE. Person re-identification method based on grayscale feature enhancement [J]. Journal of Computer Applications, 2021, 41(12): 3590-3595.
[7]	LU Rongxiu, CHEN Mingming, YANG Hui, ZHU Jianyong. Element component content dynamic monitoring system based on time sequence characteristics of solution images [J]. Journal of Computer Applications, 2021, 41(10): 3075-3081.
[8]	PEI Yiyao, GUO Huiming, ZHANG Danpu, CHEN Wenbo. Robust 3D object detection method based on localization uncertainty [J]. Journal of Computer Applications, 2021, 41(10): 2979-2984.
[9]	XING Zhiwei, QIAO Di, LIU Hong’en, GAO Zhiwei, LUO Xiao, LUO Qian. Optimization method of airport gate assignment based on relaxation algorithm [J]. Journal of Computer Applications, 2020, 40(6): 1850-1855.
[10]	WANG Benjie, NONG Liping, ZHANG Wenhui, LIN Jiming, WANG Junyi. 3D point cloud classification and segmentation network based on Spider convolution [J]. Journal of Computer Applications, 2020, 40(6): 1607-1612.
[11]	LI Dongbo, HUANG Lyuwen. Reweighted sparse principal component analysis algorithm and its application in face recognition [J]. Journal of Computer Applications, 2020, 40(3): 717-722.
[12]	ZHANG Xiaobo, YANG Yan, LI Tianrui, LU Fan, PENG Lilan. Early diagnosis and prediction of Parkinson's disease based on clustering medical text data [J]. Journal of Computer Applications, 2020, 40(10): 3088-3094.
[13]	LAI Jie, WANG Xiaodan, LI Rui, ZHAO Zhenchong. Denoising autoencoder based extreme learning machine [J]. Journal of Computer Applications, 2019, 39(6): 1619-1625.
[14]	HUANG Guangqiu, XIE Rong. Cascading failure model of carbon emission spatial correlation system considering load overload [J]. Journal of Computer Applications, 2019, 39(6): 1829-1835.
[15]	ZHOU Fei, XIA Pengcheng. Signal strength difference fingerprint localization algorithm based on principal component analysis and chi-square distance [J]. Journal of Computer Applications, 2019, 39(5): 1405-1410.

Robust joint modeling and optimization method for visual manipulators

鲁棒的视觉机械臂联合建模优化方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 23

References 21

Related Articles 15

Recommended Articles

Metrics