Journal of Computer Applications ›› 2016, Vol. 36 ›› Issue (12): 3486-3491.DOI: 10.11772/j.issn.1001-9081.2016.12.3486

Previous Articles     Next Articles

Operation control method for industrial robots based on hand gesture recognition

JIANG Suifeng, LI Yanchun, XIAO Nanfeng   

  1. School of Computer Science and Engineering, South China University of Technology, Guangzhou Guangdong 510006, China
  • Received:2016-06-01 Revised:2016-07-11 Online:2016-12-10 Published:2016-12-08
  • Supported by:
    This work is partially supported by the Special Funds for Public Welfare Research and Capacity Building in Guangdong Province (2014B010104001), the Natural Science Foundation of Guangdong Province (2015A030308018).


蒋穗峰, 李艳春, 肖南峰   

  1. 华南理工大学 计算机科学与工程学院, 广州 510006
  • 通讯作者: 李艳春
  • 作者简介:蒋穗峰(1992-),男,湖南岳阳人,硕士研究生,主要研究方向:工业机器人、智能计算;李艳春(1981-),女,湖南涟源人,讲师,博士研究生,主要研究方向:机器学习、计算机视觉、深度学习;肖南峰(1962-),男,江西南昌人,教授,博士生导师,博士,主要研究方向:人工智能、工业机器人。
  • 基金资助:

Abstract: The human-computer interaction modes between operators and industrial robots are rather mechanized currently. In order to solve the problem, a hand gesture control method by using Kinect sensor as a hand gesture acquisition equipment to control industrial robots was proposed. Firstly, the method of combining depth threshold algorithm and hand bones points was used to extract the hand gesture images accurately from the data obtained by a Kinect infrared camera. In the process of extraction, the operator did not need to wear any equipment, while it had no requirements to operator location and background environment. Then the method of combining deep autoencoder network and Softmax classifier was used for hand gesture image recognition. The hand gesture recognition included pretraining and fine tuning. The greedy layerwise approach was leveraged to train each layer of network in turn in pretraining, while all layers of the neural network were treated as a whole to fine tune the parameters of the entire network in fine tuning. The hand gesture recognition accuracy was up to 99.846%. Finally, the experiments were conducted on self-developed industrial robot simulation platform, the good results had been achieved in one hand and both hands gestures. The experimental results show that the proposed method by using hand gesture to control the industrial robot is feasible and available.

Key words: industrial robot, Kinect, hand gesture recognition, autoencoder network, neural network

摘要: 针对目前操作工人与工业机器人之间的交互还是采用比较机械化的交互方式,设计使用Kinect传感器作为手势采集设备,并使用人的手势来对工业机器人进行控制的方法。首先,使用深度阈值法与手部骨骼点相结合的方法,从Kinect传感器获取的数据中准确地提取出手部图像。在提取过程中,操作员无需佩戴任何设备,对操作员所站位置没有要求,对背景环境也没要求。然后,用稀疏自编码网络与Softmax分类器结合的方法对手势图像进行识别,手势识别过程包含预训练和微调,预训练是用逐层贪婪训练法依次训练网络的每一层,微调是将整个神经网络看成一个整体微调整个网络的参数,手势识别的准确率达到99.846%。最后,在自主研发的工业机器人仿真平台上进行实验,在单手和双手手势下都取得了不错的效果,实验结果验证了手势控制工业机器人的可行性和可用性。

关键词: 工业机器人, Kinect, 手势识别, 自编码网络, 神经网络

CLC Number: