基于Kinect的无标记手部姿态估计系统

南京大学学报(自然科学版) ›› 2015, Vol. 51 ›› Issue (2): 297–.

基于Kinect的无标记手部姿态估计系统

周文猛1，杨一品1，周余1，于耀1，金苏文2，都思丹1*

出版日期:2015-03-31 发布日期:2015-03-31
作者简介:(1.南京大学电子科学与工程学院,南京,210023;2.上海协同科技,上海,200063)
基金资助:
国家自然科学基金(61100111,61300157,61201425,61271231)

Markerless hand pose estimation system using Kinect

Zhou Wenmeng1,Yang Yipin1, Zhou Yu1, Yu Yao1, Jin Suwen2, Du Sidan1*

Online:2015-03-31 Published:2015-03-31
About author:(1.School of Electronic Science and Engineering, Nanjing University, Nanjing, 210023; 2.Shanghai Xietong Technology, Shanghai, 200063)

摘要/Abstract

摘要： 随着计算机技术的发展，人机交互的方式越来越多样化，声音、手势、肢体动作等都可以用来向计算机传递信息。本文实现的手部姿态估计系统，使用Kinect作为输入设备，在无标记的情况下对人体手部的姿态进行估计、跟踪，恢复出人体手部的姿态信息。本文采用参数化的手部三维模型，提出了有效的评价函数衡量三维模型投影结果与真实观测值之间的差异，使用改进的粒子群寻优算法求解该评价函数的最优解，进而得到手部姿态信息。针对手部运动帧间的连续性，使用前一帧的优化信息初始化当前帧粒子群，提高了粒子群优化算法的收敛速度，此外，还利用了粒子群算法的内在并行特性，使用GPU并行加速，并针对手部图像处理这一具体任务加以优化，达到了20Hz的处理速度。

Abstract: As computer technology is developing, more and more human-computer interaction methods come out, such as voice control, hand control and human body control. In this paper, we present a novel approach to recover and track the 3D position, orientation and the full articulated information of a human hand from a video sequence obtained by a Kinect sensor. By using Particle Swarm Optimization (PSO) variant to minimize the cost function which quantifies the discrepancy between projected model and the ground truth observations , we can get the motion parameters of the hand observed by the Kinect sensor. In order to accelerate PSO, continuity over frame sequence is exploited by setting the initial states of particles of current frame to the optimized ones of the previous frame. Moreover, GPU acceleration is adopted due to the inherent parallelization and optimized for images processing of hands. The overall system does not need any markers or special environment and c an be performed in 20Hz

周文猛1，杨一品1，周余1，于耀1，金苏文2，都思丹1*. 基于Kinect的无标记手部姿态估计系统[J]. 南京大学学报(自然科学版), 2015, 51(2): 297–.

Zhou Wenmeng1,Yang Yipin1, Zhou Yu1, Yu Yao1, Jin Suwen2, Du Sidan1*. Markerless hand pose estimation system using Kinect[J]. Journal of Nanjing University(Natural Sciences), 2015, 51(2): 297–.

参考文献

[1] Wang R Y, Popovi? J. Real-time hand tracking with a color glove. ACM Transactions on Graphics, 2009, 28(3): 63.

[2] Kortier H G, Antonsson J, Schepers H M, et al. Hand pose estimation by fusion of inertial and magnetic sensing aided by a permanent magnet. IEEE Transactions on Neural Systems and Rehabilitation Engineering ,2014

[3] Erol A, Bebis G, Nicolescu M, et al. Vision-based hand pose estimation: A review. Computer Vision and Image Understanding, 2007, 108(1): 52-73.

[4] Romero J, Kjellstrom H, Kragic D. Monocular real-time 3D articulated hand pose estimation. Humanoid Robots, 2009: 87-92.

[5] Doliotis P, Athitsos V, Kosmopoulos D, et al. Hand shape and 3d pose estimation using depth data from a single cluttered frame. Advances in Visual Computing, 2012: 148-158.

[6] Keskin C, K?ra? F, Kara Y E, et al. Real time hand pose estimation using depth sensors. Consumer Depth Cameras for Computer Vision, 2013: 119-137.

[7] De La Gorce M, Fleet D J, Paragios N. Model-based 3d hand pose estimation from monocular video. Pattern Analysis and Machine Intelligence, 2011, 33(9): 1793-1805.

[8] Oikonomidis I, Kyriazis N, Argyros AA. Efficient model-based 3D tracking of hand articulations using Kinect. BMVC. 2011, 1(20): 33-38.

[9] Tompson J, Stein M, Lecun Y, et al. Real-time continuous pose recovery of human hands using convolutional networks. ACM Transactions on Graphics, 2014, 33(5): 169-177.

[10] Liang H, Yuan J, Thalmann D, et al. Model-based hand pose estimation via spatial-temporal hand parsing and 3D fingertip localization. The Visual Computer, 2013, 29(6-8): 837-848.

[11] Albrecht I, Haber J, Seidel H P. Construction and animation of anatomically based human hand models. Proceedings of the 2003 ACM Eurographics symposium on Computer animation, 2003: 98-109.

[12] Rodman L. Topics in Quaternion Linear Algebra. Princeton University Press, 2014,123-135.

[13] Kinect, http://www.xbox.com/en-US/kinect.

[14] Zhang Z. A flexible new technique for camera calibration. Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.

[15] Tang H, Feng Z. Hand’s skin detection based on ellipse clustering. Computer Science and Computational Technology, 2008( 2): 758-761.

[16] Kennedy J.“Swarm intelligence,” Handbook of Nature Inspired and Innovative Computing, pp. 187–219, 2006.

[17] Yasuda T, Matsumura Y, Ohkura K , Extended pso with partial randomization for large scale multimodal problems, World Automation Congress , 2010, 1–6.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed