南京大学学报(自然科学版) ›› 2023, Vol. 59 ›› Issue (4): 590–599.doi: 10.13232/j.cnki.jnju.2023.04.006

• • 上一篇    下一篇

基于液态金属复合材料传感手环的手势意图识别系统

陈怡1, 唱睿喆1, 曹雨冬1, 张世武1,2, 孙帅帅1,2(), 贠国霖3   

  1. 1.中国科学技术大学工程科学学院, 合肥, 230022
    2.中国科学技术大学?爱博智能机器人联合实验室, 合肥, 230022
    3.剑桥大学剑桥石墨烯中心, 英国 剑桥, CB3 0FA
  • 收稿日期:2023-03-30 出版日期:2023-07-31 发布日期:2023-08-18
  • 通讯作者: 孙帅帅 E-mail:sssun@ustc.edu.cn
  • 基金资助:
    国家自然科学基金青年科学基金(52005474)

Gesture Intention recognition system based on liquid metal composites⁃enabled sensor bracelet

Yi Chen1, Ruizhe Chang1, Yudong Cao1, Shiwu Zhang1,2, Shuaishuai Sun1,2(), Guolin Yun3   

  1. 1.School of Engineering Science, University of Science and Technology of China, Hefei, 230022, China
    2.University of Science and Technology of China ? AiBLE Joint Laboratory of Intelligent Robots, Hefei, 230022, China
    3.Cambridge Graphene Centre, University of Cambridge, Cambridge, CB3 0FA, UK
  • Received:2023-03-30 Online:2023-07-31 Published:2023-08-18
  • Contact: Shuaishuai Sun E-mail:sssun@ustc.edu.cn

摘要:

手势意图识别是人机交互领域的热门研究方向,然而现有手势的识别系统大多基于肌电信号,肌电信号不可避免的信号串扰、衰减、信噪比低等问题严重影响了手势识别的准确率.为了解决这一问题,研发了基于液态金属复合材料传感手环的可穿戴手势识别系统,液态金属复合材料灵敏的压阻效应使设计的传感手环获取的传感信号表现出稳定、灵敏度高、噪声低等优异特性.基于此传感信号的手势识别系统由数据采集和模式识别两个部分组成,其平均离线识别准确率高达97.19%.更重要的是,该系统无须在手部加装设备,通过前臂肌肉即可获取信号,因此可用于手部功能缺失的残疾群体,用户范围广泛,具有重要的社会和经济效益.

关键词: 液态金属复合材料, 压力传感, 手势意图识别, 智能系统

Abstract:

Gesture intention recognition is a popular research direction in the field of Human?Computer Interaction. However,most of the gesture recognition systems are based on electromyography (EMG) signals. The inevitable problems of EMG signals,such as signal crosstalk,signal attenuation,and low signal to interference plus noise ratio,seriously affect the accuracy of hand gesture recognition. To solve this problem,this paper develops the gesture intention recognition system based on liquid metal composites enabled sensor bracelet. Thanks to the sensitive piezoresistive effect of the liquid metal composites,signals obtained by the sensing bracelet exhibit excellent characteristics such as stability,high sensitivity and low noise. The gesture recognition system based on this signal consists of data acquisition and pattern recognition,and its average offline recognition accuracy is 97.19%. What's more,this system does not require additional equipment in the hand and only acquires signals through the forearm muscles. Therefore,the system can be used for disabled groups with hand function deficiency. The system has a wide range of users and important social and economic benefits.

Key words: liquid metal composites, pressure sensing, gesture intention recognition, intelligent system

中图分类号: 

  • TH702

图1

LMMRE材料的制备"

图2

LMMRE传感单元结构1.传感盒体;2.绝缘胶带;3.导电银胶带;4.电极;5.LMMRE材料;6.加压球;7.传感盒盖"

图3

LMMRE传感单元在有/无加压球时的相对电压变化?压力曲线"

图4

LMMRE传感器手环实物图"

图5

基于LMMRE传感的手势识别系统结构"

图6

数据文件的结构"

图7

LDA和SVM分类器的应用原理"

表1

参与实验的志愿者概况"

志愿者编号性别年龄身高(cm)体重(kg)BMI
S1251706121.11
S2231707325.26
S3301775818.51
S4231788025.25
S5231504520.00
S6271685218.42

图8

传感器的佩戴"

图9

实验手势流程示意图"

图10

六名志愿者的离线识别准确率"

图11

七种手势的离线识别准确率"

图12

六名志愿者基于LDA/SVM分类器的混淆矩阵"

1 Cho E, Chen R, Merhi L K,et al. Force myography to control robotic upper extremity prostheses:A feasibility study. Frontiers in Bioengineering and Biotechnology2016(4):18.
2 Young A J, Simon A M, Fey N P,et al. Intent recognition in a powered lower limb prosthesis using time history information. Annals of Biomedical Engineering201442(3):631-641.
3 Sasidhar S, Panda S K, Xu J X. A wavelet feature based mechanomyography classification system for a wearable rehabilitation system for the elderly∥The 11th International Conference on Smart Homes and Health Telematics. Springer Berlin Heidelberg,2013:45-52.
4 Dipietro L, Ferraro M, Palazzolo J J,et al. Customized interactive robotic treatment for stroke:EMG?triggered therapy. IEEE Transactions on Neural Systems and Rehabilitation Engineering200513(3):325-334.
5 Fang Y F, Liu H H, Li G F,et al. A multichannel surface EMG system for hand motion recognition. International Journal of Humanoid Robotics201512(2):1550011.
6 Guo W C, Sheng X J, Liu H H,et al. Toward an enhanced human?machine interface for upper?limb prosthesis control with combined EMG and NIRS signals. IEEE Transactions on Human?Machine Systems201747(4):564-575.
7 Zwarts M J, Stegeman D F. Multichannel surface EMG:Basic aspects and clinical utility. Muscle & Nerve200328(1):1-17.
8 Akhlaghi N, Baker C A, Lahlou M,et al. Real?time classification of hand motions using ultrasound imaging of forearm muscles. IEEE Transactions on Biomedical Engineering201663(8):1687-1698.
9 Huang Y J, Yang X C, Li Y F,et al. Ultrasound?based sensing models for finger motion classification. IEEE Journal of Biomedical and Health Informatics201822(5):1395-1405.
10 Castellini C, Passig G, Zarka E. Using ultrasound images of the forearm to predict finger positions. IEEE Transactions on Neural Systems and Rehabilitation Engineering201220(6):788-797.
11 Chen B X, Yu L X, Meng X,et al. Visual gesture recognition based on hand key points. Journal of Physics:Conference Series2021(2024):012037.
12 Bertsch F A, Hafner V V. Real?time dynamic visual gesture recognition in human?robot interaction∥2009 9th IEEE?RAS International Conference on Humanoid Robots. Paris,France:IEEE,2009:447-453.
13 Vinjamuri R, Mao ZH, Sclabassi R,et al. Limitations of surface EMG signals of extrinsic muscles in predicting postures of human hand∥2006 International Conference of the IEEE Engineering in Medicine and Biology Society. New York,NY,USA:IEEE,2006:5491-5494.
14 Honda Y, Weber S, Lueth T C. Intelligent recognition system for hand gestures∥2007 3rd International IEEE/EMBS Conference on Neural Engineering. Kohala Coast,HI,USA:IEEE,2007:611-614.
15 Shi J, Guo J Y, Hu S X,et al. Recognition of finger flexion motion from ultrasound image:A feasibility study. Ultrasound in Medicine & Biology201238(10):1695-1704.
16 Li N, Yang D P, Jiang L,et al. Combined use of FSR sensor array and SVM classifier for finger motion recognition based on pressure distribution map. Journal of Bionic Engineering20129(1):39-47.
17 Jiang X T, Merhi L K, Menon C. Force exertion affects grasp classification using force myography. IEEE Transactions on Human?Machine Systems201848(2):219-226.
18 Zhang Q T, Yun G L, Zhao B L,et al. Highly stretchable and sensitive strain sensor based on liquid metal composite for wearable sign language communication device. Smart Materials and Structures202130(11):115005.
19 Yun G, Tang S Y, Lu H D,et al. Hybrid‐filler stretchable conductive composites:From fabrication to application. Small Science20211(6):2000080.
20 Yun G L, Tang S Y, Zhao Q B,et al. Liquid metal composites with anisotropic and unconventional piezoconductivity. Matter20203(3):824-841.
21 Yun G L, Tang S Y, Lu H D,et al. Liquid metal hybrid composites with high?sensitivity and large dynamic range enabled by micro? and macrostructure engineering. ACS Applied Polymer Materials20213(10):5302-5315.
22 Yun G L, Tang S Y, Sun S S,et al. Liquid metal?filled magnetorheological elastomer with positive piezoconductivity. Nature Communications201910(1):1300.
23 Crepin R, Fall C L, Mascret Q,et al. Real?time hand motion recognition using sEMG patterns classification∥2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Honolulu,HI,USA:IEEE,2018:2655-2658.
24 Al Omari F, Hui J, Mei C L,et al. Pattern recognition of eight hand motions using feature extraction of forearm EMG signal. Proceedings of the National Academy of Sciences,India Section A:Physical Sciences201484(3):473-480.
25 Tharwat A, Gaber T, Ibrahim A,et al. Linear discriminant analysis:A detailed tutorial. AI Communications201730(2):169-190.
26 Amsuss S, Goebel P M, Jiang N,et al. Self?correcting pattern recognition system of surface EMG signals for upper limb prosthesis control. IEEE Transactions on Biomedical Engineering201461(4):1167-1176.
No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!