A Fully Embedded Adaptive Real-Time Hand Gesture Classifier Leveraging HD-sEMG and Deep Learning

IEEE Trans Biomed Circuits Syst. 2020 Apr;14(2):232-243. doi: 10.1109/TBCAS.2019.2955641. Epub 2019 Nov 25.

Abstract

This paper presents a real-time fine gesture recognition system for multi-articulating hand prosthesis control, using an embedded convolutional neural network (CNN) to classify hand-muscle contractions sensed at the forearm. The sensor consists in a custom non-intrusive, compact, and easy-to-install 32-channel high-density surface electromyography (HDsEMG) electrode array, built on a flexible printed circuit board (PCB) to allow wrapping around the forearm. The sensor provides a low-noise digitization interface with wireless data transmission through an industrial, scientific and medical (ISM) radio link. An original frequency-time-space cross-domain preprocessing method is proposed to enhance gesture-specific data homogeneity and generate reliable muscle activation maps, leading to 98.15% accuracy when using a majority vote over 5 subsequent inferences by the proposed CNN. The obtained real-time gesture recognition, within 100 to 200 ms, and CNN properties show reliable and promising results to improve on the state-of-the-art of commercial hand prostheses. Moreover, edge computing using a specialized embedded artificial intelligence (AI) platform ensures reliable, secure and low latency real-time operation as well as quick and easy access to training, fine-tuning and calibration of the neural network. Co-design of the signal processing, AI algorithms and sensing hardware ensures a reliable and power-efficient embedded gesture recognition system.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Artificial Limbs*
  • Deep Learning*
  • Electromyography / instrumentation*
  • Equipment Design
  • Forearm / physiology
  • Gestures*
  • Hand / physiology*
  • Humans
  • Muscle, Skeletal / physiology
  • Signal Processing, Computer-Assisted / instrumentation