2D & 3D Hand gesture recognition
A subtitle for this page should go here.
.JPG)
Publication 1
A ToF 3D Database for Hand Gesture Recognition
G. Simion and C. Caleanu
2012 10th International Symposium on Electronics and Telecommunications, Timisoara
2012, pp. 363-366.
Abstract
Although 3D information presents, in the context of hand gesture recognition (and not only), multiple advantages in comparison with the 2D counterpart, up-to-date there are very few 3D hand gesture databases. Recently, the Time-of-Flight (ToF) principle – employed in certain range imaging 3D cameras – became more and more attractive. According to it, the measurement distance is derived from the propagation time of the light pulse between the camera and the subject for each point of the image. In this paper, we describe the development of UPT ToF 3D Hand Gesture Database (UPT-ToF3D-HGDB). It represents, according to the best of our knowledge, the single database of this type which is publicly available.
Publication 2
Context
Although 3D information presents, in the context of hand gesture recognition (and not only), multiple advantages in comparison with the 2D counterpart, up-to-date there are very few 3D hand gesture databases. A new dataset, UPT-ToF3D-HGDB, is under construction at the Faculty of Electronics, Telecommunications and Information Technologies (ETcTI, www.etcti.upt.ro) - Applied Electronics Department - from Politehnica University from Timişoara. Having in mind that a high quality ToF video camera is still an expensive equipment (more than 5K €), the dataset is being constructed specifically to support researchers whom are willing to experiment 3D techniques.
-1-1200x678.webp)
Publication 3
Multi-stage 3D segmentation for ToF based gesture recognition system
G. Simion and C. Căleanu
11th International Symposium on Electronics and Telecommunications (ISETC)
Timisoara, 2014, pp. 1-4.
Abstract
The Human-Computer Interaction changed a lot in the last decade and it became gesture oriented. The up-to-date target is to develop smart interfaces which are free of contact. Hand is one of the favorite mean in this interaction because is a natural way in manipulating objects. This change was supported by the hardware evolution and the development of several 3D technologies. A critical step in any gesture recognition system is represented by the segmentation step. The current work proposes a new 3D segmentation method for the hand. The segmentation algorithm is based on depth information and cluster analysis. The algorithm was tested over the UPT-ToF3D-HGDB hand gesture database and it proves to be robust and fast.
Publication 4
Fingertip-based Real Time Tracking and Gesture Recognition for Natural User Interfaces
The widespread deployment of Natural User Interface (NUI) systems in smart terminals such as phones, tablets and TV sets has heightened the need for robust multi-touch, speech or facial recognition solutions. Air gestures recognition represents one of the most appealing technologies in the field. This work proposes a fingertip-based approach for hand gesture recognition. The novelty in the proposed system is the tracking principle, where an improved version of the multi-scale mode filtering (MSMF) algorithm is used, and in the classification step, where the proposed set of geometric features provides high discriminative capabilities. Empirically, we conducted an experimental study involving hand gesture recognition – on gestures performed by multiple persons against a variety of backgrounds – in which our approach achieved a global recognition rate of 95.66%.
Publication 5
A PointNet based Solution for 3D Hand Gesture Recognition
Gesture recognition represents nowadays an intensively researched area due to several factors. One of the most important reason is represented by the numerous applications of this technology in various domains like robotics, games, medicine or automotive. Additionally, the apparition of 3D image acquisition techniques (stereovision, projected-light, time-of-flight, etc.) overcomes the limitations of the traditional 2D approaches, and, combined with the larger availability of the 3D sensors (Microsoft Kinect, Intel RealSense, PMD CamCube, etc.) sparkled the interest in this domain. Moreover, in many computer vision tasks, the traditional/statistic top approaches were outperformed by deep neural network-based solutions. Having in view these considerations, we proposed a deep neural network solution, by employing the PointNet architecture, for the problem of hand gesture recognition form depth data produced by a Time of Flight sensor. We create a custom hand gesture dataset, then proposed a multistage hand segmentation by designing filtering, clustering, finding the hand in the volume of interest and hand-forearm segmentation. For comparison purpose, two equivalent datasets were tested: a 3D point cloud dataset and a 2D image dataset, obtained from the same stream. Besides the advantages of the 3D technology, the accuracy of the 3D method using PointNet is proven to outperform in all circumstances the 2D method that employs a deep neural network as well.