Volume 13, Issue 3 (Journal of Control, V.13, N.3 Fall 2019)                   JoC 2019, 13(3): 29-39 | Back to browse issues page


XML Persian Abstract Print


Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Rajabi S, Mousavinia A. Persian sign language detection based on normalized depth image information. JoC 2019; 13 (3) :29-39
URL: http://joc.kntu.ac.ir/article-1-613-en.html
1- K.N. Toosi University
Abstract:   (6150 Views)
There are many reports of using the Kinect to detect hand and finger gestures after release of device by Microsoft. The depth information is mostly used to separate the hand image in the two-dimension of RGB domain. This paper proposes a method in which the depth information plays a more dominant role. Using a threshold in depth space first the hand template is extracted. Then in 3D domain the perpendicular vector to the hand surface is found. Using the rotation matrix all the rotations along three axes are compensated in a way that the camera z- coordinate lies perpendicular to hand surface. Then the resulted 3d image is translated to a distance of 80 to 100 cm from the Kinect. Wavelet transform with a new descriptor, called Circular Descriptor are used to extract required features. A trained MLP neural network in conjunction with a SVM is used to classify the signs. Empirical results show an average accuracy of 96.7 % with a two seconds delay for online recognition of Persian Sign Language.
Full-Text [PDF 898 kb]   (2705 Downloads)    
Type of Article: Review paper | Subject: Special
Received: 2018/09/1 | Accepted: 2018/12/31 | Published: 2019/12/31

References
1. 1] Moghadam M., Nahvi, M. Hassanzadeh, 2011, "Static Persian Sign Language Recognition Using Kernel-Based Feature Extraction", 7th Iranian IEEE Machine Vision and Image Processing (MVIP). [DOI:10.1109/IranianMVIP.2011.6121539]
2. [2] Wang, L. C., Wang, R., Kong, D., Yin, B., 2014, "Similarity Assessment Model for Chinese Sign Language Videos", IEEE Transaction on Multimedia, 16, pp.751 - 761. [DOI:10.1109/TMM.2014.2298382]
3. [3] Huang, J., Zhou, W., Li, H., Li, W., 2015, "Sign language recognition using 3d 1430 convolutional neural networks", Multimedia and Expo (ICME), IEEE International Conference on IEEE, pp.1-6.
4. [4] Agris, U, V., Zieren, J., Canzler, U., Bauer, B., Kraiss, K. 2008, "Recent developments in visual sign language recognition", Universal Access in the Information Society, 6(4), pp.323-362. [DOI:10.1007/s10209-007-0104-x]
5. [5] Shah, N, K., Rathod, R, K., Agravat, J, S. 2014, "A survey on Human Computer Interaction Mechanism Using Finger Tracking", International Journal of Computer Trends and Technology (IJCTT). 7(3), pp. 174-177. [DOI:10.14445/22312803/IJCTT-V7P148]
6. [6] Verma, H. V., Aggarwal, E., Chandra, S., 2013, "Gesture recognition using kinect for sign language translation", IEEE Second International Conference on IEEE Image Information Processing (ICIIP). [DOI:10.1109/ICIIP.2013.6707563]
7. [7] Yang, H.D., 2014, "Sign language recognition with the kinect sensor based on conditional random fields", Sensors 15(1), pp.135-147. [DOI:10.3390/s150100135]
8. [8] Sun, C., Zhang, T., Bao, B., Xu, C., Mei, T., 2013, "Discriminative Exemplar Coding for Sign Language Recognition with Kinect", IEEE Transaction on Cybernetics, 43, pp.1418 - 1428. [DOI:10.1109/TCYB.2013.2265337]
9. [9] Li, S.Z., Yu, B., Wu, W., Su, S.Z., Ji, R.R., 2015, "Feature learning based on SAE-PCA network for human gesture recognition in rgbd images", Neurocomputing, 151, pp.565-573. [DOI:10.1016/j.neucom.2014.06.086]
10. [10] Liu, T., Zhou, W., Li, H., 2016, "Sign language recognition with long short-term memory", IEEE International Conference on Image Processing (ICIP), pp.2871-2875. [DOI:10.1109/ICIP.2016.7532884]
11. [11] Almeida, S.G.M., Guimar˜aes, F.G., Ram'ırez, J.A., 2014, "Feature extraction in Brazilian sign language recognition based on phonological structure and using rgb-d sensors", Expert Systems with Applications, 41, pp.7259-7271. [DOI:10.1016/j.eswa.2014.05.024]
12. [12] Lim, M.K., Tan, W.C. A., Tan, C.S., 2016, "A feature covariance matrix with serial particle filter for isolated sign language recognition", Expert Systems with Applications, 54, pp.208-218. [DOI:10.1016/j.eswa.2016.01.047]
13. [13] Zhao, R., Martinez, M. A., 2016, "Labeled Graph Kernel for Behavior Analysis", IEEE Transaction on Pattern Analysis and Machine Intelligence, 38, pp.1640 - 1650. [DOI:10.1109/TPAMI.2015.2481404]
14. [14] Elakkiya, R., Selvamani, K. 2017, "Enhanced dynamic programming approach for subunit modelling to handle segmentation and recognition ambiguities in sign language", Journal of Parallel and Distributed Computing.
15. [15] Karami, A., Zanj B., Kiani A. 2011, "Persian sign language (PSL) recognition using wavelet transform and neural networks", Expert Systems with Applications, 38(3), pp.2661-2667. [DOI:10.1016/j.eswa.2010.08.056]
16. [16] Wang, C., Liu, Z., Chan S., 2015, "Superpixel-Based Hand Gesture Recognition with Kinect Depth Camera", IEEE Transaction on Multimedia, 17, pp. 29 - 39. [DOI:10.1109/TMM.2014.2374357]
17. [17] Neiva D. H., Zanchettin C. 2018, "Gesture Recognition: a Review Focusing on Sign Language in a Mobile Context", Expert Systems with Applications.
18. [18] Albrecht, I., Haber, J., Seidel, H. 2003, "Construction and animation of anatomically based human hand models", Proceedings of ACM SIGGRAPH / Eurographics Symposium on Computer Animation SCA '03, pp.98-109.
19. [19] Caillette, F., Galata, A., Howard, T., 2008, "Real-time 3-d human body tracking using learnt models of behavior", CVIU, 109(2), pp 112-125. [DOI:10.1016/j.cviu.2007.05.005]
20. [20] Francke, H., Solar J, R., Verschae, R., 2007, "Real time hand gesture detection and recognition using boosted classifiers and active learning", Advances in Image and Video Technology, 4872, pp.533-547. [DOI:10.1007/978-3-540-77129-6_47]
21. [21] Krotosky, S., Trivedi, M., 2006, "Registration of Multimodal Stereo Images Using Disparity Voting from Correspondence Windows", IEEE International Conference on Video and Signal Based Surveillance. pp. 91-91. [DOI:10.1109/AVSS.2006.98]
22. [22] Markelj, P., Tomaževič, D., Likar, B., Pernuša, F. 2012, "A review of 3D/2D registration methods for image-guided interventions", Medical image analysis, 16(3), pp.642-661. [DOI:10.1016/j.media.2010.03.005]
23. [23] Pizzoli, M., Forster, C., Scaramuzza, D. 2014, "REMODE: Probabilistic, monocular dense reconstruction in real time", IEEE International Conference on Robotics and Automation (ICRA). [DOI:10.1109/ICRA.2014.6907233]
24. [24] Craig, J, J. 2009, Introduction to Robotics: Mechanics and Control (3rd Edition).
25. [25] Van den Bergh, M., Van Gool L., 2011, "Combining RGB and ToF cameras for real-time 3D hand gesture interaction", IEEE Workshop on Applications of Computer Vision (WACV). [DOI:10.1109/WACV.2011.5711485]
26. [26] Vogl, T. P., Mangis, J.K., Rigler, A.K., Zink, W.T., Alkon, D.L. 1988, "Accelerating the convergence of the backpropagation method", Biological Cybernetics, 59, pp.257-263. [DOI:10.1007/BF00332914]
27. [27] M. T., Demuth, H. B., Beale, M. H., De Jesús O., Neural network design, 2nd edition, Hagan, (2014),
28. [28] Duda, R. O., Hart, P. E., Stork, D. G., 1973, "Pattern classification", Journal of Classification, 24, pp.305-307.
29. [29] Fukunaga, Keinosuke, Introduction to statistical pattern recognition, Elsevier, 2013.

Add your comments about this article : Your username or Email:
CAPTCHA

Send email to the article author


Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

© 2024 CC BY-NC 4.0 | Journal of Control

Designed & Developed by : Yektaweb