A comparative analysis of Indian sign language recognition using deep learning models

Saini, B; Venkatesh, D; Chaudhari, N; Shelake, T; Gite, S; Pradhan, B

A comparative analysis of Indian sign language recognition using deep learning models

Saini, B Venkatesh, D Chaudhari, N Shelake, T Gite, S Pradhan, B

Permalink

Publisher:: Whioce Publishing
Publication Type:: Journal Article
Citation:: Forum for Linguistic Studies, 2023, 5, (1), pp. 197-222
Issue Date:: 2023-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published versionAdobe PDF (3.98 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Saini, B
dc.contributor.author	Venkatesh, D
dc.contributor.author	Chaudhari, N
dc.contributor.author	Shelake, T
dc.contributor.author	Gite, S
dc.contributor.author	Pradhan, B https://orcid.org/0000-0001-9863-2054
dc.date.accessioned	2023-11-08T00:05:44Z
dc.date.available	2023-11-08T00:05:44Z
dc.date.issued	2023-01-01
dc.identifier.citation	Forum for Linguistic Studies, 2023, 5, (1), pp. 197-222
dc.identifier.issn	2705-0610
dc.identifier.issn	2705-0602
dc.identifier.uri	http://hdl.handle.net/10453/173217
dc.description.abstract	Sign language is a form of communication where people use bodily gestures, particularly those of hands and arms. This method of communication is put into motion when spoken communication is unattainable or disfavored. There are very few people who can translate sign language and readily understand them. It would be convenient for the hearing-impaired to have a platform where their sign language could be translated easily. Hence, through this study, with the help of artificial neural networks, we wish to compare how various widely implemented deep learning architectures respond to faultless translation of Indian sign language for the native audience. This research would streamline the development of software tools that can accurately predict or translate ISL. For the purpose of understanding the method of training the machine and exploring our model’s performance without any optimizations, a Convolutional Neural Network architecture was implemented. Over the course of our research, there have been several Pre-trained Transfer Learning Models implemented that have yielded promising results. The research aims to contrast how various convolutional neural networks perform while translating Indian Sign Actions on a custom dataset that factors in illumination, angles, and different backgrounds to provide a balanced and distinctive set of images. The goal of this study is to make clear comparisons between the various deep learning frameworks. Hence, a fresh Indian sign language dataset is introduced. Since every dataset in the field of deep learning has special properties that may be utilized for the betterment of the existing models, the development of a fresh dataset could be viewed as a development in the field. The optimum model for our task: classification of these gestures is found to be ResNet-50 (Accuracy = 98.25% and F1-score = 99.34%), and the least favorable was InceptionNet V3 (Accuracy = 66.75%, and F1-score = 70.89%).
dc.language	en
dc.publisher	Whioce Publishing
dc.relation.ispartof	Forum for Linguistic Studies
dc.relation.isbasedon	10.18063/FLS.V5I1.1617
dc.rights	info:eu-repo/semantics/restrictedAccess
dc.title	A comparative analysis of Indian sign language recognition using deep learning models
dc.type	Journal Article
utslib.citation.volume	5
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Civil and Environmental Engineering
pubs.organisational-group	/University of Technology Sydney/Strength - CAMGIS - Centre for Advanced Modelling and Geospatial lnformation Systems
utslib.copyright.status	open_access	*
dc.date.updated	2023-11-08T00:05:43Z
pubs.issue	1
pubs.publication-status	Published
pubs.volume	5
utslib.citation.issue	1

Abstract:

Sign language is a form of communication where people use bodily gestures, particularly those of hands and arms. This method of communication is put into motion when spoken communication is unattainable or disfavored. There are very few people who can translate sign language and readily understand them. It would be convenient for the hearing-impaired to have a platform where their sign language could be translated easily. Hence, through this study, with the help of artificial neural networks, we wish to compare how various widely implemented deep learning architectures respond to faultless translation of Indian sign language for the native audience. This research would streamline the development of software tools that can accurately predict or translate ISL. For the purpose of understanding the method of training the machine and exploring our model’s performance without any optimizations, a Convolutional Neural Network architecture was implemented. Over the course of our research, there have been several Pre-trained Transfer Learning Models implemented that have yielded promising results. The research aims to contrast how various convolutional neural networks perform while translating Indian Sign Actions on a custom dataset that factors in illumination, angles, and different backgrounds to provide a balanced and distinctive set of images. The goal of this study is to make clear comparisons between the various deep learning frameworks. Hence, a fresh Indian sign language dataset is introduced. Since every dataset in the field of deep learning has special properties that may be utilized for the betterment of the existing models, the development of a fresh dataset could be viewed as a development in the field. The optimum model for our task: classification of these gestures is found to be ResNet-50 (Accuracy = 98.25% and F1-score = 99.34%), and the least favorable was InceptionNet V3 (Accuracy = 66.75%, and F1-score = 70.89%).

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/173217