Real Time Hand Sign Language Detection using Tensorflow Object Detection API
Hand sign language is important for those disabled people to understand what others are trying to say or to express their feelings. However, not everyone can understand what the meaning of each hand sign language is representing. That’s why real-time hand sign language detection comes in handy for this situation. We can make it using Tensorflow Object Detection API. First, we need to take the sample image data of the hand sign language that we would like to detect and label the gesture meaning onto the image accordingly. After that, we are using TensorFlow to train the model. Once we trained the model until it has at least 80% accuracy, we can deploy this detection system in the practical. At a certain stage, we only can detect the static motion sign language. In the future, this technology can be further expanded to detect motion language so that it is more convenient for detection of the hand sign language.
In this model, we can detect the following hand sign language:
Pose available
● hello
● thank you
● I love you
● Like
● Dislike
● Hear
● Wakanda Forever
● Peace(v sign)
● ok sign
● to left
● to right
● to up
● to down
● GDSC sign
The technology behind it is deep learning. To be more specific, it is Tensorflow technology which is the machine learning technology developed by Google to develop an AI solution. For the training part, we only achieved total_loss with 0.464364 which is we can improve it in the future with better equipment. The training took a lot of computing power such as CPU and GPU. Hence, the computer specs play an important role if we want to train a deep learning model.
We also use tensorboard to visualize our training result. Tensorboard is an interactive website provided by Tensorflow to visualize the machine learning model training result.
For now we are more interested in evaluation matrics, and we can visualize how’s the expected prediction looks like:
About Us
Google Developer Student Clubs (GDSC) is a program presented by Google Developers. DSCs are university-based community groups for students. Students from all undergraduate or graduate programs with an interest in growing as a developer are welcome
