Pengembangan Deteksi Realtime untuk Bahasa Isyarat Indonesia dengan Menggunakan Metode Deep Learning Long Short Term Memory dan Convolutional Neural Network
Abstract
Kehadiran Computer Vision mampu mempengaruhi bidang kajian Sign Language Recognition System (SLRS). Adapun penelitian dibidang SLRS terhadap dua standar bahasa isyarat di Indonesia yaitu standar SIBI (Sistem Bahasa Isyarat Indonesia) dan BISINDO (Bahasa Isyarat Indonesia). Tantangan dalam penelitian ini adalah kendala dalam memproses gambar dinamis dan gambar statis ketika setelah melalui preprocessing rekognisi. Perlakuan yang berbeda saat recognisi awal pada gambar bergerak dengan gambar statis mempengaruhi waktu memunculkan hasil dengan cepat sehingga dibutuhkan model dengan training yang baik dan cepat. Tujuan penelitian ini adalah untuk mengetahui faktor akurasi yang mempengaruhi tingkat akurasi penerapan objek deteksi dan klasifikasi gambar maupun video secara realtime untuk BISINDO (Bahasa Isyarat Indonesia) dengan menggunakan metode Deep Learning Long Short Term Memory (LSTM) dan Convolution Neural Network (CNN). Pentingnya penelitian ini karena hasilnya dapat dijadikan dasar untuk mempercepat pengembangan lebih lanjut aplikasi sign language recognition khusus untuk BISINDO yang bisa dimanfaatkan oleh penyandang disabilitas maupun masyarakat agar komunikasi dua arah lebih mudah dilakukan dimasa depan secara real-time.
Full Text:
PDFReferences
Aljabar, A., & Suharjito. (2020). BISINDO (Bahasa isyarat indonesia) sign language recognition using CNN and LSTM. Advances in Science, Technology and Engineering Systems, 5(5), 282–287. https://doi.org/10.25046/AJ050535
Aloysius, N., & Geetha, M. (2020). A scale space model of weighted average CNN ensemble for ASL fingerspelling recognition. International Journal of Computational Science and Engineering, 22(1), 154–161. https://doi.org/10.1504/IJCSE.2020.107268
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8(1), 1–74. https://doi.org/10.1186/s40537-021-00444-8
Amin, M., Hefny, H., & Mohammed, A. (2021). Sign Language Gloss Translation using Deep Learning Models. International Journal of Advanced Computer Science and Applications, 12(11), 686–692. https://doi.org/10.14569/IJACSA.2021.0121178
Chevtchenko, S. F., Vale, R. F., Macario, V., & Cordeiro, F. R. (2018). A convolutional neural network with feature fusion for real-time hand posture recognition. Applied Soft Computing Journal, 73, 748–766. https://doi.org/10.1016/j.asoc.2018.09.010
Damatraseta, F., Novariany, R., & Ridhani, M. A. (2021). Real-time BISINDO Hand Gesture Detection and Recognition with Deep Learning CNN. Jurnal Informatika Kesatuan, 1(1), 71–80. https://doi.org/10.37641/jikes.v1i1.774
Hesamian, M. H., Jia, W., He, X., & Kennedy, P. (2019). Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges. Journal of Digital Imaging, 32(4), 582–596. https://doi.org/10.1007/s10278-019-00227-x
Ikram, K., Khairunizam, W., Aziz, A. A., Bakar, S. A., Razlan, Z. M., Zunaidi, I., & Desa, H. (2018). Adaptive gesture recognition based on human physical characteristic. Proceedings - 2018 IEEE 14th International Colloquium on Signal Processing and Its Application, CSPA 2018, March, 129–134. https://doi.org/10.1109/CSPA.2018.8368699
Irwanto, Kasim, E. R., Fransiska, A., Lusli, M., & Siradj, O. (2010). Analisis Situasi Penyandang Disabilitas di Indonesia: Sebuha Desk Review. In Pusat Kajian Disabilitas Fakultas Ilmu Sosial dan Politik UI (Vol. 1, Issue S2). https://doi.org/10.5694/j.1326-5377.1981.tb135719.x
Kenstantinidis, D., Dimitropoulos, K., & Daras, P. (2018). A Deep Learning Approach for Analyzing Video and Skeletal Features in Sign Language Recognition. IEEE, 1–6.
Kurniawan, A. A., & Mustikasari, M. (2021). Implementasi Deep Learning Menggunakan Metode CNN dan LSTM untuk Menentukan Berita Palsu dalam Bahasa Indonesia. Jurnal Informatika Universitas Pamulang, 5(4), 544. https://doi.org/10.32493/informatika.v5i4.6760
Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
Mohammed, A. A. Q., Lv, J., & Islam, M. D. S. (2019). A deep learning-based end-to-end composite system for hand detection and gesture recognition. Sensors (Switzerland), 19(23), 1–23. https://doi.org/10.3390/s19235282
Mujahid, A., Awan, M. J., Yasin, A., Mohammed, M. A., Damaševičius, R., Maskeliūnas, R., & Abdulkareem, K. H. (2021). Real-time hand gesture recognition based on deep learning YOLOv3 model. Applied Sciences (Switzerland), 11(9), 1–15. https://doi.org/10.3390/app11094164
Papastratis, I., Chatzikonstantinou, C., Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2021). Artificial intelligence technologies for sign language. Sensors, 21(17), 1–25. https://doi.org/10.3390/s21175843
Rakun, E., Andriani, M., Wiprayoga, I. W., Danniswara, K., & Tjandra, A. (2013). Combining depth image and skeleton data from Kinect for recognizing words in the sign system for Indonesian language (SIBI [Sistem Isyarat Bahasa Indonesia]). 2013 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2013, 387–392. https://doi.org/10.1109/ICACSIS.2013.6761606
Rastgoo, R., Kiani, K., & Escalera, S. (2021). Sign Language Recognition: A Deep Survey. Expert Systems with Applications, 164(February 2020), 113794. https://doi.org/10.1016/j.eswa.2020.113794
Schmidhuber, J. (2014). Deep Learning in neural networks: An overview. Neural Networks, 61, 85–117. https://doi.org/10.1016/j.neunet.2014.09.003
Shahriar, S., Siddiquee, A., Islam, T., Ghosh, A., Chakraborty, R., Khan, A. I., Shahnaz, C., & Fattah, S. A. (2019). Real-Time American Sign Language Recognition Using Skin Segmentation and Image Category Classification with Convolutional Neural Network and Deep Learning. IEEE Region 10 Annual International Conference, Proceedings/TENCON, 2018-Octob(October), 1168–1171. https://doi.org/10.1109/TENCON.2018.8650524
Sharma, S., & Singh, S. (2021). Vision-based hand gesture recognition using deep learning for the interpretation of sign language. Expert Systems with Applications, 182(July), 1–12. https://doi.org/10.1016/j.eswa.2021.115657
Subburaj, S., & Murugavalli, S. (2022). Survey on sign language recognition in context of vision-based and deep learning. Measurement: Sensors, 23(May), 1–11. https://doi.org/10.1016/j.measen.2022.100385
Suharjito, Gunawan, H., Thiracitta, N., & Nugroho, A. (2019). Sign Language Recognition Using Modified Convolutional Neural Network Model. 1st 2018 Indonesian Association for Pattern Recognition International Conference, INAPR 2018 - Proceedings, 1–5. https://doi.org/10.1109/INAPR.2018.8627014
Suharjito, Thiracitta, N., & Gunawan, H. (2021). SIBI Sign Language Recognition Using Convolutional Neural Network Combined with Transfer Learning and non-trainable Parameters. Procedia Computer Science, 179(2019), 72–80. https://doi.org/10.1016/j.procs.2020.12.011
Supria, S., Herumurti, D., & Khotimah, W. N. (2016). Pengenalan Sistem Isyarat Bahasa Indonesia Menggunakan Kombinasi Fitur Statis Dan Fitur Dinamis Lmc Berbasis L-Gcnn. JUTI: Jurnal Ilmiah Teknologi Informasi, 14(2), 217. https://doi.org/10.12962/j24068535.v14i2.a574
Tarimo, W., Sabra, M. M., & Hendre, S. (2020). Real-Time Deep Learning-Based Object Detection Framework. IEEE Symposium Series on Computational Intelligence (SSCI), 1829–1836. https://medium.com/@arifwicaksanaa/pengertian-use-case-a7e576e1b6bf
Vo, A. H., Pham, V. H., & Nguyen, B. T. (2019). Deep learning for Vietnamese Sign Language recognition in video sequence. International Journal of Machine Learning and Computing, 9(4), 440–445. https://doi.org/10.18178/ijmlc.2019.9.4.823
Wang, Z., Zhao, T., Ma, J., Chen, H., Liu, K., Shao, H., Wang, Q., & Ren, J. (2022). Hear Sign Language: A Real-Time End-to-End Sign Language Recognition System. IEEE Transactions on Mobile Computing, 21(7), 2398–2410. https://doi.org/10.1109/TMC.2020.3038303
Yang, J., Zhu, C., & Yuan, J. (2017). Real time hand gesture recognition via finger-emphasized multi-scale description. Proceedings - IEEE International Conference on Multimedia and Expo, July, 631–636. https://doi.org/10.1109/ICME.2017.8019348
DOI: https://doi.org/10.37012/jtik.v9i1.1272
Refbacks
- There are currently no refbacks.
Copyright (c) 2023 eka altiarika
This work is licensed under a Creative Commons Attribution 4.0 International License.
Address:
Universitas Mohammad Husni Thamrin
Jl. Raya Pd. Gede No.23-25, RT.2/RW.1, Dukuh, Kec. Kramat jati, Kota Jakarta Timur, Daerah Khusus Ibukota Jakarta 13550
Jurnal Teknologi Informatika & Komputer Mohammad Husni Thamrin is licensed under a Creative Commons Attribution 4.0 International License.