Deteksi Objek Video Bahasa Isyarat Untuk Anak Tuna Rungu dan Tuna Wicara Menggunakan YOLOv8

Authors

  • Muhammad Maheza Fresmanda Universitas Widyagama
  • Istiadi Universitas Widyagama
  • Syahroni Wahyu Iriananda Universitas Widyagama

DOI:

https://doi.org/10.53697/jkomitek.v4i2.1895

Keywords:

YOLOv8, Gerakan tangan, BISINDO

Abstract

Penelitian ini bertujuan untuk mengimplementasikan keberhasilan model algoritma YOLOv8 dalam mendeteksi abjad bahasa isyarat Indonesia (BISINDO) berbasis video. BISINDO merupakan adaptasi American Sign Language (ASL) yang disesuaikan dengan budaya Indonesia agar lebih mudah digunakan. Itu bergantung pada gerakan dua tangan untuk komunikasi. Model YOLOv8 mampu mendeteksi dan mengklasifikasikan bahasa isyarat Indonesia dengan tingkat akurasi yang cukup baik, mencapai precision 93,35%, recall 70,25%, mAP50 64,05% dan mAP50-95 45,88%. Temuan ini diharapkan dapat memberikan kontribusi positif dalam pendidikan dan komunikasi untuk anak tunarungu dan tunawicara

References

Marília Prata. (2020). Bisindo Indonesian Sign Language MP4. Https://Www.Kaggle.Com/Code/Mpwolke/Bisindo-Indonesian-Sign-Language-Mp4/Notebook.

Mulyana, D. I., Lazuardi, M. F., & Yel, M. B. (2022). Deteksi Bahasa Isyarat Dalam Pengenalan Huruf Hijaiyah Dengan Metode YOLOV5. Jurnal Teknik Elektro Dan Komputasi (ELKOM), 4(2), 145–151. https://doi.org/10.32528/elkom.v4i2.8145

Rachardi, F. (2020). Deteksi Gambar Gestur Kosakata Bahasa Isyarat Indonesia dengan Convolutional Neural Network. Institutional Repository UIN Syarif Hidayatullah, 192. https://repository.uinjkt.ac.id/dspace/handle/123456789/56075

Rasuandi, M. (2023). PENGENALAN ALFABET A-Z BAHASA ISYARAT AMERICAN SIGN LANGUAGE MENGGUNAKAN HALAMAN PENGESAHAN SKRIPSI PENGENALAN ALFABET A-Z BAHASA ISYARAT AMERICAN SIGN LANGUAGE MENGGUNAKAN METODE SUPPORT VECTOR MACHINE.

Susanty, M., Fadillah, R. Z., & Irawan, A. (2021). Model Penerjemah Bahasa Isyarat Indonesia (BISINDO) Menggunakan Pendekatan Transfer Learning. Petir, 15(1), 1–9. https://doi.org/10.33322/petir.v15i1.1289

Abdullah, A. (2019). Strategi Penanganan Guru Pendidikan Agama Islam Bagi Anak Berkebutuhan Khusus Di Sekolah Dasar Luar Biasa Negeri Keleyan Socah Bangkalan. Al-Ibrah: Jurnal Pendidikan Dan Keilmuan Islam. https://www.ejournal.stital.ac.id/index.php/alibrah/article/view/81

Ahmad, N. (2023). Transforming Sign Language using CNN Approach based on BISINDO Dataset. 2023 International Conference on Informatics, Multimedia, Cyber and Information Systems, ICIMCIS 2023, 543–548. https://doi.org/10.1109/ICIMCIS60089.2023.10349011

Andreas, R. (2023). Mobile Application for Children to Learn BISINDO Sign Language. 6th International Conference on Inventive Computation Technologies, ICICT 2023 - Proceedings, 774–780. https://doi.org/10.1109/ICICT57646.2023.10134183

Astriani, M. S. (2023). TELEMEDICINE SIGN LANGUAGE CLASSIFICATION FOR COVID-19 PATIENTS WITH DISABILITY BASED ON LSTM MODEL. Journal of Engineering Science and Technology, 18, 140–152.

Candra, A. (2024). Development of Machine Learning-based Sign Language Translator for Bahasa Isyarat Indonesia (BISINDO). AIP Conference Proceedings, 2987(1). https://doi.org/10.1063/5.0199747

Caraka, R. E. (2024). Empowering deaf communication: a novel LSTM model for recognizing Indonesian sign language. Universal Access in the Information Society. https://doi.org/10.1007/s10209-024-01095-1

Elyondri, N., & Azizah, N. (2023). Analisis Pengembangan Komunikasi, Persepsi, Bunyi, dan Irama (PKPBI) Anak Tunarungu dan Kebutuhan Media Pembelajarannya. Jurnal Obsesi: Jurnal Pendidikan Anak Usia …. https://www.academia.edu/download/108761726/pdf.pdf

Enri, U. (2023). Sign Language Detection Using Mediapipe and Long-Short Term Memory Network. 2023 International Conference on Informatics, Multimedia, Cyber and Information Systems, ICIMCIS 2023, 617–622. https://doi.org/10.1109/ICIMCIS60089.2023.10349016

Izzalhaqqi, M. Y. D. (2023). Gesture Recognition in Indonesian Sign Language Using Hybrid Deep Learning Models. Proceedings - IWIS 2023: 3rd International Workshop on Intelligent Systems. https://doi.org/10.1109/IWIS58789.2023.10284666

Joan, D. (2023). BISINDO Hand-Sign Detection Using Transfer Learning. 8th International Conference on Recent Advances and Innovations in Engineering: Empowering Computing, Analytics, and Engineering Through Digital Innovation, ICRAIE 2023. https://doi.org/10.1109/ICRAIE59459.2023.10468194

Limantoro, E. (2023). Indonesian Sign Language Recognition Using Kinect Sensor Based on Deep Neural Network. AIP Conference Proceedings, 2689(1). https://doi.org/10.1063/5.0114983

Maheswara, Y. D. (2023). Real-Time BISINDO Sign Language Recognition: A Dynamic Approach with GRU and LSTM Models Leveraging MediaPipe. 6th International Seminar on Research of Information Technology and Intelligent Systems, ISRITI 2023 - Proceeding, 226–232. https://doi.org/10.1109/ISRITI60336.2023.10467586

Rafikayati, A., & Jauhari, M. N. (2020). Studi tentang Pelaksanaan Pelatihan Terapi Wicara Anak Tunarungu Usia 3-5 Tahun untuk Orangtua dalam Setting Blended Learning di SLB Karya Mulia Surabaya. Jurnal Leverage, Engagement …. https://journal.uc.ac.id/index.php/LeECOM/article/view/1417

Saputra, M. A. (2024). Recognizing Indonesian sign language (Bisindo) gesture in complex backgrounds. Indonesian Journal of Electrical Engineering and Computer Science, 36(3), 1583–1593. https://doi.org/10.11591/ijeecs.v36.i3.pp1583-1593

Sujatmiko, D. (2023). AlexNet Architecture Based Convolution Neural Network for Realtime Audio to Text Translator of Bisindo Hand Sign. 2023 International Seminar on Application for Technology of Information and Communication: Smart Technology Based on Industry 4.0: A New Way of Recovery from Global Pandemic and Global Economic Crisis, ISemantic 2023, 429–434. https://doi.org/10.1109/iSemantic59612.2023.10295322

Sutjiadi, R. (2023). Android-Based Application for Real-Time Indonesian Sign Language Recognition Using Convolutional Neural Network. TEM Journal, 12(3), 1541–1549. https://doi.org/10.18421/TEM123-35

Syah, A. R. (2022). APLIKASI PENERJEMAH BAHASA ISYARAT MENGGUNAKAN METODE K-NN (K-NEARST NEIGHBOUR). Jurnal Teknologi Pintar. http://teknologipintar.org/index.php/teknologipintar/article/view/143

Yap, S. (2023). Enhancing BISINDO Recognition Accuracy Through Comparative Analysis of Three CNN Architecture Models. Proceedings of 2023 International Conference on Information Management and Technology, ICIMTech 2023, 732–737. https://doi.org/10.1109/ICIMTech59029.2023.10277780

Zikky, M., Akbar, Z. F., & Utomo, S. (2019). Kamus sistem isyarat bahasa Indonesia (KASIBI) dengan voice recognition sebagai pendukung belajar bahasa isyarat berbasis android. JST (Jurnal Sains Terapan). http://jurnal.poltekba.ac.id/index.php/jst/article/view/732

Downloads

Published

2024-10-18

How to Cite

Maheza Fresmanda, M., Istiadi, & Syahroni Wahyu Iriananda. (2024). Deteksi Objek Video Bahasa Isyarat Untuk Anak Tuna Rungu dan Tuna Wicara Menggunakan YOLOv8. Jurnal Komputer, Informasi Dan Teknologi, 4(2). https://doi.org/10.53697/jkomitek.v4i2.1895

Issue

Section

Articles