2021.Nov.05

REPORTS

The 1st BAIRAL Research Meeting for 2021 Report on “The Development of the Dictionary Enabling Deaf Children to Look up Basic Japanese Verbs from Japanese Sign Language”

On April 24, 2021, BAIRAL invited members of a research project that develops a dictionary for deaf children by using deep learning. This group, a part of the “Easy Japanese Project,” consists of researchers from language education and linguistic research, teachers at Meisei Gakuen which is a school for deaf children, and engineers at BBED (Bilingual Bicultural Education Center for Deaf Children) that develops bilingual educational methods for deaf children. We, including 80 audiences, communicated with a translation between (spoken) Japanese and Japanese Sign Language (JSL) with support from Meisei Gakuen.

 

Masaru Otsuka from BBED, Akira Morita from Meisei Gakuen and Asuka Ando from Graduate School of Arts and Sciences at the University of Tokyo gave a presentation, and Prof. Isao Iori from Hitotsubashi University, who is the project leader, added some supplementary information. Mr. Morita explained that deafness is a culture, not a disability, so we should understand the Deaf as cultural/linguistic minorities who use sign language as a first language. We also learned that JSL and signed Japanese (sign made from the Japanese language) have different origins and grammatical structures. Most hearing people understand that “sign language” is the latter, but it is the former that is used in the Deaf community. Therefore, Ms. Ando emphasized that deaf children need a dictionary that can be read in JSL to learn the Japanese language as their second language. The project has tried to meet this need with AI since 2018 because a paper-based dictionary is not adequate for the conditions.

 

Mr. Otsuka mainly reported on the technical side. They chose 30 basic verbs and recorded sign language talked by the teachers at Meisei Gakuen 20 times each with a tablet computer. As a result, the system succeeded in recognizing 7 of them as practical level. Additionally, in the demonstration with ten deaf children, it can identify 3-4 verbs. According to their report, the children enjoyed this system like a game as of now, not as the practical dictionary.

 

In the latter part of the meeting, we discussed many things: concrete technological problems like recognizing movement of back and forth; practical tasks for enhancing practical usability like whether AI can cope with structural differences in vocabulary patterns between JSL and Japanese language. AI may also contribute to the realization of social fairness, but disability issues have not been dealt with as multicultural issues in the discussion regarding AI. The collaboration of people with disabilities and AI engineers will become increasingly important even though the project will need many years to be realized.