Solution uses brain signals and eye movements to articulate thoughts through an avatar
A graduation project developed by a team from Rochester Institute of Technology (RIT) of Dubai has delivered an innovative solution to help verbally impaired patients to communicate through an avatar-based artificial intelligence model. The project is the work of Emirati electrical engineering graduate, Abdallah Al-Serkal and classmate Ahmad Yousef who spent two years developing and refining the technology during their undergraduate degree studies.
Using electro-encephalography (EEG) and electro-oculography (EOG) technology to capture brain signals and eye movements, the solution is able to extract linguistic content to help people with speech-related disabilities to communicate in an alternative way. The words are articulated through an interactive avatar, which dynamically delivers the user’s thoughts through textual or visual expression.
Explaining the background to the innovation, Al-Serkal said, “The project started out as a lab assignment earlier in our program, which we were encouraged to continue working on by our professor. The goal is to address communication challenges faced by patients suffering from paralysis, such as stroke survivors, to enhance their quality of life.”
Discussing the development of the prototype, Al-Serkal continued, “The process involved data collection among 50 participants of different ages and backgrounds, to capture the micro neural and ocular responses that occur when thinking about different words. The test vocabulary focussed on terms related to patient wellbeing, such as help, pain, medication and sleep. We used machine learning to translate the bio signals into words and to test and validate the results. The user’s thoughts are then communicated by the innovative AI avatar interface we developed.”
The technology behind the innovation is also the subject of an academic paper authored by the development team and published by the Institute of Electrical and Electronics Engineers. Supervising the team throughout the project were Associate Professors of Electrical Engineering, Dr. Jinane Mounsef and Dr. Boutheina Tlili.
Dr. Mounsef said, “It is vital to work on projects, in the university that have the potential to enhance wellbeing, such as this one focused on using AI to interpret brain signals and help patients communicate essential needs like hunger or help.”
This drive to support public health and wellbeing has provided the impetus for RIT Dubai to develop a micromaster program in health informatics and health management, which will launch soon. Dr. Mounsef added, “At RIT, we foster an environment where technical innovation meets academic rigor, which empowers students to apply their knowledge for real-world impact. This project exemplifies how we strive to combine theory and practice to create solutions that improve the human condition.”
Dr. Yousef Al Assaf, President of RIT Dubai added: “One of our main responsibilities is to nurture an entrepreneurial environment and to support students to secure sponsors for such innovative projects that start in Dubai and have the potential for positive impact across the globe.”
Al-Serkal, who is now pursuing a master’s degree in engineering at RIT Dubai alongside his work in industry, hopes that ultimately the technology will advance to the stage of real world application. He concluded, “The accuracy rate achieved with the prototype averaged 90% overall, which was higher than we had anticipated. We hope that we can enhance this in the future with more testing in a specially controlled environment. We are also exploring grant funding opportunities to help develop the technology even further.”
To find out more about Rochester Institute of Technology of Dubai please,