David Rozado would like to help people access computers even if they can't use a mouse or keyboard.
Modern technologically driven societies rely on information technology for almost everything: social interaction, work, entertainment, education, and banking for example. This means that people who are unable to use a mouse or keyboard are severely limited in their ability to engage with society. This affects people with a range of conditions including motor neuron disease, a spinal cord injury, some congenital conditions, severed limbs, or stroke.
“Due to the relatively small size of the severely motor-impaired community and the large fixed costs of developing accessibility software, commercial accessibility software solutions are often relatively expensive,” says David.
“Until recently, open-source accessibility software solutions have been relatively scarce and the existing ones do not match the range of commercial programs.
“We have leveraged capstone projects in our Bachelor of Information Technology (BIT) degree programme to guide students towards the creation of a range of open-source accessibility software solutions.”
David Rozado's vision is to provide low-cost user-friendly accessibility software for this small but high-needs group, relying on their facial gestures, voice prompts and eye movement. Development on these projects is continuing, and the accessibility software is open source to allow skilled software developers to add features or improve existing functionality.
“The software is available through an online repository to anyone “regardless of financial circumstances”, David says. “During 2017 the students and I refined and polished what had been done the previous year. In 2018 we are going to try to expand the software to control the home environment (e.g. television, lights, air conditioning) via an interface that expands our gaze control software.”
Examples of the applications created by teams of Otago Polytechnic Bachelor of Information technology students under David's guidance are:
Gaze Control Toolbar
This software allows the user to fully control a Windows computer using only their eye movements and a low-cost eye tracker (less than NZ$150). The interaction is achieved using a two-step process by which the user selects a desired computer control task (e.g. left button mouse click, scrolling, etc) by gazing at a reactive taskbar on the border of the computer screen. The software also allows the user to augment the system via accessibility switches (e.g. to speed up interaction).
A gaze-reactive on-screen keyboard that allows motor-impaired users who cannot read or write to communicate with their eyes, using a hierarchy of gaze-aware semantic icons and a speech synthesizer.
FaceSwitch is an application that transforms predefined facial gestures to specific keyboard keystrokes. The program leverages a deformable face-tracker to create a multi-switch system driven by face gestures. FaceSwitch is intended to help computer users with limited mobility from the neck down but who have a good level of control of facial muscles by allowing those users to use facial gestures as on-off switches to control accessibility software.