Four electrical engineering students explore possibilities for nonoral communication in capstone electroencephalogram project
For people with brain injuries, communication can come in, and with, a blink. And, for a quartet of fresh Norwich University graduates, blinks plus engineering may equal better lives for people incapable of oral speech.
In a study dubbed Neuro-Comm and advised by electrical and computer engineering professor David Feinauer, the Class of 2020’s Madeline Long, Mary Mota, Rafael Pereira and Christian Thompson used blink pattern data to fuel a machine learning system, aiming to determine whether electrical brain signals could enable communication.
To yield the data, a test subject wore an electrode-studded headset which covered his head like a shower cap. The machine learning system used this data, based on blink duration (long, short), to identify patterns that were associated with Morse code characters (letters) to build Morse code sequences (words).
“The students seemed to have a pretty good grasp of the basic idea of how (machine learning) worked … and that with more work, they could have taken it much further.” Dr. Michael Prairie, Norwich University electrical and computer engineering professor
The project was sponsored by Electrical and Computer Engineering Department Chairman Michael Prairie, who said he had long been interested in technologies to help disability- or disease-impeded people. He’d belonged to the Rehabilitation Engineering and Assistive Technology Society of North America and attended its conferences. Prairie said he and fellow Norwich engineering professor David Feinauer saw Neuro-Comm as a way to integrate machine learning into the engineering curriculum. (Feinauer has since left Norwich for another professorship.)
Machine learning, a form of predictive modeling, is a computer science offshoot that uses computer algorithms that improve automatically through experience. Machine learning builds new data and learns from existing data, aiming to create models that find patterns and give accurate predictions. The predictions could be as simple as knowing whether an email is or isn’t spam, whether a pictured fruit is a banana or an apple or whether the objects moving in front of a self-driving car are other cars or people.
Although machine learning projects are often reserved for graduate students, Prairie said he and Feinauer deemed this group fit to try one. Long and Pereira had proved themselves motivated when they volunteered for a small project involving a voice recognition chip the year before. They pounced at the chance to continue research started by neuroscience major and 2020 classmate Olivia Bloom.
Bloom, who has entered the U.S. Marine Corps, had worked to use brain signals from EEGs to see how concussions affect rapid decision-making and to use basic function tests to determine at what level brain damage was treatable. Prairie said Bloom introduced the Neuro-Comm quartet to her research and lab protocols.
Prairie and Feinauer’s four students, who became Norwich engineering’s first group to complete a machine learning capstone project, followed the ‘V model’ used in engineering, which steps from initial concept to integration and product implementation. They imported prerecorded EEG blink data into MATLAB, a computer language, to process the data using a machine-learning algorithm.
The algorithm used the timing and durations of blinks in the data to predict which Morse code letters were intended, then generated sequences that would be translated to English.
Pereira said the students learned about brain function from neuroscience professors, settling on blinks as a data source. The students tapped previous MATLAB experience to develop the script to test the data and train the model.
In an email, Pereira said the team divided (tasks) and conquered. Sometimes half the team would work in the lab to improve data collection as the other half preprocessed code. When campus closed for coronavirus curve flattening, the team kept working through group and video chats on Microsoft Teams.
“We pretty much transitioned from meeting in person regularly to meeting virtually regularly,” Long wrote in a separate email.
Pereira added, “As long as we were on the same page mentally (knowing our tasks/what needed to be done by a deadline), the project could progress smoothly without many hiccups.”
Pereira, who developed the project’s preprocessing strategy and finalized its machine learning methodology, said many answers came through trial and error as data were labeled and recorded and the machine learning was trained.
“Little by little, small breakthroughs were made that kept the project moving forward,” he wrote in an email. “At times, however, different parts of the project could be worked on simultaneously, by other members of the group. In theory, if both parts work, you can just plug them together like a puzzle piece.”
The power of small
Long said Norwich’s small size contributed to the project’s success. The students could tap the wisdom of the university’s professors, and those professors could share in the students’ joy of discovery.
“On one occasion, we had made a huge breakthrough in our script and when we told our adviser, Professor Feinauer, and he was just as excited as we were and encouraged us to go further … We knew that no matter what time of day, Prof F. would be just as excited as we were when we made progress.”
Prairie said Neuro-Comm and projects like it inform faculty curriculum decisions. If small groups of students can handle new challenges like machine learning, they reasoned, maybe larger groups can handle it in courses. The projects also aid admissions, he added, showing prospective students that Norwich offers learning in emerging technologies.
“The students seemed to have a pretty good grasp of the basic idea of how this worked,” Prairie said, “being exposed to (machine learning) and knowing what it’s capable of … and that with more work, they could have taken it much further.”
The now-graduated students, meanwhile, developed career-applicable skills. Long, for example, said she’ll use her newfound knowledge in her new job as a systems engineer with Waltham, Massachusetts-based Raytheon Technologies.
“I know this project has helped me to understand the basics of machine learning and signal analysis. I hope that this experience has taught me how to work with all types of people,” she said. “Both of these skills can be applied throughout my career.”
Pereira said he could see his capstone experience feeding a “next big thing” project for machine learning or artificial intelligence.
“Next step is just to get out there and earn some experience using everything I’ve learned from Norwich University,” he said.
EXPLORE:
- Electrical and computer engineering study at Norwich University
- David Crawford School of Engineering
- College of Professional Schools
READ MORE:
