Jul 9 2013
Latest advances in capturing data on brain activity and eye movement are being combined to open up a host of ‘mindreading’ possibilities for the future. These include the potential development of a system that can detect when drivers are in danger of falling asleep at the wheel.
An eye-tracking, brain monitoring experiment in progress. The infra-red camera is on the small black console on the desk in front of the main PC screen
The research has been undertaken at the University of Leicester with funding from the Engineering and Physical Sciences Research Council (EPSRC), and in collaboration with the University of Buenos Aires in Argentina.
The breakthrough involves bringing two recent developments in the world of technology together: high-speed eye tracking that records eye movements in unprecedented detail using cutting-edge infra-red cameras*; and high-density electroencephalograph** (EEG) technology that measures electrical brain activity with millisecond precision through electrodes placed on the scalp.
The research has overcome previous technological challenges which made it difficult to monitor eye movement and brain activity simultaneously. The team has done this by developing novel signal processing techniques.
This could be the first step towards a system that combines brain and eye monitoring to automatically alert drivers who are showing signs of drowsiness. The system would be built into the vehicle and connected unobtrusively to the driver, with the EEG looking out for brain signals that only occur in the early stages of sleepiness. The eye tracker would reinforce this by looking for erratic gaze patterns symptomatic of someone starting to feel drowsy and different from those characteristic of someone driving who is constantly looking out for hazards. Fatigue has been estimated to account for around 20 per cent of traffic accidents on the UK’s motorways.***
The breakthrough achieved by the University of Leicester could also ultimately be built on to deliver many other everyday applications in the years ahead. For example:
- Computer games of the future could dispense with the need for the player to physically interact with any type of console, mouse or other hand-operated system. Instead, eye movement and brain activity data would be collected and processed to indicate what action the player wants to take. By distinguishing the tiny differences in various types of brain activity, the EEG would identify the precise action the player desires (e.g. run, jump or throw), while the eye movement data would show exactly where on the screen the player was looking when they had this thought. This information could be combined to enable the correct action to occur. An unobtrusive headset would be all that would be required to capture the necessary data.
- People who have no arm functionality could move their wheelchairs simply through their eye movements. These movements could be tracked and the corresponding brain activity analysed to identify when these indicate a desire to move in a certain direction. This would then automatically activate a steering and propulsion mechanism that would drive the wheelchair to that place.
- The breakthrough could also provide the basis for improved tests to diagnose dyslexia and other reading disorders. Current tests revolve around a rapid succession of single words flashed onto a computer screen, with the resulting brain activity monitored by EEG. The new technique could enable the person being tested to move their eyes and read longer passages of text in a natural way, making the tests much more realistic and revealing.
With the basic concept now demonstrated successfully, the team aim to continue their work and eventually develop software that, in real time, automatically monitors both eye movement and brain activity.
Dr Matias Ison, who has led the research, says: “Historically, eye-tracking and EEG have evolved as independent fields. We have managed to overcome the challenges that were standing in the way of integrating these technologies. This is already leading to a much better understanding of how the brain responds when the eyes are moving. Monitoring the alertness of drivers is just one of many potential applications for this work. Building on the foundation provided by our EPSRC-funded project, we hope to see the first of these starting to become feasible within the next three to five years.”
Notes for Editors
The 14-month research project, ‘Bridging the Gap between Eye Movements and Event-related Potentials’, received total EPSRC funding of just over £101,000.
The experiments were conducted at the University of Leicester’s new Centre for Systems Neuroscience.
* The infra-red cameras used in this project worked in conjunction with arrays of red light-emitting diodes (LEDs); these LEDs illuminated the subject's eyes, with the camera sensing this reflection.
** An electroencephalograph (EEG) measures neural activity within the brain. It is usually attached to the head via electrodes. Conventional EEG has been in use for several decades.
*** Source: Department for Transport. http://webarchive.nationalarchives.gov.uk/20120606181145/ http://www.dft.gov.uk/publications/rsrr-theme3-literature-review/
The Engineering and Physical Sciences Research Council (EPSRC) is the UK’s main agency for funding research in engineering and the physical sciences. EPSRC invests around £800 million a year in research and postgraduate training, to help the nation handle the next generation of technological change. The areas covered range from information technology to structural engineering, and mathematics to materials science. This research forms the basis for future economic development in the UK and improvements for everyone’s health, lifestyle and culture. EPSRC works alongside other Research Councils with responsibility for other areas of research. The Research Councils work collectively on issues of common concern via Research Councils UK. www.epsrc.ac.uk