Sep 28 2018
Duke University engineers have come up with a novel way to extract a series of images from light scattered through a mostly opaque material, or even off a wall, from a single, long photographic exposure. This method can be used in a variety of fields ranging from healthcare to security to astronomy.
Duke’s Michael Gehm has figured out a way to gather scattered light from multiple images on a single exposure, separate them, and then reconstruct each individual image. (Credit: Duke University)
The study has been reported in the journal, Scientific Reports.
“When I explain to people what this algorithm can do, it sounds like magic,” stated Michael Gehm, associate professor of electrical and computer engineering at Duke. “But it’s really just statistics and a ton of data.”
As light travels via a translucent material, it gets scattered and the emerging pattern of “speckle” appears as random as static on a television screen without any signal. However, the speckle pattern is not random. Since the light emerging from one point of an object passes a path which is just like that of the light emerging from a neighboring point, the speckle pattern from each point appears quite the same, and just slightly shifted.
With sufficient images, astronomers traditionally employ this “memory effect” phenomenon to generate more distinct images of the heavens through a chaotic atmosphere, but the object being imaged has to be adequately compact.
However, with the development of adaptive optics, the method fell out of favor. These optics perform the same job by utilizing adjustable mirrors to offset the scattering. Conversely, some years ago, the memory effect phenomenon once again became popular with researchers. Since contemporary cameras are capable of recording hundreds of millions of pixels at a time, only one exposure is required to make the statistics work.
Although this method is capable of reconstructing a scattered image, it also has certain drawbacks. The scattering medium needs to be constant and the object has to stay motionless.
By extracting a series of images from a one, long exposure. Gehm’s latest approach to memory effect imaging overcomes these drawbacks. The trick is to utilize a coded aperture. Consider this as a set of filters that enable the passage of light through certain areas but not others in a particular pattern. Knowing this pattern will allow researchers to computationally extract what the original image appeared like.
A sequence of coded apertures is used by Gehm’s latest method to stamp which light is emerging from which moment in time. However, since each image is gathered on one, long photographic exposure, the ensuing speckle becomes even more of a disorderly mess than usual.
“People thought that the resulting speckle pattern would be too random to separate out the individual frames,” stated Gehm. “But it turns out that today’s cameras have such amazing resolution that if you look closely, there’s still enough of a pattern to computationally get a toehold and tease them apart.”
In their experiment, an easy sequence of four backlit letters emerged consecutively behind a scattering material and a coded aperture. During the sequence, the shutter of a 5.5-megapixel CCD camera was left open for over a minute to collect the images.
Although, a 100-second exposure time provided the best results, much shorter exposure times could still provide good results. Following just a few seconds of processing, the computer effectively returned each image of a D, U, K, and E from the sequence. The team subsequently demonstrated that the method also works upon changing the scattering medium, and even when scattering mediums as well as the images are changing.
When the letters appeared for 25 seconds each, they provided the best results. This is because the backlight intensity was not extremely high to begin with, and was much further reduced by the scattering material and coded aperture. However, in the case of a brighter source or a more sensitive camera, there the approach can easily be used for capturing live-action images, said Gehm.
The novel method holds many promising applications. It not only works for light scattering through a material but would also work for light scattering off of a surface – for example, the paint on a wall. This can enable security cameras to operate around corners or perhaps even through frosted glass.
Many light-based devices in the medical field look to collect data through the skin and other types of tissues, for example, a Fitbit capturing the pulse of persons through their wrist. However, when the light gets scattered as it passes through the skin and flowing blood cells, it poses a difficulty to more sophisticated measurements. This method may now offer a path forward.
“We’re also looking to see if this approach can be used to separate different aspects of light, particularly color,” stated Gehm. “One could imagine using coded apertures to gain more information about a single image rather than using it to obtain a sequence of images.”
The Defense Advanced Research Projects Agency (HR0011-16-C-0027) supported the work.
Reconstructing Single Exposures of Scattered Light into a Series of Clear Frames