Penn State electrical engineering researchers developed a metasurface, an optical element that resembles a glass slide and uses tiny nanostructures positioned at different angles to control light, in an attempt to mimic the simple, instantaneous image processing power of the human eye.
Image Credit: air009/Shutterstock.com
The group published their invention in Nature Communications under the direction of corresponding author Xingjie Ni, an Associate Professor of electrical engineering and computer science (EECS) at Penn State.
The researchers found that in order for artificial intelligence (AI) systems to process images and identify objects, they can be slow and require a large amount of computing power and energy. On the other hand, images can be preprocessed and altered using the metasurface before they are taken by a camera, requiring less processing power and data bandwidth from a computer and artificial intelligence.
The way the metasurface operates is by taking an image and converting it from the Cartesian coordinate system, which uses a bullseye-like pixel distribution, to the log-polar coordinate system, where image pixels are arranged in straight rows and columns along the x and y axes.
Like the arrangement of light receptors inside the human eye, the metasurface takes images and arranges them in a log-polar coordinate system - with denser pixels for the central, focused features and sparser pixels for the peripheral regions. This allows for the more important aspects of a photo to come through clearly while others remain less in focus, thereby saving data bandwidth.
Xingjie Ni, Corresponding Author and Associate Professor, Electrical Engineering and Computer Science (EECS), The Pennsylvania State University
The image is converted from the Cartesian system into log-polar coordinates by the metasurface when it is positioned in front of a camera, allowing light to pass through it before the image is digitalized and sent to a computer. The metasurface operates at the speed of light and requires no power because it uses nanostructures to bend light.
As an image of an object can vary in size or orientation, it is desirable to preprocess images to make them resistant to scale and rotation changes. This preprocessing helps AI applications more easily recognize them as the same object.”
Xingjie Ni, Corresponding Author and Associate Professor, Electrical Engineering and Computer Science (EECS), The Pennsylvania State University
Researchers are also able to convert the log-polar image back into the original image with Cartesian coordinates by putting an alternative metasurface in front of a camera.
The researchers claimed that there are numerous possible uses for the invention, such as target tracking and surveillance to map a car's movement through a city.
A metasurface can be used in tandem with AI systems as a preprocessor, making it easier to recognize the same car from multiple street view cameras. Or if it is applied to a satellite, it could potentially track planes from takeoff to landing.”
Xingjie Ni, Corresponding Author and Associate Professor, Electrical Engineering and Computer Science (EECS), The Pennsylvania State University
Zingwang Zhang, a former Postdoctoral Scholar in EECS; Ziaojie Zhang, a Graduate Student in EECS at the time of research; Yao Duan, a Penn State doctorate holder in EECS; and Lidan Zhang, also a Graduate Student in EECS, round out the co-authors in addition to Ni.
This work was supported by the National Science Foundation of the United States, NASA, the Office of Naval Research, the National Eye Institute of the National Institutes of Health, and the Gordon and Betty Moore Foundation.
Journal Reference:
Zhang, X., et.al (2023) All-optical geometric image transformations enabled by ultrathin metasurfaces. Nature Communications. doi.org/10.1038/s41467-023-43981-x