A research team at the Institute for Basic Science (IBS) Center for Nanoparticle Research, led by Professor Dae-Hyeong Kim, along with Professor Young Min Song at the Gwangju Institute of Science and Technology, has developed a perovskite-based camera with an object detection specialty. Researchers took inspiration from the structures and operations of avian eyes to build this camera. This study was published in the journal Science Robotics.
Structures and functions of bird’s eye. (a) Bird vision. (b) Deep central fovea and four types of cones. (c) Foveated vision and tetrachromatic vision. Image Credit: Institute for Basic Science
Raptors, such as eagles, can accurately perceive prey from kilometers away. The question is, is it possible to model a camera technology after the excellent vision of these birds?
Over the years, species have developed and modified their eyes to fit their specific habitats and survival environments better. Bird eyes also have special structures and functions related to vision because of their many years of evolutionary adaption to the environment of living and flying at high altitudes.
In the retina of an animal's eye, there's a small pit called the fovea that bends incoming light. Bird eyes, unlike human eyes with shallow foveae, have deep central foveae, which bend incoming light extensively. The area of highest cone density lies within these foveae (Figure 1b), enabling birds to clearly see distant objects through magnification (Figure 1c). This specialized vision is known as foveated vision.
While human eyes are limited to perceiving visible light, bird eyes possess four cones sensitive to ultraviolet (UV) in addition to the visible spectrum (red, green, blue; RGB) of light. This tetrachromatic vision means that birds can see a wealth of visual data, enhancing their ability to detect target objects effectively in a dynamic environment (Figure 1c).
Motivated by these features, the IBS research team created a novel class of cameras with an artificial fovea and a multispectral image sensor that can detect both RGB and UV. These cameras are specifically built for object detection.
Using an optical simulation, the researchers first created an artificial fovea that was improved by imitating the deep central foveae found in the eyes of birds. As a result, the camera was able to magnify far-off target objects without experiencing image distortion.
After that, the group created the multispectral image sensor using perovskite, a material renowned for its superior electrical and optical qualities. Different perovskite materials that absorb various wavelengths were used to create four distinct types of photodetectors, which were then stacked vertically to create the multispectral image sensor.
We also developed a new transfer process to vertically stack the photodetectors. By using the perovskite patterning method developed in our previous research, we were able to fabricate the multispectral image sensor that can detect UV and RGB without additional color filters.
Dr. Jinhong Park, Study First Co-Author, Center for Nanoparticle Research, Institute for Basic Science
The drawback of conventional cameras that enlarge objects with a zoom lens is that they only focus on the target object and not its surroundings. However, the camera inspired by a bird's eye offers a view of the surrounding peripheral region as well as an enlarged view of the foveal region.
The camera inspired by bird vision outperforms traditional ones in detecting motion across their fields of view. Additionally, it accomplishes this without extra color filters by discerning between RGB and UV light, making it lighter and more economical.
Through simulations, the study team validated the object identification and motion detection capabilities of the newly developed camera. The camera achieved an object identification confidence score of 0.76, nearly doubling the previous system's score of 0.39.
Furthermore, the motion detection rate increased by 3.6 times compared to the existing camera system, showcasing a significantly enhanced sensitivity to motion.
Birds’ eyes have evolved to quickly and accurately detect distant objects while in flight. Our camera can be used in areas that need to detect objects clearly, such as robots and autonomous vehicles. In particular, the camera has great potential for application to drones operating in environments similar to those in which birds live.
Dae-Hyeong kim, Professor, Center for Nanoparticle Research, Institute for Basic Science
This cutting-edge camera technology has several potential applications in various sectors and represents a substantial leap in object detection.
Journal reference:
Park, J., et al. (2024) Avian eye–inspired perovskite artificial vision system for foveated and multispectral imaging. Science Robotics. doi.org/10.1126/scirobotics.adk6903