Real-Time Panoramic Imaging and Ultrafast Motion Detection with ACEcam

An international team of researchers, led by Professor Xuming Zhang from the Department of Applied Physics, Photonics Research Institute & Research Institute for Advanced Manufacturing at The Hong Kong Polytechnic University, has developed a biomimetic artificial compound eye (ACE) as a panoramic camera, known as ACEcam. Their work has been published in the journal Light: Science & Applications.

Robert Hooke first studied natural compound eyes (NCEs) in 1664 after observing neatly organized "pearls" in the cornea of a grey drone fly. Inspired by these NCEs, ACEs have since been developed using planar microlens arrays, curved microlens arrays, and metasurfaces.

However, none of these ACEs have matched the capability of NCEs in simultaneously achieving dynamic motion detection and real-time panoramic direct imaging. A key challenge for ACEs based on curved microlens arrays is transferring light rays collected by multiple microlenses on a curved surface to a flat imaging sensor, like a CMOS chip, while maintaining their spatial relationships.

The ACEcam overcomes this issue with a 180 º field of view (FOV), surpassing the typical 150–180 º FOV of most arthropods. This makes it especially useful for surveillance applications. Its real-time panoramic direct imaging capability eliminates the need for post-processing, making it ideal for tasks such as imaging and measuring distances between moving objects in real-world scenarios.

Additionally, its nearly infinite depth of field could enhance augmented reality experiences, making them more immersive and distinctive compared to existing technologies.

The ACEcam also features ultrafast angular motion detection (5.6 × 10⁶ deg/s) along with translational and rotational motion perception capabilities, enabling precise kinesthetic tracking and motion state control for various machines, from regular vehicles to high-speed aircraft and spacecraft. These unique features place the ACEcam in a specialized market.

For instance, its 180 º FOV and rapid angular motion detection make it an excellent candidate for integration into obstacle avoidance systems for high-speed UAVs. This capability eliminates the need for additional obstacle avoidance lenses, reducing unnecessary bulk and weight. The compact size and wide FOV also make the ACEcam well-suited for endoscopy applications.

In the proposed ACEcam, lensed plastic optical fibers act as artificial ommatidia. These fibers, with conical microlenses attached to their distal ends, collect and transmit light to the sensing unit, mimicking the function of natural ommatidia.

To simulate NCEs, a bundle of lensed plastic optical fibers is uniformly arranged on a hemispherical surface, enabling the ACEcam to achieve superior dynamic motion detection and static imaging capabilities. The researchers summarized the working principle of the camera as follows:

“We design a microlens with a conical shape onto the distal end of the plastic optical fiber to decrease the light acceptance angle of a plastic optical fiber and thus increase the angular resolution,” the researchers added.

They noted, “Our analysis and simulations show that a half-apex angle of 35^o is the best choice for the conical microlens, reducing the acceptance angle of the fiber from 60 º to 45 º. Moreover, the sharp tip of the conical microlens is rounded during the fabrication process. This rounded tip is beneficial since it ensures that light information in the central angular range is not lost.”

“The conical microlens plastic optical fibers are fabricated in batches by a sequence of 3D printing, electroplating, and two molding processes, which is a novel approach to add a microlens onto the distal end of an optical fiber. Approximately 200 conical-microlens optical fibers are obtained in each batch process, and each conical microlens has a smooth surface and naturally a rounded tip, ” they added.

The scientists said, “271 lensed plastic optical fibers are attached to a 3D-printed perforated dome so that all the lensed ends of the fibers are on the dome surface, while the bare ends of the fibers are placed into a perforated planar buncher. Light leaving the bare fiber ends is projected onto a flat imaging sensor via an imaging lens. The dome, the buncher, the imaging lens, and a flat imaging sensor chip are hosted in a screwed hollow tube.”

“In the assembly, the 3D-printed dome has a black color to absorb the leaked or stray light, functioning the same as the pigment cells in the NCEs to prevent crosstalk. The lensed plastic fibers confine the collected light, preventing crosstalk and the associated ghost images, and the buncher maintains the relative positions of the microlenses on the dome,” they added.

The researchers added, “This setup enables the light collected at the curved surface to be transmitted to a flat image sensor, thus faithfully replicating the ommatidia in an NCE. In the image formation process, the light emitted by the object is captured at different angles by the microlenses on the dome. At the bare fiber ends, the planar images are projected onto the flat imaging sensor chip.”

“Then, the final images are obtained for digital image processing. The imaging lens prevents contact of the bare fiber ends with the vulnerable image chip surface” they further explained.

“ACEcam holds the promise of becoming the cornerstone for future ACEs, owing to its synergy with diverse disciplines. For instance, the imaging optical fiber bundles can emulate natural ommatidia to replicate optical and neural superposition observed in NCEs, potentially enhancing ACEcam's imaging resolution and dynamic perception speed. Furthermore, the integration of optofluidic lenses with ACEcam presents an opportunity to harness the advantages of both arthropods' compound eyes and vertebrate monocular eyes,” the scientists forecasted.

Journal Reference:

Jiang, H., et al. (2024) Optical fiber-based artificial compound eyes for direct static imaging and ultrafast motion detection. Light: Science & Applications. doi.org/10.1038/s41377-024-01580-5.