Super-resolution methods based on frequency shift can boost the resolution of microscope images by two fold. STORM, STED and other methods in fluorescence microscopy can boost spatial resolution to more than 10 times the diffraction limit, and the latter received the Nobel Prize in Chemistry in 2014.
A, Sketch of the apparatus. Speckle illumination modes generated by the RGG were divided into a reference path that was directly measured by a pixilated camera and an object path that was measured by a single-pixel detector after interacting with the sample. B, raw speckle patterns H (top) and intensity sequence I (bottom) measured by the camera and the single-pixel detector, respectively. C, correlating H and I one can get a low-quality result especially when the sampling ratio β in our case is as low as 10%. Then, we feed it into the neural network and keep it fixed. The output of the neural network is taken as the estimated object, which is then numerically multiplied with h to generate the estimated intensity sequence. We measure the mean square error between i and the estimated intensity sequence as the loss function to guide the update of weights in the neural network. Image Credit: Fei Wang, Chenglong Wang, Mingliang Chen, Wenlin Gong, Yu Zhang, Shensheng Han, and Guohai Situ.
However, promoting and using these technologies for long-distance super-resolution imaging is problematic. The main technique to enhance the resolution of long-distance imaging systems, which is still limited by the diffraction limit, is to increase the aperture of the imaging system and lower its focal length.
A group of researchers headed by Professor Guohai Situ of the Chinese Academy of Sciences’ Institute of Optics and Fine Mechanics and co-workers created an AI-driven super-resolution technology in a recent publication published in Light: Science & Applications.
To begin with, the researchers encoded the object information into single-pixel measurements using GI, a method with excellent information acquisition efficiency and detection sensitivity. Furthermore, they used a physics-enhanced deep neural network (DNN) to recreate the object.
They fed the DNN with the low-quality GI picture derived by linear correlation, used the DNN output as the estimation of high-quality GI image, and utilized the forward propagation model of GI to evaluate the estimation of GI measurements commencing from the DNN output.
The DNN’s settings were then tweaked to reduce the difference between the raw and estimated GI measures. DNN’s output also converges to a significantly superior outcome, in addition to error minimization. The proposed method is termed Ghost Imaging with Deep Neural Network Constraints (GIDC) by the researchers.
GIDC does not need lots of labeled data pairs to train a DNN, all it needs is the raw GI measurements and the forward propagation model of GI both are available in a typical GI system.
Guohai Situ, Professor, Institute of Optics and Fine Mechanics, Chinese Academy of Sciences
It is, in fact, a general approach that is unaffected by the dataset.
The researchers used a pseudothermal GI system to illustrate their methods. A reference path and a test path are created from the random illumination speckle field. A camera detects the light of the reference path without engaging with the item, while the light of the test path is caught by a single-pixel detector after passing through the object.
Although neither detector directly captures a resolvable image of the item, by spatial correlating the obtained time-varying patterns and the synchronized bucket signal, a simple linear technique may be used to rebuild its image.
The researchers conducted a comparative analysis using a variety of difficult real-world settings, such as a flying drone, and found that the suggested method surpasses existing widely used GI methods.
GIDC has the potential to break the diffraction limit of GI and reduce the sampling ratio required to obtain a high signal-to-noise ratio (SNR) image.
Guohai Situ, Professor, Institute of Optics and Fine Mechanics, Chinese Academy of Sciences
“Compared with conventional imaging, GI can collect more information about the object, but it is highly coupled to the 1D bucket signal. The proposed method allows more information to be effectively extracted and is suitable for many computational imaging systems that adopt encoding-decoding strategy.” the researchers added.
They also conclude by stating, “Considering that the GI is a promising method for long-distance imaging, we believe the proposed method has great potential for high-resolution remote sensing.”
Journal Reference:
Wang, F., et al. (2022) Far-field super-resolution ghost imaging with a deep neural network constraint. Light: Science & Applications. doi.org/10.1038/s41377-021-00680-w.