Artificial Intelligence (AI) use in Huygens
An overview
Artificial Intelligence (AI) is an ever growing field and has opened new possibilities for microscopy imaging and analysis1. This page gives an overview of what Huygens has to offer in three key areas.
Image Reconstruction:
Open-source AI tools such as CARE2 and NOISE2VOID3 are effective in denoising images with low Signal to Noise Ratios (SNR). Yet by design, they are trained to recognize patterns and do not attempt to restore the true underlying object of the image. The presence of out-of-focus signal in images due to the point spread function (PSF) is completely ignored, which makes these approaches prone to producing hallucinations and biases. In comparison, Huygens deconvolution use a priori knowledge of how the image was acquired, as it includes an advanced poisson noise model, powerful background correction, and a sophisticated PSF model. With these options combined into one deconvolution process, Huygens removes noise, corrects the background, and increases resolution and contrast. For more information visit this webpage. Huygens also offers a Hybrid AI Filament deconvolution option, which combines the robustness of classic deconvolution algorithms with the pattern recognition and adaptability of AI.
Object Segmentation:
Image segmentation is a key technique to identify objects, such as cells or nuclei in an image, facilitating object classification and analysis. Aside from existing segmentation based on threshold and watershed, AI-driven segmentation has shown its additional value in studying microscopy data1. These AI tools can identify and isolate cells or intracellular structures in both 2D and 3D images, providing improved visualization, as well as enabling detailed measurements of morphology and location. Huygens now offers a NEW AI segmentation tool based on Meta's Segment Anything Model 24 within the powerful Object Analyzer:
Tracking:
Similar to segmentation, AI-based object tracking allows for monitoring of cells or subcellular structures, such as vesicles, over time. This helps gather data on their movement and localization through different time points. Huygens Object Tracker works by manually selecting just a few objects and background regions, after which the machine learning algorithm automatically detects new objects and tracks these. Tracks can be filtered based on specific parameters, edited, and analyzed with the included Track Analyzer.
References
1. Liu Z, Jin L, Chen J, Fang Q, Ablameyko S, Yin Z, Xu Y. A survey on applications of deep learning in microscopy image analysis. Comput Biol Med. 2021 Jul;134:104523. doi: 10.1016/j.compbiomed.2021.104523. Epub 2021 May 29. PMID: 34091383.
2. Weigert, M., Schmidt, U., Boothe, T. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat Methods 15, 1090–1097 (2018). https://doi.org/10.1038/s41592-018-0216-7
3. Krull, A., Buchholz, T., Jug, F., Noise2Void - Learning Denoising from Single Noisy Images. (2019) https://doi.org/10.48550/arXiv.1811.10980
4. Ravi N., Gabeur V., Hu Y., Hu R., Ryali C., Ma T., Khedr H., Rädle R., Rolland C., Gustafson L., Mintun E., Pan J., Vasudev Alwala K., Carion N., Wu C., Girshick R., Dollár P., Feichtenhofer C. SAM 2: Segment Anything in Images and Videos. https://ai.meta.com/research/publications/sam-2-segment-anything-in-images-and-videos/