Precision Autofocus in Optical Microscopy with Liquid Lenses Controlled by Deep Reinforcement Learning

Academic Background

Microscopic imaging plays a crucial role in scientific research, biomedical studies, and engineering applications. However, traditional microscopes and autofocus techniques face hardware limitations and slow software speeds in achieving system miniaturization and rapid, precise focusing. Traditional microscopes typically employ multiple fixed-focus lenses and mechanical structures to achieve magnification and focusing functions, resulting in bulky devices, slow focusing speeds, and difficulties in rapid operation within confined spaces. Liquid lenses, which lack mechanical components and adjust focus through electrical signals, offer advantages such as compact size, rapid response, and low manufacturing costs, making them a potential solution to these problems.

In recent years, advancements in artificial intelligence and new optical components have brought new research directions to microscope autofocus technology. Traditional autofocus methods rely on image sharpness evaluation, often requiring multiple image acquisitions and assessments, leading to slow speeds. The introduction of deep learning techniques has made it possible to predict the focal plane position directly from a single image, but these methods depend on the quality and quantity of training data, making it difficult to handle new, unseen samples. Deep Reinforcement Learning (DRL), which learns optimal decision-making strategies through continuous interaction with the environment, can utilize sequential information and is particularly well-suited for autofocus tasks.

Source of the Paper

This paper was co-authored by Jing Zhang, Yong-Feng Fu, Hao Shen, Quan Liu, Li-Ning Sun, and Li-Guo Chen, affiliated with the School of Mechanical and Electrical Engineering and the School of Computer Science and Technology at Soochow University. The paper was published in 2024 in the journal Microsystems & Nanoengineering under the title Precision Autofocus in Optical Microscopy with Liquid Lenses Controlled by Deep Reinforcement Learning.

Research Process and Experimental Design

1. Construction of the Liquid Lens Microscope System

The research team designed and fabricated a liquid lens based on the Electrowetting-on-Dielectric (EWOD) effect. The lens consists of an upper glass cover plate, a lower ITO cover plate, a chamber, a Parylene C dielectric film, and a hydrophobic Teflon AF film. The chamber of the liquid lens adopts an inverted truncated cone structure with a 60-degree tilt angle, which helps stabilize the optical axis center and reduce image distortion. By adjusting the voltage applied to the liquid lens, rapid changes in focal length can be achieved.

2. Design of the Deep Reinforcement Learning Autofocus Model

The research team proposed a Deep Reinforcement Learning-based Autofocus (DRLAF) method. In this approach, the liquid lens is treated as an “agent,” raw images as the “state,” and voltage adjustments as the “actions.” Through deep reinforcement learning, the model learns the focusing strategy directly from captured images, achieving end-to-end autofocus. Unlike methods that rely solely on sharpness evaluation as model labels or inputs, this study designed a targeted reward function that significantly enhances the performance of microscope autofocus tasks.

3. Dataset Processing and Training Methods

The research team utilized image sequences collected during the autofocus process of the liquid lens as the state input for the reinforcement learning agent. The experimental setup employed a microscope with a fixed object distance, equipped with a voltage-driven liquid lens. The voltage was adjusted in increments of 0.1V to change the focal length, capturing the complete focusing process (defocused-focused-defocused) of the sample. After image processing, a “state” dataset suitable for input into DRLAF was formed. The study also proposed a random sampling training method, combining multiple state datasets into a list for training to enhance the model’s adaptability to unknown samples.

4. Action Space Design

In the reinforcement learning autofocus model, the research team designed executable actions for the agent, adjusting the voltage applied to the liquid lens with different step sizes. The size of the action space significantly impacts the speed and accuracy of autofocus. Through experiments, the team determined the optimal action space size and compared the focusing performance under different action set configurations.

Main Research Findings

1. Performance of the Liquid Lens

The research team successfully fabricated a high-performance liquid lens with a response time of 98 milliseconds under a 40V driving voltage. The relationship between focal length and driving voltage varied significantly under different magnification levels. Experimental results demonstrated that the rapid response and electrical adjustment advantages of the liquid lens significantly improved the structural compactness and zoom speed of the microscope.

2. Impact of Action Space on Autofocus Performance

Experimental results showed that as the action space size increased, the autofocus deviation significantly decreased, and the success rate and accuracy improved markedly. When the action space size was 7, the model achieved a balance between speed and accuracy, with the average number of autofocus time steps significantly reduced and stabilized.

3. Impact of State Random Sampling on Autofocus Performance

By training with random sampling from multiple state datasets, the model’s autofocus success rate and generalization ability were significantly enhanced. When the number of state datasets increased to 50, the model achieved a 97.2% success rate on the test set, with a root mean square error (RMSE) of 2.85×10^-3V for the predicted voltage, indicating effective adaptation to different samples.

4. Design and Optimization of the Reward Function

The research team designed a hybrid reward function combining sharpness reward, time step reward, stop reward, and additional reward. Experimental results demonstrated that this reward function significantly improved the model’s performance in autofocus tasks, particularly in reducing the number of focusing time steps.

Conclusions and Significance

This study proposed a liquid lens microscope system based on deep reinforcement learning, achieving rapid and precise autofocus. By designing a high-performance liquid lens and a deep reinforcement learning model, the research team successfully reduced the average number of autofocus time steps to 3.15, representing a 79% improvement in speed compared to traditional search algorithms. Additionally, by training with random sampling from multiple state datasets, the model’s generalization ability was significantly enhanced, enabling reliable autofocus across different samples and fields of view.

This research has broad application prospects in fields such as optoelectronic reconnaissance, microscopic imaging, digital lens imaging, and endoscopy, providing robust support for automation and intelligent processing in related fields.

Research Highlights

1. Innovation: First integration of deep reinforcement learning with liquid lenses, achieving end-to-end autofocus.
2. Efficiency: Significant improvement in autofocus speed, with an average of only 3.15 steps required for focusing.
3. Generalization Ability: Enhanced model adaptability to different samples through random sampling from multiple state datasets.
4. Low Cost: Use of liquid lenses reduces system complexity and cost, offering high application value.

Other Valuable Information

The research team also conducted generalization experiments and ablation experiments on the reward function, further validating the model’s performance on unknown samples and the effectiveness of the reward function design. Experimental results demonstrated that the random sampling training strategy significantly improved the model’s generalization ability, and the hybrid reward function design excelled in reducing the number of focusing time steps.

This study provides new insights and solutions for the development of microscope autofocus technology, with significant scientific value and application prospects.