High-Resolution Dynamic Facial Micro-Expression Capture: Innovations in Multi-Focus Camera Array

Background and Research Issues

Capturing high-quality dynamic facial images is of critical importance in various fields such as biomedicine, emotion recognition, disease diagnosis, surgical outcome evaluation, facial prosthetic design, and genetic feature studies. Human facial expressions, particularly micro-expressions, provide an abundance of biomedical information. For instance, studies indicate that capturing high-resolution dynamic facial expressions can enhance the accuracy of affective computing, facilitate disease diagnosis, evaluate surgical outcomes, and create highly accurate facial prostheses. Against this backdrop, the ability to capture detailed curved facial surface features at high resolution has become a pressing challenge for the scientific community.

Traditional single-camera imaging systems face inherent limitations in balancing depth of field (DOF), field of view (FOV), and resolution. Current popular datasets, such as BP4D-SPONTANEOUS from 2014 and SAMM (Spontaneous Micro-Facial Movement Dataset), fail to meet the requirements of detailed expression capture in terms of both resolution and DOF.

To address these technical bottlenecks, a research team from Duke University and Ramona Optics Inc. has proposed an innovative multi-camera array microscope (MCAM) system. This system employs a multi-focus strategy to simultaneously solve the challenges of achieving high resolution and extended depth of field for curved surface imaging.

Research Origin and Publication Details

This study, led by Lucas Kreiss and Weiheng Tang and supported by researchers including Ramana Balla, Xi Yang, and Amey Chaware, was a collaborative effort between Duke University and Ramona Optics Inc. The research was published on February 1, 2025, in Volume 16, Issue 2 of Biomedical Optics Express. The core findings of the research are publicly available via DOI link (https://doi.org/10.1364/boe.547944).

Research Process and Experimental Methods

The research adopted a multi-step design and experimental validation to demonstrate the advantages of the multi-focus camera array in capturing dynamic high-resolution facial images. The workflow comprised four key steps: system configuration, performance characterization, facial image capture, and dynamic expression recording.

1. System Design and Multi-Focus Configuration

The study employed a compact 9×6 array consisting of 54 individual cameras. Each camera was equipped with a lens featuring a focal length of 25.05 mm, a numerical aperture of 0.04, and a 13-megapixel Onsemi AR1335 CMOS sensor with a pixel width of 1.1 µm. These cameras were arranged on a printed circuit board (PCB) at a spacing of 13.5 mm.

To achieve multi-focus imaging, the research team used an anatomically accurate foam facial model as a benchmark. Depth distribution across the model (ranging from 0–40 mm) was measured using a digital caliper, and focal planes of the cameras were adjusted accordingly, forming a “multi-focus depth profile.” For instance, working distances were set between 200 mm and 240 mm, with focal adjustments being calibrated using a high-resolution reference sample.

Image stitching was accomplished using the Hugin algorithm. Calibration steps first defined stitching parameters for in-focus images captured by each camera, which were subsequently utilized to compose high-resolution panoramic images.

2. Performance Characterization of the Multi-Focus System

To validate and characterize the optical performance of the system, the researchers used a resolution target combined with a high-precision translation stage (accuracy 0.01 mm). The system’s resolution, DOF, and extended DOF (eDOF) were comprehensively assessed for each camera at various focal planes.

First, a series of focal stack images were collected, and the image “sharpness metric” for each camera was calculated at every plane. Using Gaussian profiles, the full width at half maximum (FWHM) was used to estimate the DOF. Next, the sharp edges of calibrated images were analyzed using the edge spread function (ESF) and modulation transfer function (MTF) to calculate the lateral resolution, which was measured to be ∼26.14 µm ± 5.8 µm. Additionally, measurements indicated that the system achieved an overall extended DOF of approximately 43 mm, a 10-fold improvement compared to single-focal-plane setups.

3. Facial Imaging and Dynamic Video Capture

To capture the natural curvature of human faces, the researchers built an experimental setup with three LED ring lights positioned in front, to the left, and to the right of the face. Volunteers kept their faces stationary on a chin rest, and facial images were captured indirectly using a 45° angled mirror. The system successfully captured high-resolution images (over 13,000×9,000 pixels), encompassing the entire face.

The research also captured dynamic facial expressions at a frame rate of 12 frames per second (fps), effectively recording fine details such as wrinkles, pores, and other micro-features. These experiments demonstrated the system’s ability to perform dynamic imaging while maintaining high-resolution stitching.

Results and Conclusions

Key Findings

• The DOF of each camera unit was ∼4.7 mm, and the entire camera array achieved an extended DOF of 43 mm through its multi-focus configuration.
• The resolution of the stitched image reached ∼13,394×9,062 pixels, with a lateral resolution of approximately 26 µm.
• Dynamic expression capture successfully demonstrated high-resolution facial details, including wrinkles, pores, and other micro facial features.

Conclusions

The study introduced an innovative imaging system design that successfully overcomes the DOF-resolution trade-offs of traditional single-camera systems, achieving high resolution, extended DOF, and dynamic expression capture for curved facial surfaces. The system shows significant promise for applications in micro-expression detection, clinical diagnosis, and other fields.

Research Highlights

• Major Technological Innovation: The first use of a multi-camera array to address the challenge of imaging non-planar curved surfaces.
• Combination of High Resolution and Extended DOF: Delivers a 50-fold improvement in resolution compared to existing public datasets.
• Vast Application Potential: Particularly suited for biomedical diagnostics, virtual and augmented reality, affective computing, and security systems.

Future Research and Application Potential

The research team plans to integrate tunable focus lenses and real-time motion compensation algorithms in future systems to further enhance adaptive focusing capabilities and robustness. Improvements in lighting uniformity and magnification calibration are expected to enable the MCAM system to excel in more complex application scenarios.

This research represents a transformative advance in high-resolution facial imaging, with significant implications for both the scientific and industrial communities. It paves the way for future technological enhancements and practical benefits across a wide range of applications.