Figure 1 presents a schematic of the custom-built DM-based sensorless AO FFOCT system. The FFOCT system comprises a spatially incoherent light-emitting diode (LED) with a center wavelength of 850 nm and a bandwidth of 30 nm (M850L3, Thorlabs), providing a theoretical axial resolution of approximately 8 μm m in water. The LED is focused by a condenser (focal length: 50 mm) positioned in front of the system’s pupil plane, implementing a Köhler illumination configuration. A field diaphragm in front of the LED is optically conjugated to both the retina and the reference mirror. A 50:50 cubic non-polarizing beam splitter (BS) divides the illumination beam into the reference and sample arms. In the reference arm, a pair of achromatic doublets (focal length: 150 mm) direct light toward a microscope objective (Olympus 10X/0.25 NA plan Achromat), which focuses the beam onto a silicon reference mirror. Both the microscope objective and the reference mirror are mounted on a high-speed voice-coil translation stage (X-DMQ12P-DE52, Zaber Technologies Inc.), enabling rapid optical path length adjustments in the refenrence arm. In the sample arm, a deformable mirror (DM) with a clear aperture of 7.2 mm (DM97-08, Alpao, France) is relayed to the eye’s pupil with 1:1 magnification using a pair of achromatic doublets (focal length: 150 mm). The aperture of the system is ultimately limited by the 7.2 mm DM size. To compensate for dispersion effects and curvature mismatch between the FFOCT coherence gate and the retina, an optical window is introduced in the sample arm between L4 and the eye (not shown in Fig. 1) [23]. Photons reflected from the reference mirror and backscattered from the sample recombine at the BS and are focused by an achromatic doublet (focal length: 200 mm) onto a high-speed 2D megapixel (1440 × 1440 pixel) camera (Q-2A750-Hm/CXP-6, Adimec), with 1 μm/pixel retinal sampling.

A spectral-domain optical coherence tomography (SD-OCT) system (GAN611, Thorlabs) is integrated into the setup. The SD-OCT system utilizes a broadband superluminescent diode with a center wavelength of 930 nm and a 60 nm bandwidth, resulting in a theoretical axial resolution of 4.5 μm in water. Unlike our previous setup [18], the pupil plane of the system is now optically conjugated to the midpoint between both galvanometer scanners of the SD-OCT, reducing pupil wobble [24], and enhancing the quality of the B-scan, and field of view. This conjugation is achieved using a pair of achromatic doublets (focal lengths: 150 mm and 200 mm, respectively). A long-pass dichroic mirror couples the SD-OCT to the FFOCT optical path. To compensate for dispersion in the SD-OCT, additional optical windows are incorporated into the reference arm of the SD-OCT, while two short-pass filters are used to block the SD-OCT light in both the reference and detection arms of the FFOCT. The numerical aperture of the SD-OCT is also limited by the DM’s 7.2 mm clear aperture.

B-scan measurements are performed at a 100 kHz A-scan rate with 256 scanning points on a 1^∘ field of view. Each acquired B-scan is laterally averaged and serves three key purposes: 1) measuring the retinal axial position to minimize optical path length differences between the reference and sample arms in FFOCT, allowing for real-time retinal axial motion correction [25,26]; 2) providing a real-time cross-sectional view for precise positioning of the FFOCT coherence gate at a specific retinal layer of interest; 3) acting as a merit function for sensorless wavefront optimization by maximizing the brightness of a selected retinal layer.

Merit function: Although the brightness of the FFOCT signal is affected by ocular aberrations, it cannot be used as a reliable merit function due to intensity fluctuations also arising from inaccurate two-phase modulation induced by axial retinal motion [26]. To circumvent this issue, we used the brightness of the SD-OCT B-scan as a surrogate for FFOCT SNR optimization. Given the close wavelengths of both systems, wavefront optimization using SD-OCT brightness as the merit function effectively enhances the FFOCT SNR as well (see Supplement 1 Figure S1).

Algorithms for wavefront optimization: In this study, two algorithms, specifically optimized for in vivo retinal imaging, with rapid convergence and robust optimization, were investigated: the Hill Climbing (HCl) algorithm proposed by Camino et al. [15] and the Data-based Online Nonlinear Extremum Seeker (DONE) algorithm introduced by Verstraete et al.[27,28].

•

Hill Climbing (HCl) algorithm optimizes the merit function mode by mode, using adaptive step sizes to find the optimized Zernike coefficient for each mode. The step size is updated on the basis of the gradient sign of the merit function from the previous iteration. Empirical validation in a phantom eye yielded the selection of the following parameters: an initial step size of 0.3 μm, a maximum step size limit of 2.5 μm, 9 iterations per mode, an ϵ parameter of 0.3 μm to prevent local minima trapping, and 3 optimization cycles to mitigate mode coupling [29].

It is worth noting that for a small number of modes, the optimization duration was sufficient to converge towards a stable maximum of the SD-OCT signal intensity (merit function). However, this was not always the case when optimizing 5 or 8 modes, where additional correction cycles (for HCL) or iterations (for DONE) would be necessary. Since dynamic ocular aberrations evolve over a timescale of only a few seconds, extending the optimization beyond approximately 2 s would not necessarily improve convergence and could even reduce performance. Therefore, we limited the maximum optimization duration to 2 s.

Optimization of the DM response: An important consideration for efficient sensorless AO is to ensure that the merit function is not affected by DM overshoots or oscillations during optimization. Most DMs, such as the Alpao DM97-08 used in this study, are designed for closed-loop operation with small step sizes to correct residual aberrations, achieving a typical settling time of 0.62 ms to within ±10% of the command value. However, the sensorless approach often requires larger step sizes (e.g., 2 μm), which can lead to transient oscillations of the mirror surface before full stabilization. These oscillations introduce fluctuations in the merit function (SD-OCT signal intensity) that are unrelated to aberration correction, thereby reducing the accuracy and repeatability of the optimization.

To minimize this effect while maintaining a fast response, the DM slew rate was adjusted. A higher slew rate reduces oscillations but increases response time, whereas a lower slew rate shortens the settling time at the cost of larger overshoots. In this study, we selected a slew rate of 25 steps, which provided a good compromise between speed and stability, achieving rapid DM stabilization ( 7 ms) with negligible oscillations.

Protocol to investigate the optimal sensorless AO design: To investigate the number of Zernike modes that should be corrected in time-domain FFOCT retinal imaging, we used a phantom eye previously described in [21]. In summary, we placed the same microscope objective in the sample arm as in the reference arm, ensuring symmetry between both arms, and a lens-cleaning tissue was positioned at the focal place of the microscope objective as the imaging sample, to mimic the scattering behavior of the retina. To simulate the effect of static ocular aberrations, we used the 50 eye ocular aberration database collected by Jarosz et al. [22]. This database provides high spatiotemporal resolution ocular aberrations up to the 8th Zernike radial order, collected at 236 Hz over a period of 3.5 s. Initially, the database was recorded for a 5-mm pupil size. In this study, to simulate aberrations at a pupil diameter of 7.2 mm (the typical aperture for high-resolution retinal imaging), we converted the Zernike coefficients using the method described by Campbell [31]. The converted Zernike coefficients were consistent with data from the literature on static aberrations for pupils around 7 mm [32].

To assess the effect of sensorless AO on different levels of aberrations, we classified the 50 eyes into three groups based on their high-order aberration (HOA) wavefront error, excluding defocus and astigmatism. The HOA wavefront error was defined as WFEHOA=∑ck2 , where c are the Zernike coefficients of HOA (excluding piston, tip/tilt, defocus and astigmatism). These groups were: 1) Group 1 (10 eyes with the weakest HOA), 2) Group 2 (10 eyes with mid HOA), and 3) Group 3 (10 eyes with the strongest HOA). For each group, we randomly selected five subjects, ensuring that their overall aberrations were representative of the entire group.

For each subject, we applied the temporally averaged Zernike coefficients of the first 38 modes (excluding piston, tip, and tilt) to the DM to simulate static ocular aberrations. We then executed both sensorless AO algorithms (HCl and DONE) to optimize different numbers of Zernike modes: 1, 3, 5, and 8 modes (starting from defocus). For each subject aberrations, FFOCT images and SDOCT B-scans were acquired under six conditions and repeated five times: 1)under static ocular aberrations to simulate photoreceptor layer (PRL) imaging (no sensorless AO correction), 2) only defocus optimized (1 mode), 3) defocus and astigmatisms optimized (3 modes), 4) first 5 modes (up to coma) optimized, 5) first 8 modes (up to spherical aberration) optimized, and 6) at the diffraction limit (no aberration added, with the DM correcting for residual system aberrations). Furthermore, nerve fiber layer (NFL) imaging was simulated by adding 1.5 μm RMS defocus (equivalent to 0.5 D) to the static ocular aberrations.

For each subject and sensorless AO correction condition, 50 FFOCT images were acquired at 500 Hz using a 2-phase modulation step consistent with in vivo retinal imaging. The 2-phase step was implemented by modulating the reference mirror of the FFOCT with a piezoelectric transducer (PZT, ENT400, TRIOPTICS France). To analyze the FFOCT signal, we first minimized the effect of phase instability and signal fluctuation by averaging only the five brightest en-face images (where the 2-phase modulation is the most precise) out of the 50 acquired images. The signal level was then computed as the mean value of the 10,000 brightest pixel values, excluding areas without signal. The noise level was computed as the standard deviation of a region of interest (100 × 100 pixels) where no signal was present. Finally, the SNR was calculated as the ratio of the signal level to the noise level.

In vivo retinal imaging was performed in six healthy subjects (aged 25 to 35), with a spherical equivalent refraction between 0 and -4.5 diopters. The research procedures adhered to the principles outlined in the Declaration of Helsinki. Informed consent was obtained from all subjects after explaining the nature of the study and potential outcomes. The study was approved by the relevant ethics review boards [CPP and ANSM (IDRCB number: 2019-A00942-55)]. All imaging sequences were acquired in a dark room. The subjects’ eyes were cyclopleged and dilated using 0.5% tropicamide. Each subject was seated in front of the system and stabilized with a chin and forehead rest mounted on a 3D translation stage and instructed to fixate on a target. For some subjects, an initial manual defocus compensation was performed prior to starting sensorless AO correction. This step increased the brightness of the targeted retinal layer, which was used as the merit function, thereby enhancing the accuracy and reproducibility of the wavefront optimization. Images were captured immediately after wavefront optimization and during real-time correction of axial eye motion. Each image sequence comprised 500 frames of 1440 × 1440 pixels (FOV: 5∘×5∘ ) acquired within 1 s. For five subjects, only defocus and astigmatism (3 Zernike modes) were optimized. For one subject who had undergone Lasik surgery, which is known to increase high-order aberrations [33], wavefront optimization included the first 5 Zernike modes. During image acquisition, the total optical power entering the eye was 1.3 mW from the FFOCT illumination (for the 1-s acquisition) and 0.25 mW from the SD-OCT (for continuous scanning), respectively, which are below the ocular safety limits established by ISO standards for group 1 devices. Image processing steps, previously detailed in [25], included: 1) image normalization by dividing each frame by its mean intensity; 2) 2-phase demodulation by subtracting consecutive images and computing their absolute value; 3) image selection to exclude frames acquired during blinks, micro-saccades, or those with low SNR due to poor phase modulation; 4) image registration to correct both translation and rotation using a phase correlation method described in [34]; and 5) image averaging.

To study the optimal number of Zernike modes to be corrected, we used the phantom-eye protocol previously described. Figure 2 (a,b) shows the average signal level (SNR) for SD-OCT and FFOCT as a function of the number of optimized Zernike modes, using DONE or HCl algorithms. The SD-OCT signal, used as the merit function, demonstrates that both the DONE and the HCl algorithms produce similar performance and convergence trends (Fig. 2(a)). Groups 1 and 2 achieved optimal signal enhancement when correcting up to astigmatism, reaching 90% and 80% of the signal strength compared to the non-aberration condition, while the correction for coma and spherical aberration provided minor additional gains. In contrast, for group 3, correcting at least up to coma yields optimal performance, reaching the same signal recovery (80% of diffraction-limited) that group 2 attained with correction up to astigmatism.

Given that the SD-OCT and FFOCT illumination wavelengths are close, the SD-OCT signal intensity can serve as a surrogate merit function for FFOCT. As shown in Fig. 2(b), the evolution of the FFOCT signal closely mirrors that of the SD-OCT signal. For both DONE and HCL algorithms, the FFOCT signal shows comparable optimization performance, with HCL converging slightly faster for a small number of corrected modes and DONE exhibiting smoother convergence when optimizing a larger number of modes. Notably, when the wavefront is optimized up to astigmatism, the signal improvement saturates. Increasing the number of corrected modes yields negligible additional improvement, except for Group 3, which shows an extra gain of approximately 3 dB with the HCL algorithm.

For the NFL imaging case, correction up to astigmatism results in a significant increase in the FFOCT SNR (13-17 dB) in the three HOA groups and an improvement in the SNR of 5-7 dB for photoreceptor imaging. Compared to the diffraction limit (45 dB with no aberration), the sensorless AO method correcting for 3 modes (defocus and astigmatism) achieves 40 dB, 37 dB and 35 dB FFOCT SNR for groups 1, 2, and 3, respectively. These results are consistent with expectations, as group 3 presents the strongest high-order aberrations that are not corrected for in the sensorless AO optimization.

Figure 2(c-f) also includes FFOCT images of lens cleaning tissue acquired from group 1, illustrating that wavefront optimization up to astigmatism not only optimizes SNR but also results in high-resolution images that approach the diffraction-limit case.

To verify that the findings from the static phantom eye are consistent with sensorless AO performance in vivo, we applied the same protocol using SD-OCT for two different subjects representing respectively HOA Group 1 and 3. Subject 1, with -0.5 D myopia, and Subject 2, with a history of Lasik surgery leading to increased high-order aberrations [33]. For each aberration correction condition (number of optimized modes), five repeated measurements were collected. Figure 3 shows the normalized intensity (mean ± SD) versus the number of optimized modes for both subjects. The SD-OCT signal was normalized by the single intensity of the PRL imaging without aberration correction.

For both subjects, the performance of the DONE and HCl algorithms was similar with minor differences observed: DONE performed slightly better when correcting up to coma for Subject 1 and up to astigmatism for Subject 2, while HCl performed slightly better when correcting up to spherical aberration for both subjects. For Subject 1 (Fig. 3(a)), after optimizing for 3 modes, the signal increased by a factor of 2.3. This improvement was slightly further enhanced to 2.5 when the DONE algorithm was used to correct up to coma (5 modes). The optimization trend for Subject 1 aligns with the behavior observed in groups 1 and 2 (Fig. 2(a)), with optimal correction achieved with 3-mode correction. Subject 2 (Fig. 3 (b)) exhibited a significant signal enhancement when correcting for 8 Zernike modes, consistent with the optimization trend in group 3 with a stronger HOA (Fig. 2(a)). While SD-OCT showed progressive signal increase, by factors of 13–15 with 3-mode correction and 20–24 with 8-mode correction, this improvement did not always translate into a significant gain for the FFOCT signal, which seemed to saturate after 5-mode correction (Fig. 2(b)).

Given that the evolution of the SD-OCT signal in vivo with different numbers of corrected Zernike modes follows the same optimization behavior as observed in phantom eye simulations, we can conclude that the optimal sensorless AO design for FFOCT in in vivo retinal imaging involves correcting 3 modes (defocus and astigmatism) for subjects in groups 1 and 2 (weak to mid high-order aberrations) and 5 modes (up to coma) for subjects in group 3 (strong high-order aberrations).