FIG: Flow with Interpolant Guidance for Linear Inverse Problems

Diffusion and flow matching models have been recently used to solve various linear inverse problems such as image restoration. Using a pre-trained diffusion or flow-matching model as a prior, most existing methods modify the reverse-time sampling process by incorporating the likelihood information from the measurement. However, they struggle in challenging scenarios, e.g., in case of high measurement noise or severe ill-posedness. In this paper, we propose Flow with Interpolant Guidance (FIG), an algorithm where the reverse-time sampling is efficiently guided with measurement interpolants through theoretically justified schemes. Experimentally, we demonstrate that FIG efficiently produce highly competitive results on a variety of linear image reconstruction tasks on natural image datasets. We improve upon state-of-the-art baseline algorithms, especially for challenging tasks.

The above figure shows the overview of our FIG algorithm during the conditional sampling process. Black arrows (\( \mathbf{\rightarrow} \)) denote the unconditional update. Orange arrows (\( \mathbf{\rightarrow} \)) represent \( K \) times conditional updates with unconditional sample \( \boldsymbol{x}'_t \) and measurement interpolant \( \boldsymbol{y}_t \) at each timestep \( t \). Blue arrows (\( \mathbf{\rightarrow} \)) indicate the measurement interpolation. Below are additional experimental results on CelebA-HQ, AFHQ-Cat, and LSUN-Bedroom datasets. FIG delivers impressive performance.