FIG: Flow with Interpolant Guidance for Linear Inverse Problems

Diffusion and flow matching models have recently been used to solve various linear inverse problems in image restoration, such as super-resolution and inpainting. Using a pre-trained diffusion or flow-matching model as a prior, most existing methods modify the reverse-time sampling process by incorporating the likelihood information from the measurement. However, they struggle in challenging scenarios, such as high measurement noise or severe ill-posedness. In this paper, we propose Flow with Interpolant Guidance (FIG), an algorithm where reverse-time sampling is efficiently guided with measurement interpolants through theoretically justified schemes. Experimentally, we demonstrate that FIG efficiently produces highly competitive results on a variety of linear image reconstruction tasks on natural image datasets, especially for challenging tasks.

The above figure shows the overview of our FIG algorithm during the conditional sampling process. Black arrows (\( \mathbf{\rightarrow} \)) denote the unconditional update. Orange arrows (\( \mathbf{\rightarrow} \)) represent \( K \) times conditional updates with unconditional sample \( \boldsymbol{x}'_t \) and measurement interpolant \( \boldsymbol{y}_t \) at each timestep \( t \). Blue arrows (\( \mathbf{\rightarrow} \)) indicate the measurement interpolation. Below are additional experimental results on CelebA-HQ, AFHQ-Cat, and LSUN-Bedroom datasets. FIG delivers impressive performance.