Learning to read residual images for single-pixel photography

| Source: arXiv AI

Tags: MGV-Flow, single-pixel imaging, video reconstruction, computational imaging, RTX 4090

MGV-Flow reconstructs video from a single-pixel camera stream in 1.55 seconds at 22.9 dB PSNR, versus 398 seconds and 18.1 dB for per-clip optimization.

Details

Many monitoring tasks need events, not full video, and can run on a compact single-pixel measurement stream. Recovering a chosen interval as video has been slow and noisy at 0.24% of the pixel count. The authors show that correlating an estimate's residual with the coding patterns yields a residual image that, lifted through a pretrained video decoder, tracks the true error. Video-latent inversion (VLI) built on this beats frame-wise reconstruction, and MGV-Flow, trained once to read the lifted residual, reconstructs five frames in 1.55 s at 22.9 dB median PSNR against 398 s and 18.1 dB for VLI. It works on a physical system without retraining and reaches 3.2 frames per second on one RTX 4090.