DiffNowcast

Overview

Nowcasting is the short-horizon end of weather forecasting, the next few hours rather than the next few days. Numerical weather prediction is too slow for it. Plain U-Nets are fast but trained under MSE[1], so they hedge and produce blurry, unusable forecasts. Diffusion models fix the blurriness, but in my experiments they introduced a worse problem instead.

I nowcast the Thermal Infrared (TIR-1) channel of ISRO's INSAT-3DR geostationary satellite, training on triplets of t−30min, t, t+30min frames. The headline difficulty is temporal: INSAT-3DR delivers frames every 30 minutes, three times sparser than the 10-minute GOES data used in the reference work[1]. Larger gaps mean larger, less predictable changes between frames, and far fewer steps in which to accumulate error.

Three Experiments

Each fix exposed the next problem, so the three experiments are worth walking through in order.

• 1 · Standard score-based diffusion → generative collapse I trained EDM[2]-framework diffusion denoisers, following Chase et al.[1], with several conditioning setups (2-frame TIR-1, 4-frame TIR-1, mixed TIR-1 + water-vapour). All produced plausible single-step forecasts. All failed catastrophically autoregressively: once the model's slightly imperfect prediction is fed back as the "clean" conditioning input, small artefacts are amplified at every step until the output degenerates into blocky, physically meaningless texture.
• 2 · Pivot to CorrDiff → stable, but physically wrong I split the task in two, following the CorrDiff recipe[1]. A base U-Net trained under MSE predicts t+30 and, as expected, produces a blurry but stable low-variance forecast. A second EDM diffusion model is then trained to predict only the residual (true − base_pred), conditioned on the base prediction itself. The simple model handles large-scale advection; the diffusion model paints the sharp detail back over the blur. This removed the collapse. But it revealed a subtler failure: significant positional drift, and a complete inability to model cloud dissipation, so the model kept forecasting cloud systems that had already decayed.
• 3 · Physics-conditioned CorrDiff → dissipation starts to appear I gave the base U-Net the physical context it was missing. Two channels of dense optical flow (Farneback, computed on the two input frames) supply an explicit motion field. One channel of normalised minute-of-day supplies the diurnal cycle, which governs the formation and dissipation of convective cloud. The base U-Net now takes 5 input channels, and a new CorrDiff model is trained on its residuals.

Results

The physics-conditioned CorrDiff model gives a tangible, though modest, improvement over the 2-channel version. Generative collapse is decisively solved: the forecasts stay sharp and structurally plausible across the full 3-hour horizon. And with flow and time conditioning, the base prediction now attempts to model large-scale dissipation, which the 2-channel base completely failed to do, giving the diffusion stage a better foundation to refine.

Challenges

The core difficulty is autoregressive instability: small prediction errors are amplified when fed back, and the 30-minute resolution of INSAT-3DR makes this almost unavoidable, since each step must predict a much larger and less predictable change than 10-minute data would demand.

There is also a conceptual mismatch I could not afford to fix. Diffusion models are trained on a single-step denoising objective, but inference is a multi-step sampling process, so there is no straightforward way to impose a true autoregressive loss. Doing it properly would require running the full ~30–50 step EDM sampler inside every training step, which was computationally out of reach.

Conclusion

This work validates that CorrDiff is a substantially more stable architecture than plain diffusion for autoregressive nowcasting, but demonstrates that stability alone is insufficient. Adding explicit physical conditioning (motion and diurnal phase) to the base model is a necessary further step, because it begins to address the physical processes that purely statistical models simply ignore.

References

1. R. J. Chase, K. Haynes, L. Ver Hoef, I. Ebert-Uphoff. Score-based diffusion nowcasting of GOES imagery. arXiv preprint, 2025.
2. T. Karras, M. Aittala, T. Aila, S. Laine. Elucidating the Design Space of Diffusion-Based Generative Models (EDM). NeurIPS, 2022.
3. S. Ravuri, K. Lenc, M. Willson, et al. Skilful precipitation nowcasting using deep generative models of radar. Nature, 597:672-677, 2021.
4. ISRO. INSAT-3DR satellite imagery, MOSDAC data archive.