FundusFlow++

Controllable generation of retinal fundus images together with their anatomical annotations

Harishwar Rao J. · Chandra Sekhar Seelamantula

Spectrum Lab, Indian Institute of Science (IISc), Bengaluru · Home institute: IISER Bhopal

Paper (in preparation) Code (on publication)

Note. This work is ongoing and unpublished, so this page describes only what the project is about and shows a few sample outputs. Method and results are held back until the manuscript is out.

What This Project Is About

Deep learning for retinal imaging is bottlenecked by annotation, not by images. Labelling retinal anatomy is slow, needs expertise, and the labels that matter most clinically, down to which vessels are arteries and which are veins, are the most expensive of all.

FundusFlow++ approaches this from the generative side. Rather than generating a fundus image and trying to annotate it afterwards, the system generates the anatomy and the photograph together, so every synthetic sample arrives with a pixel-perfect, clinically meaningful label attached. The image synthesis is built on flow-matching generative models[1].

My work extends an earlier system from the lab along two lines: making the generated anatomy richer and more clinically informative: in particular, distinguishing arteries from veins rather than treating all vessels alike, and making that anatomy anatomically consistent and directly controllable, so you can ask for a specific layout instead of sampling and hoping.

Controllable Anatomy

A synthetic dataset is much more useful if you can steer it rather than only sample from it. Here the optic disc is placed at nine different positions across the retina, and a complete, coherent vessel layout is produced around each requested placement.

Controllable optic disc placement — Controllable anatomy: the optic disc (green) is requested at nine different locations, and the surrounding vasculature is generated to match. Arteries in red, veins in blue.

Anatomy Taking Shape

Two samples, animated as they are synthesised.

Retinal anatomy being synthesised — Arteries in red, veins in blue, optic disc in green.

Sample Outputs

Each sample is an anatomy map paired with the fundus photograph synthesised from it, so the label and the image are produced as one unit, so the annotation is exact by construction rather than estimated after the fact.

Generated anatomy masks and fundus images — Generated pairs: artery/vein-labelled anatomy map (left of each pair) and the corresponding synthesised fundus image (right).

Artery/vein labelling overlaid on a fundus image. This is the distinction that makes the synthetic data useful for downstream clinical tasks.

Status

The pipeline runs end to end and produces controllable, artery/vein-labelled samples. Current directions include broader evaluation, and extending the system to pathological retinas rather than healthy ones only. A manuscript is in preparation; code and full results will follow publication.

References

Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, M. Le. Flow Matching for Generative Modeling. ICLR, 2023.
G. Lepetit-Aimon, C. Playout, M. C. Boucher, R. Duval, M. H. Brent, F. Cheriet. MAPLES-DR: MESSIDOR Anatomical and Pathological Labels for Explainable Screening of Diabetic Retinopathy. Scientific Data, 2024.
K. Jin, X. Huang, J. Zhou, Y. Li, Y. Yan, et al. FIVES: A Fundus Image Dataset for Artificial Intelligence based Vessel Segmentation. Scientific Data, 2022.

Method references are held back until the manuscript is public.

Flow Matching Medical Imaging Controllable Generation Synthetic Data Ongoing