PLATO++

Pose-Conditioned Part-Aware Object Generation via Residual Structure Learning

Varghese P Kuruvilla¹ · Harishwar Rao J.² · Ravi Kiran Sarvadevabhatla¹

¹IIIT Hyderabad · ²IISER Bhopal

IEEE/CVF CVPR Workshops 2026, Workshop on AI for Creative Visual Content Generation, Editing and Understanding (CVEU), pp. 6012-6021

Paper Project Page Workshop Code (soon)

PLATO++ teaser — Overview of PLATO++: the framework disentangles coarse pose control from fine-grained structural synthesis.

Overview

Generative models are very good at drawing a horse. They are much worse at drawing a horse facing north-east, with the parts you asked for, in an anatomically consistent arrangement. PLATO++ handles this by generating structure first and pixels second.

Modern text-to-image systems[7][8] inherit a canonical bias from their training data: the overwhelming majority of images of an articulated object show it from a handful of familiar viewpoints. Ask for a non-standard pose and the model either quietly reverts to the canonical one, or produces anatomy that falls apart, with parts detached, duplicated, or connected in ways the object class never allows.

PLATO++ separates the problem into two stages. A hierarchical graph-convolutional VAE first predicts a part layout conditioned on the object class, the requested part specification, and a target orientation. A layout-conditioned diffusion transformer then renders that layout into a photorealistic image. Because the pose is decided in the layout stage, where it is a cheap, explicit, low-dimensional variable, the image stage never has to negotiate between "look realistic" and "obey the pose."

Contributions

Dual-granularity pose control

Coarse global orientation via discrete cardinal-direction embeddings (N, S, E, W, NE, NW, SE, SW), and fine-grained articulation through FiLM-style modulation of the latent.

Residual structure learning

A residual adjacency formulation for graph-based VAEs: part connectivity is modelled as a correction to a learned class-specific base skeleton, rather than predicted from scratch every time.

Generative layout-to-image pipeline

End-to-end conversion of predicted part layouts into ellipse-inscribed control representations, fed to an OminiControl[3] diffusion transformer alongside a text prompt.

Long-tail pose synthesis

The combination makes it possible to synthesise object poses that sit far out in the tail of the training distribution, which is where baseline generators tend to collapse.

Method

PLATO++ layout generator architecture — Architecture of the PLATO++ layout generator, showing pose conditioning and residual adjacency prediction.

Stage 1: Layout generator

Objects are represented as graphs over their semantic parts. A GCN encoder compresses the input graph into a latent code; a hierarchical decoder then reconstructs the part bounding boxes using two conditioning signals:

Pose embeddings: a discrete embedding per cardinal direction sets the global orientation of the layout.
Residual adjacency prediction: instead of predicting the full part-connectivity matrix, the decoder predicts a residual on top of a learned per-class base skeleton, so the model only has to learn how a given instance deviates from the class norm.

Adjacency is discrete, which normally breaks gradients. We use a Gumbel-Sigmoid relaxation with a straight-through estimator so the connectivity decisions stay differentiable during training while remaining hard at inference.

Why residual adjacency helps. A quadruped's parts connect the same way almost every time. Predicting the whole adjacency matrix from scratch spends capacity re-learning that shared skeleton on every forward pass and makes rare, valid deviations look like noise. Predicting a residual keeps the prior for free and lets the model spend its capacity on the part that actually varies.

Stage 2: Image generator

Generated layouts are converted into ellipse-inscribed control images: each predicted part box becomes an inscribed ellipse, giving the renderer a soft, shape-like spatial hint rather than a hard rectangle. These control images condition a diffusion transformer (OminiControl[3]) alongside a descriptive text prompt, so the synthesised image respects both the semantic content of the prompt and the structural constraints of the layout.

Layout-to-image control pipeline — Layout-to-image pipeline: predicted part layouts are converted into ellipse-based control images.

Results

77.6% Pose accuracy
cardinal (N/S/E/W)

61.2% Pose accuracy
all 8 directions

0.57 Layout mIoU
vs 0.46 for PLATO

Layout reconstruction

Method	Layout mIoU ↑
PLATO[1]	0.46
PLATO++ (residual adjacency)	0.57

Residual adjacency accounts for an 11-point mIoU gain over the PLATO[1] baseline. PLATO++ also improves on both part-level metrics, Part-Layout-F1 (PL-F1) and Part-Layout-IoU (PL-IoU).

Pose control

Accuracy is measured as the fraction of generations whose realised orientation matches the requested cardinal direction. The gap between the 4-direction (77.6%) and 8-direction (61.2%) settings is the expected one: the diagonal directions are rarer in the training data and harder to distinguish from their neighbours.

Pose-conditioned generation results — Qualitative results on pose-conditioned generation across diverse object categories.

Adjacency prediction results — Qualitative results on adjacency prediction, showing non-canonical part interactions that a fixed class skeleton cannot express.

Part-level metric comparison — Part-Layout-F1 (PL-F1) and Part-Layout-IoU (PL-IoU) comparison against the PLATO baseline.

My Role

I worked on this project as a research intern at CVIT, IIIT Hyderabad under Prof. Ravi Kiran Sarvadevabhatla, contributing to the layout-generation stage and the layout-conditioned diffusion pipeline, and to the experimental analysis of how structured conditioning survives the transition from layout space into image space.

Citation

@InProceedings{Kuruvilla_2026_CVPR,
    author    = {Kuruvilla, Varghese P and J., Harishwar Rao and Sarvadevabhatla, Ravi Kiran},
    title     = {PLATO++: Pose-Conditioned Part-Aware Object Generation via Residual Structure Learning},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2026},
    pages     = {6012-6021}
}

References

A. Muthal, V. P. Kuruvilla, R. K. Sarvadevabhatla. PLATO: Generating Objects from Part Lists via Synthesized Layouts. ACM Multimedia, pages 9940-9949, 2025.
R. Baghel, A. Trivedi, T. Ravichandran, R. K. Sarvadevabhatla. MeronymNet: A Hierarchical Model for Unified and Controllable Multi-Category Object Generation. ACM Multimedia, pages 318-326, 2021.
Z. Tan, S. Liu, X. Yang, Q. Xue, X. Wang. OminiControl: Minimal and Universal Control for Diffusion Transformer. ICCV, pages 14940-14950, 2025.
R. Parihar, V. Agrawal, Sachidanand VS, R. V. Babu. Compass Control: Multi-Object Orientation Control for Text-to-Image Generation. CVPR, pages 2791-2801, 2025.
Y. Li, H. Liu, Q. Wu, F. Mu, J. Yang, J. Gao, C. Li, Y. J. Lee. GLIGEN: Open-Set Grounded Text-to-Image Generation. CVPR, pages 22511-22521, 2023.
G. Zheng, X. Zhou, X. Li, Z. Qi, Y. Shan, X. Li. LayoutDiffusion: Controllable Diffusion Model for Layout-to-Image Generation. CVPR, pages 22490-22499, 2023.
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer. High-Resolution Image Synthesis with Latent Diffusion Models. CVPR, 2022.
D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, R. Rombach. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. 2023.
J. Hessel, A. Holtzman, M. Forbes, R. Le Bras, Y. Choi. CLIPScore: A Reference-Free Evaluation Metric for Image Captioning. 2022.

The list above covers the works this page discusses directly. The paper's full bibliography is in the PDF.

Diffusion Transformer Graph VAE Layout Generation Pose Conditioning CVPRW 2026