Demeter: A Parametric Model of Crop Plant Morphology from the Real World

AI summary

Overview

Research area: Computer Vision and Graphics — data-driven 3D parametric shape modeling, applied to crop plants (plant phenotyping and biophysical simulation).

Technical level: Advanced.

Scope: The paper introduces Demeter, a learned parametric model that represents a whole crop plant's topology, articulation, shape, and non-rigid deformation, trained on real-world farm data and demonstrated on soybean, maize, pepper, rose, tobacco, and papaya.

What This Paper Is About

Powerful learned parametric shape models exist for human bodies and animals, but there is no equally expressive model for plants, which vary in structure (topology) from one individual to the next. The authors build Demeter, a compact, data-driven parametric model that encodes a plant as topology, articulation, shape, and deformation parameters, and train it on real-world 3D soybean data captured at an Illinois farm over a full growing season. The goal is a model that can represent, reconstruct, animate, and simulate plants of a given species rather than relying on hand-crafted procedural rules.

Key Contributions

1. A data-driven parametric plant model. Demeter factorizes plant morphology into four components — topology (Γ), articulation (θ), shape (β), and non-articulated deformation (γ) — with shape and deformation compressed into learned PCA bases (Φ = {Φ_s, Φ_d}). Unlike prior human/animal models such as SMPL, it also encodes variable structural topology across individuals.

2. Real-world field data and a trained soybean model (Demeter-Soybean). The authors collected 3D soybean samples from a smart soybean farm in Illinois across an entire growing season — around 600 plants spanning multiple genotypes and all life stages from germination to pod formation — and used them to fit the model.

3. A new annotated dataset and benchmarks. Over 300 soybean plants were selected for detailed leaf and stem annotation, producing a dataset with 3D geometry, part segmentation, and topology annotations, plus benchmarks for 2D/3D semantic segmentation and 3D morphological shape reconstruction from images or point clouds.

4. Demonstrated reconstruction and simulation pipelines. Demeter parameters are used for multi-view and single-image 3D reconstruction, and the resulting meshes are fed into the Helios simulator to model photosynthesis.

Main Findings

• Multi-view reconstruction beats baselines on accuracy. In Chamfer distance (CD, normalized scale), Demeter-Soybean scored 0.0016 on soybean, compared with 0.0030 for NKSR and 0.0376 for SimpProc; on maize it scored 0.0071 versus 0.0023 (NKSR) and 0.0557 (SimpProc).

• Demeter is far more compact than a dense neural mesh. Storage size was 3.3750 KB (soybean) and 1.7656 KB (maize) versus 5785.6 KB and 3686.4 KB for NKSR, though SimpProc is smaller still at 0.2754 KB and 0.2236 KB. The paper reports that this is a comparable CD with far fewer parameters.

• NKSR's geometry is not biophysically usable. The paper reports NKSR achieves low error yet produces a triangular mesh with thick, unsmooth leaves and stems and poor connectivity, which prevents a functional biophysically plant model.

• Single-image reconstruction aligns better with the input. Measured by IoU between the rendered mask and the input image, Demeter reached 0.328, compared with 0.206 for Meshy and 0.296 for Zero12345++.

• The model generalizes across species. Demeter was learned for pepper and rose (from PLANesT3D), tobacco (from PLANT3D), and maize (from Pheno4D and a web dataset), and the papaya model was even used to fit a Ribes point cloud because the two species have similar leaves and no 2D leaf scans were available for Ribes.

• SimpleProc is too coarse. The procedural baseline SimpleProc, adapted from CropCraft, uses 8 global shape parameters (such as average leaf length and node count) fitted by Bayesian optimization of Chamfer distance, and the paper reports it is misaligned with the input and cannot capture individual leaf and stem shapes.

• Latent parameters are interpretable and controllable. The authors show branch cutting and duplication to change topology, PCA coefficient changes for leaf and stem shape, and stem deformation from straight to curved — all while keeping the plant plausible.

• Simulation works downstream. Small crop fields built from repeated Demeter-Soybean models were passed to Helios to simulate net photosynthesis over a day, using weather variables from a flux tower (temperature, humidity, radiation, and other environmental variables).

• Stated limitation. The model assumes leaves are 2D shapes and stems are curves with uniform thickness, excluding species with complex structures such as cacti, algae, and banyan trees, though it applies to many common species.

Methodology in Plain English

The authors treat a plant like a small skeleton of connected parts. A tree-structured graph records which leaf or stem attaches to which parent. Articulation parameters say how each part is rotated (as a quaternion), positioned relative to its parent, and scaled. Shape parameters describe the intrinsic form of each organ while ignoring bending. Deformation parameters describe how those organs bend and curl under external forces.

To keep the model small, they learn linear bases with PCA. Leaf shape comes from thousands of 2D leaf scans in the FGLIR and Folio datasets: each leaf is annotated at its base and tip, masked with SAM2, and its contour is divided in two; a smooth bijective mapping between a UV domain and the leaf domain is solved via the Laplace equation, so a grid of control points can be placed consistently across leaves. For deformation, they turn each leaf's joints into a 2D skeletal structure — one main vein with sub-veins — so bending preserves length and area; the same is done for stems with a 1D skeleton.

The 3D data itself came from a custom capture pipeline: over 600 multi-view RGB videos shot with a GoPro indoors and outdoors, structure-from-motion with Hloc for camera poses, 4DGS to reconstruct plants moving in wind, a freeze at t = 0 to render static views, and 2DGS to extract a mesh, with missing parts (especially stems) corrected by hand. Because shape, articulation, and deformation are entangled in real scans, the authors fit them in multiple stages, minimizing Chamfer distance between template meshes and point clouds — first fitting shape with no articulation constraints, then learning the deformation PCA basis, then jointly optimizing deformation and shape, and finally running forward kinematics to fit the whole plant and adjust stems.

For reconstruction at test time, where no annotations exist, they segment the point cloud with a PointTransformer-V3 network (trained on 80 cleaned soybean point clouds and 5 maize point clouds), cluster with DBSCAN, infer topology from a minimum spanning tree, and fit parameters. For single-image reconstruction they use SAM for masking, Mask-RCNN trained on 32K images across 60 plants for instance segmentation into leaf, stem, and main stem, and an off-the-shelf depth estimator to lift instances to 3D.

Why This Matters

Impact on research: Demeter brings plant modeling into the same learned-parametric-model framework that transformed human body modeling, replacing hand-crafted procedural rules with a compact, data-driven representation. Because it is reproducible and differentiable-ish in structure (primitives plus PCA bases), it makes inverse problems — estimating plant parameters from images or point clouds — far more tractable than inverse procedural modeling. The released dataset of over 300 annotated soybeans also gives the plant phenotyping community a real-world benchmark for 2D/3D segmentation and 3D shape reconstruction.

Real-world applications:

• Crop yield analysis and breeding: fitting Demeter models to field plants lets researchers quantify morphological traits across genotypes and growth stages.
• Biophysical simulation: the meshes feed directly into tools like Helios to simulate photosynthesis and stomatal conductance, connecting plant architecture to productivity.
• Environmental and agricultural monitoring: accurate 3D plant models support tracking crop health under real field conditions rather than only in controlled lab settings.
• Graphics and rendering: the supplementary shows Demeter models rendered in Blender with textures extracted from leaf images, useful for realistic vegetation in visual content.

Industry relevance: Agriculture technology, seed and agronomy companies, and phenotyping hardware/software vendors can use a compact parametric plant model as a shared representation for measuring, comparing, and simulating crops. The authors state they will open-source a Demeter-based platform inviting vision and plant phenotyping researchers to create and share models of various species.

Future Directions

1. Extending beyond 2D leaves and uniform-thickness stems. The stated limitation excludes cacti, algae, and banyan trees, so generalizing the primitives is an open problem.

2. Modeling omitted organs. The current model omits structures such as flowers and fruit, which matters for yield-related simulation.

3. Scaling the species library. The paper positions Demeter as a platform; an open question is how well the largely species-agnostic pipeline generalizes when many more species and 2D leaf scan sets are added.

4. Closing the gap between cheap procedural models and accurate learned ones. SimpProc remains far smaller in storage (0.2754 KB versus 3.3750 KB for soybean), so further compression and better inverse fitting for compact models remain open.

Target Audience

Researchers and graduate students in computer vision, computer graphics, and 3D reconstruction who work on parametric shape models; plant scientists and phenotyping engineers interested in quantitative crop morphology; and agricultural technology practitioners who need simulated or measured 3D plant geometry for biophysical modeling such as photosynthesis. Readers should be comfortable with PCA, kinematic chains, Chamfer distance, and Gaussian-splatting-based reconstruction.

Authors’ abstract

Learning 3D parametric shape models of objects has gained popularity in vision and graphics and has showed broad utility in 3D reconstruction, generation, understanding, and simulation. While powerful models exist for humans and animals, equally expressive approaches for modeling plants are lacking. In this work, we present Demeter, a data-driven parametric model that encodes key factors of a plant morphology, including topology, shape, articulation, and deformation into a compact learned representation. Unlike previous parametric models, Demeter handles varying shape topology across various species and models three sources of shape variation: articulation, subcomponent shape variation, and non-rigid deformation. To advance crop plant modeling, we collected a large-scale, ground-truthed dataset from a soybean farm as a testbed. Experiments show that Demeter effectively synthesizes shapes, reconstructs structures, and simulates biophysical processes. Code and data is available at https://tianhang-cheng.github.io/Demeter/.