Learning to Reconstruct Botanical Trees from Single Images

We introduce a novel method for reconstructing 3D geometry of botanical trees from a single photograph. Faithfully reconstructing a tree from single-view sensor data is a challenging and open problem because many possible 3D trees exist that fit the tree's shape observed from a single view. We address this challenge by defining a reconstruction pipeline based on three neural networks. The networks simultaneously mask out trees in input photographs, identify the species of a tree, and obtain its 3D radial bounding volume. Radial bounding volumes (RBV) are used to orchestrate a procedural model primed on learned parameters to grow a tree that matches the main branching structure and the overall shape of the captured tree. While the bounding volume allows us to faithfully reconstruct the main branching structure, we use the procedural model's morphological constraints to generate realistic branching. This constraints the number of solutions of tree models for a given photograph of a tree. We show that our method reconstructs various tree species even when the trees are captured in front of complex backgrounds. Moreover, although our neural networks have been trained on synthetic data with data augmentation, we show that our pipeline performs well for real tree photographs. We evaluate the reconstructed geometries with a number of tree form metrics, including density histograms, fractal dimension, and statistics.

• Machine Intelligence

• Machine Perception