The geometry based workflowLink
The first workflow we introduce is the one we called geometry based workflow as it transform the obtained point cloud, after reconstruction, to extract a semantic skeleton (called tree graph) and use it to compute the fruit orientation in space. From there we can fulfill our goal: to estimate the fruits' successive angles and internodes lengths.
The aim of the reconstruction part is to reconstruct a 3D model of the plant, here a point cloud, from a set of RGB images.
We sought at combining structure from motion with space carving to obtain a quick and reliable 3D reconstruction of the plant.
The rational is mainly in two part:
- We have to use structure from motion to get accurate estimate of the true camera positions because of the uncertainties from the motors (see open-loop-design).
- We use space carving, instead of multiview stereo (as in the second part of Colmap reconstruction pipeline), because we want a fast reconstruction of a small portion of the scene.
The second step is fast because we use a simple linear filter to detect the plant position, and we select a small region where there is only the plant to reconstruct.
However, without a precise and repeatable acquisition procedure with the Plant Imager, you might have to often change the bounding-box manually, which breaks the full automation of the reconstruction procedure.
- We start with the
Colmaptask to estimate both intrinsic and extrinsic parameters using a structure from motion algorithm.
- The camera intrinsics are used by the
Undistortedtask with a
SIMPLE_RADIALmodel to fix the original RGB images.
- Then the
Maskstask detect the plant position in each image and create a binary mask for each.
- This is later used by the
Voxelstask, in combination with the camera extrinsics (also called camera poses), to perform the space carving of a 3D volume. This reconstructs the volume occupied by the plant in the selected portion of the scene.
- Finally, this is turned into a point cloud describing the envelope of the reconstructed plant structure by the
The aim of the quantification part is estimate the fruits' successive angles and internodes lengths, from the 3D point cloud.
We sought at extracting the skeleton of the plant to be able to estimate the organs direction, here the fruits or leaves, thanks to a single trajectory of points describing it.
- We start by transforming the point cloud into a triangular mesh with the
- From there we can extract the plant skeleton thanks to the
- We then augment this structure with "biological meaning" to this skeleton by defining the root of the tree, labelling the points as fruits or main stem using the
- Finally, the
AnglesAndInternodestask will compute the fruits direction and branching points, allowing us to estimate the successive angles and internode lengths between the fruits.