How to use the romi_run_task
generic command ?Link
We here assume you followed the "installation instructions" available here.
Getting startedLink
There are some requirements to use the different algorithms in the pipeline. Most of them are installed automatically from the requirements file when using pip. The most important part is Colmap (v3.6).
The two requirements that are not shipped with pip
are:
- Colmap (v3.6 or v3.7) for the structure from motion algorithms
- Blender (>= 2.81) to be able to use the virtual plant imager
Preferably, create a virtual environment for python 3.7 or python 3.8 using virtualenv
or a conda environment specific to the 3D Scanner.
Warning
If using python 3.8, Open3D binaries are not yet available on pip
, therefore you have to build Open3D from sources!
Basic usageLink
Every task is launched through the romi_run_task
command provided in the romitask
library.
It is a wrapper for luigi
, with preloaded tasks from the romitask
, plantimager
& plant3dvision
modules.
The general usage is as follows:
romi_run_task [-h]
[--config CONFIG]
[--luigicmd LUIGICMD]
[--module MODULE]
[--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
[local-scheduler]
task
[dataset_path]
CONFIG
is either a file or a folder. If a file, it must be JSON or TOML and contains the configuration of the task to run. If a folder, it will read all configuration files in JSON or TOML format from the folder.LUIGICMD
is an optional parameter specifying an alternative command forluigi
.MODULE
is an optional parameter for running task from external modules (see TODO).LOG_LEVEL
is the level of logging. Defaults toINFO
, but can be set toDEBUG
to increase verbosity.task
is the name of the class to run, see here for a list of task or have a look at the output ofromi_run_task -h
for a more up-to-date listdataset_path
is the location of the targetScan
on which to process the task. It is of the form$DB_LOCATION/SCAN_ID
, where$DB_LOCATION
is a path containing theplantdb
marker.
Defining the dataset pathLink
To defines the dataset_path
you can specify a dataset name like my_scan_007
.
You may also use Unix pattern matching with "*
" and "?
" to select a list of scan dataset.
For example, in a DB with the following scan dataset: test_1
my_scan_test
, my_scan_002
, my_scan_007
, my_scan_011
$DB_LOCATION/*
will match ALL scan dataset$DB_LOCATION/my_scan_*
will matchmy_scan_test
,my_scan_002
,my_scan_007
&my_scan_011
$DB_LOCATION/my_scan_???
will matchmy_scan_002
,my_scan_007
&my_scan_011
$DB_LOCATION/my_scan_00?
will matchmy_scan_002
&my_scan_007
$DB_LOCATION/my_scan_00*
will matchmy_scan_002
&my_scan_007
Configuration filesLink
The configuration is in the form of a dictionary, in which each key is the ID of a given task.
In TOML format, it reads as follows:
[FirstTask]
parameter1 = value1
parameter2 = value2
[SecondTask]
parameter1 = value1
parameter2 = value2
An example TOML configuration file for the geometric reconstruction pipeline of a point-cloud is:
[Colmap]
upstream_task = "ImagesFilesetExists"
matcher = "exhaustive"
compute_dense = false
align_pcd = true
use_gpu = true
single_camera = true
robust_alignment_max_error = 10
[Undistorted]
upstream_task = "ImagesFilesetExists"
[Masks]
upstream_task = "Undistorted"
query = "{\"channel\":\"rgb\"}"
type = "linear"
parameters = "[0, 1, 0]"
dilation = 3
binarize = true
threshold = 0.2
[Voxels]
upstream_mask = "Masks"
upstream_colmap = "Colmap"
voxel_size = 0.5
type = "carving"
[PointCloud]
upstream_task = "Voxels"
level_set_value = 0.0
To run the full reconstruction pipeline use this configuration file with romi_run_task
:
romi_run_task --config scanner.json AnglesAndInternodes /path/to/db/scan_id/ --local-scheduler
This will process all tasks up to the AnglesAndInternodes
task.
Every task produces a Fileset
, a subdirectory in the scan directory whose name starts the same as the task name.
The characters following are a hash of the configuration of the task, so that the outputs of the same task with different parameters can coexist in the same scan.
Any change in the parameters will make the needed task to be recomputed with subsequent calls of romi_run_task
.
Already computed tasks will be left untouched.
To recompute a task, just delete the corresponding folder in the scan directory and rerun romi_run_task
.
Default task referenceLink
MODULES = {
# Scanning module:
"Scan": "plantimager.tasks.scan",
"ScannerToCenter": "plantimager.tasks.scan",
"VirtualPlant": "plantimager.tasks.lpy",
"VirtualScan": "plantimager.tasks.scan",
"CalibrationScan": "plantimager.tasks.scan",
"IntrinsicCalibrationScan": "plantimager.tasks.scan",
# Calibration module:
"CreateCharucoBoard" : "plant3dvision.tasks.calibration",
"DetectCharuco" : "plant3dvision.tasks.calibration",
"ExtrinsicCalibration" : "plant3dvision.tasks.calibration",
"IntrinsicCalibration" : "plant3dvision.tasks.calibration",
# Geometric reconstruction module:
"Colmap": "plant3dvision.tasks.colmap",
"Undistorted": "plant3dvision.tasks.proc2d",
"Masks": "plant3dvision.tasks.proc2d",
"Voxels": "plant3dvision.tasks.cl",
"PointCloud": "plant3dvision.tasks.proc3d",
"TriangleMesh": "plant3dvision.tasks.proc3d",
"CurveSkeleton": "plant3dvision.tasks.proc3d",
# Machine learning reconstruction module:
"Segmentation2D": "plant3dvision.tasks.proc2d",
"SegmentedPointCloud": "plant3dvision.tasks.proc3d",
"ClusteredMesh": "plant3dvision.tasks.proc3d",
"OrganSegmentation": "plant3dvision.tasks.proc3d",
# Quantification module:
"TreeGraph": "plant3dvision.tasks.arabidopsis",
"AnglesAndInternodes": "plant3dvision.tasks.arabidopsis",
# Evaluation module:
"VoxelsGroundTruth": "plant3dvision.tasks.evaluation",
"VoxelsEvaluation": "plant3dvision.tasks.evaluation",
"PointCloudGroundTruth": "plant3dvision.tasks.evaluation",
"PointCloudEvaluation": "plant3dvision.tasks.evaluation",
"ClusteredMeshGroundTruth": "plant3dvision.tasks.evaluation",
"SegmentedPointCloudEvaluation": "plant3dvision.tasks.evaluation",
"Segmentation2DEvaluation": "plant3dvision.tasks.evaluation",
"AnglesAndInternodesEvaluation": "plant3dvision.tasks.evaluation",
"CylinderRadiusGroundTruth": "plant3dvision.tasks.evaluation",
"CylinderRadiusEstimation": "plant3dvision.tasks.evaluation",
# Visualization module:
"Visualization": "plant3dvision.tasks.visualization",
# Database module:
"Clean": "romitask.task"
}
Warning
This is for reference only, please update the changes in the code. This will be later replaced by a reference doc generated from the code!
Class name: Scan
Module: plantimager.tasks.scan
Description: A task for running a scan, real or virtual.
Default upstream tasks: None
Parameters:
- metadata (DictParameter) : metadata for the scan
- scanner (DictParameter) : scanner hardware configuration (TODO: see hardware documentation)
- path (DictParameter) : scanner path configuration (TODO: see hardware documentation)
Class name: CalibrationScan
Module: plantimager.tasks.scan
Description: A task for running a scan, real or virtual, with a calibration path. It is used to calibrate
Colmap poses for subsequent scans. (TODO: see calibration documentation)
Default upstream tasks: None
Parameters:
- metadata (DictParameter) : metadata for the scan
- scanner (DictParameter) : scanner hardware configuration (TODO: see hardware documentation)
- path (DictParameter) : scanner path configuration (TODO: see hardware documentation)
- n_line : number of shots taken on the orthogonal calibration lines
Class name: Clean
Module: plantimager.tasks.scan
Description: Cleanup a scan, keeping only the "images" fileset and removing all computed pipelines.
Default upstream tasks: None
Parameters:
- no_confirm (BoolParameter, default=False) : do not ask for confirmation in the command prompt.
Class name: Colmap
Module: plant3dvision.tasks.colmap
Description: Runs colmap on a given scan.
Default upstream tasks: Scan
Upstream task format: Fileset with image files
Output fileset format: images.json, cameras.json, points3D.json, sparse.ply [, dense.ply]
Parameters:
- matcher (Parameter, default="exhaustive") : either "exhaustive" or "sequential" (TODO: see Colmap documentation)
- compute_dense (BoolParameter) : whether to run the dense Colmap to obtain a dense point cloud
- cli_args (DictParameter) : parameters for Colmap command line prompts (TODO: see Colmap documentation)
- align_pcd (BoolParameter, default=True) : align point cloud on calibrated or metadata poses ?
- calibration_scan_id (Parameter, default="") : ID of the calibration scan.
Class name: Undistorted
Module: plant3dvision.tasks.proc2d
Description: Undistorts images using computed intrinsic camera parameters
Default upstream tasks: Scan, Colmap
Upstream task format: Fileset with image files
Output fileset format: Fileset with image files
Class name: Masks
Module: plant3dvision.tasks.proc2d
Description: compute masks using several functions
Default upstream tasks: Undistorted
Upstream task format: Fileset with image files
Output fileset format: Fileset with grayscale or binary image files
Parameters:
- type (Parameter) : "linear", "excess_green", "vesselness", "invert" (TODO: see segmentation documentation)
- parameters (ListParameter) : list of scalar parameters, depends on type
- dilation (IntParameter) : by how much to dilate masks if binary
- binarize (BoolParameter, default=True) : binarize the masks
- threshold (FloatParameter, default=0.0) : threshold for binarization
-
Class name: Segmentation2D
Module: plant3dvision.tasks.proc2d
Description: compute masks using trained deep learning models
Default upstream tasks: Undistorted
Upstream task format: Fileset with image files
Output fileset format: Fileset with grayscale image files, each corresponding to a given input image and class
Parameters:
- query (DictParameter) : query to pass to upstream fileset. It filters file by metadata, e.g {"channel": "rgb"} will process
only input files such that "channel" metadata is equal to "rgb".
- labels (Parameter) : string of the form "a,b,c" such that a, b, c are the identifiers of the labels produced by the neural
network
- Sx, Sy (IntParameter) : size of the input of the neural network. Input pictures are cropped in the center to this size.
- model_segmentation_name : name of ".pt" file that can be found at `https://db.romi-project.eu/models`
-
Class name: Voxels
Module: plant3dvision.tasks.cl
Description: Computes a volume from back-projection of 2D segmented images
Default upstream tasks:
- upstream_mask: Masks
- upstream_colmap: Colmap
Upstream task format:
- upstream_mask: Fileset with grayscale images
- upstream_colmap: Output of Colmap task
Output fileset format: npz file with as many arrays as classes
Parameters:
- use_colmap_poses (BoolParameter, default=True): Either use precomputed camera poses or output from the Colmap task
- voxel_size (FloatParameter): size of one side of voxels
- type (Parameter): "carving" or "averaging" (TODO: See 3D documentation)
- multiclass (BoolParameter, default=False): whether input data is single class or multiclass (e.g as an output of Segmentation2D)
- log (BoolParameter, default=True), in the case of "averaging" type, whether to apply log when averaging values.
Class name: PointCloud
Module: plant3dvision.tasks.scan
Description: Computes a point cloud from volumetric voxel data (either single or multiclass)
Default upstream tasks: Voxels
Upstream task format: npz file with as many 3D array as classes
Output task format: single point cloud in ply. Metadata may include label name if multiclass.
Class name: TriangleMesh
Module: plant3dvision.tasks.scan
Description: Triangulates input point cloud. Currently ignores class data and needs only one connected component.
Default upstream tasks: PointCloud
Upstream task format: ply file
Output task format: ply triangle mesh file
Class name: CurveSkeleton
Module: plant3dvision.tasks.scan
Description: Creates a 3D curve skeleton
Default upstream tasks: TriangleMesh
Upstream task format: ply triangle mesh
Output task format: json with two entries "points" and "lines" (TODO: precise)
Class name: TreeGraph
Module: plant3dvision.tasks.arabidopsis
Description: Creates a tree graph of the plant
Default upstream tasks: CurveSkeleton
Upstream task format: json
Output task format: json (TODO: precise)
Class name; AnglesAndInternodes
Module: plant3dvision.tasks.arabidopsis
Description: Computes angles and internode
Default upstream tasks: TreeGraph
Upstream task format: json
Output task format: json (TODO: precise)
Scanner API referenceLink
ObjectsLink
/objects
(GET): retrieve the list ofobj
files in the data folder that can be loaded./load_object/<object_id>
(GET) load the given object in the scene. Takes a translation vector as URL parameters (dx
,dy
,dz
)
ClassesLink
/classes
(GET): retrieve the list of classes.
BackgroundsLink
/backgrounds
(GET): retrieve the list ofhdr
files in thehdri
folder that can be loaded./load_background/<background_id>
(GET) load the given background in the scene.
CameraLink
/camera_intrinsics
(POST): set camera intrinsics. Keys:width
,height
,focal
/camera_pose
(POST): set camera pose. Keys:tx
,ty
,tz
,rx
,ry
,rz
RenderingLink
/render
(GET): gets the rendering of the scene/render_class/<class_id>
(GET) renders the scene, with everything transparent except the given class
TODO: missing endpoints
# Setup camera
http -f post http://localhost:5000/camera_intrinsics width=1920 height=1080 focal=35
# Load arabidopsis_0
http get 'http://localhost:5000/load_object/arabidopsis_0.obj?dx=10&dy=20&dz=1'
# Load "old tree in the park" background
http get http://127.0.0.1:5000/load_background/old_tree_in_city_park_8k.hdr
# Move camera
http -f post http://localhost:5000/camera_pose tx=-60 ty=0 tz=50 rx=60 ry=0 rz=-90
# Render scene and download image
http --download get http://localhost:5000/render
# Render only leaves
http --download get http://localhost:5000/render_class/Color_7