1. Installation requirements

BIAFLOWS runs in Docker containers, the only requirement is to install Docker. Check official Docker documentation to install Docker for Ubuntu. Choose Install using the repository, set up the repository and install Docker CE.

2. Retrieve BIAFLOWS installation files

Enter the following commands in a terminal:

mkdir Biaflows
cd Biaflows
git clone https://github.com/Neubias-WG5/Biaflows-bootstrap.git
cd Biaflows-bootstrap

3. Configure the local instance

Edit configuration.sh and, if needed, update URLs (CORE_URL, IMS_URL, UPLOAD_URL).Make sure to use URLs that are not already used by other applications (avoid localhost) to prevent conflicts.

# These URLs have to be accessible (add them to /etc/hosts if you run it locally)
CORE_URL=biaflows
IMS_URL=biaflows-ims
UPLOAD_URL=biaflows-upload

In /etc/hosts of the host machine, add the following lines, adapting them accordingly to chosen XXX_URL in configuration.sh, for example:

127.0.0.1 biaflows
127.0.0.1 biaflows-ims
127.0.0.1 biaflows-upload

If needed, update data path variables (IMS_STORAGE_PATH …).

# These paths must exist
IMS_STORAGE_PATH=/biaflows/data/images
IMS_BUFFER_PATH=/biaflows/data/_buffer
SOFTWARE_DOCKER_IMAGES_PATH=/biaflows/data/workflows
SERVER_SSHKEYS_PATH=/biaflows/data/ssh

All data paths must be valid and mappable in the Docker engine. If they don't exist, create all the directories (mkdir) corresponding to the above variables, for example:

mdkir -p /biaflows/data/images
mdkir -p /biaflows/data/_buffer
mdkir -p /biaflows/data/workflows
mdkir -p /biaflows/data/ssh

A reference to these URLs and paths is provided here: https://doc.uliege.cytomine.org/display/PubOp/Cytomine+configuration+reference

Configure BIAFLOWS_WORKFLOWS_METRICS to true or false depending if you want to perform benchmarking. If set to true, ground truth annotations are then required for all images. Setting this flag to false is the valid option if you plan to manage and process local images.

4. Initialize the deployment

Run the installation script: sudo bash init.sh

5. Deploy the local instance

Run the generated deployment script: sudo bash start.sh

6. Check the running instance

When start up is finished, check that the application is running in your browser at the URL specified by CORE_URL (default: http://biaflows).

Three accounts with different access rights are automatically created:

• username: admin / password: admin
• username: guest / password: guest
• username: neubias / password: neubias

Passwords should be updated from the Account page (top right).

7. Install sample projects (images and ground-truth data)

After BIAFLOWS is successfully installed locally, the local instance is empty. All projects available in BIAFLOWS online instance can be imported to the local instance. For this, get the public and private keys of the admin account (Account page), then run:

cd Biaflows-bootstrap
sudo bash ./inject_demo_data.sh ADMIN_PUBLIC_KEY ADMIN_PRIVATE_KEY

where ADMIN_PUBLIC_KEY and ADMIN_PRIVATE_KEY have been substituted with their respective values.

The script starts to download projects and import them in your local BIAFLOWS. The list of imported projects can be tweaked by editing the file Biaflows-bootstrap/configs/project_migrator/projects.txt.

The whole data injection procedure can take several minutes, depending on your Internet connection and the number of projects being imported.

1. Create a new problem

To create a new problem, provided your user has rights, go to the Problems tab

and click on

2. Choose a meaningful problem name and save

3. Problem configuration

The problem is ready to be configured, if you hold annotations for all the images of the problem and plan to perform benchmarking the following configuration is recommended:

If you do not hold ground truth annotations for your images or do not want to perform benchmarking, untick the next to last option.

It is also possible to specificy if annotations from workflow results should be created and uploaded to BIAFLOWS after every workflow run with the last option.

Note: Currently, the ground truth annotations layer is not created automatically but by a specific Python script, this will be automated in a future release. It however does not affect the whole benchmarking process in any way.

4. Assign the problem to a problem class

Assign your problem to a problem class (see Problem Class, Ground truth annotations and reported metrics) by clicking on Change problem class. The problem class specifies the format of ground truth annotations (and workflow outputs), as well as the associated benchmark metrics to be computed (if benchmark is enabled).

5. Configure project members

If you work alone, you can leave contributors and project managers to default user. This can be done from the Members tab in the problem configuration.

6. Customize the User Interface

The problem can be fully configured to display or hide panels / tabs / tools in the user interface. This is achieved from the Custom UI tab in the problem configuration.

7. Add a description to the problem

A description of the problem can optionally be added from the Information (left sidebar). The description is displayed in Problems list.

1. Connect as user with image upload rights

2. Supported format

• 2D images: 8-bit/16-bit OME-TIFF files (or TIFF files)
• Multi-dimensional images (Z, C, T): 8-bit/16-bit Multi-TIFF OME-TIFF files.

3. Go to storage section

4. Select the Problem

Select the Problem to which the images should be associated with (Link with problem).

5. Add the images

Click on Add files… and select the files from the file browser.

6. Upload the images

Start upload with Start upload and wait until completion

The status can be:

• DEPLOYED/CONVERTED: The image is correctly imported to BIAFLOWS
• ERROR FORMAT: The file format is not supported
• ERROR EXTRACTION: Something went wrong during metadata extraction
• ERROR CONVERSION: Something went wrong during the conversion of the image into the BIAFLOWS internal image format
• ERROR DEPLOYMENT: Something went wrong during the communication with BIAFLOWS API. It can be due to access rights, or other unexpected error

1. Add the Ground Truth Images

Image annotations (e.g. binary masks) should be uploaded as 16-bit OME-TIFF (or TIFF) files for 2D images and as Multi-TIFF 16-bit OME-TIFF files for multidimensional (C,Z,T) images. They should be uploaded by following the procedure described in the previous section and by setting the same name as their corresponding image + _lbl suffix (e.g. myImage.ome.tif and myImage_lbl.ome.tif).

Other required annotations (e.g. SWC, division text file) should be added to the images as attached files. To do so, expand the image (blue arrow) in the list and click on Add next to Attached files.

2. Convert the Ground Truth Images to Annotations

In order to use the ground truth uploader tool you need the following tool:

• git
• docker

2.1 Get the tool

git clone https://github.com/Neubias-WG5/ground-truth-uploader.git

2.2 Install the tool

git clone https://github.com/Neubias-WG5/ground-truth-uploader.git
cd ground-truth-uploader/
docker build -t gt_uploader .

2.3 Generate the ground truth

To run the tool, you need the following information:

Cytomine host name :

Enter the domain name where is installed biaflows, e.g. biaflows-sandbox.neubias.org

Biaflows public/private keys:

Go to your account setting:

Copy the public /private key from there

Project ID number :

Go to the project and copy from the URL the project number

You can now generate the ground truth annotation from the labels by running the following command:

docker run -t TAG --cytomine_host CYTOMINE_HOST --cytomine_public_key PUBLIC_KEY --cytomine_private_key PRIVATE_KEY --cytomine_project_id PROJECT_ID							
							

where, in this example, TAG is gt_uploader CYTOMINE_HOST is biaflows-sandbox.neubias.org, PROJECT_ID is 14740414

1. Connect with a user having project management rights

Connect as administrator by clicking Open admin session.

2. Add trusted source

In the administration page, go to Trusted sources tab and click New trusted source.

3. Save the trusted source

Fill the form and Save

For instance, to add NEUBIAS curated set of workflows, the trusted source has to be configured as follows:

• Source code provider: github
• Source code provider username: Neubias-WG5
• Environment provider: docker
• Environment provider username: neubiaswg5
• Prefix: W_

4. Manually import the new workflows

Trusted sources are periodically checked (about every 5/10 minutes) to automatically add new versions of existing workflows or new workflows, but you can also click on Refresh to trigger the check.

5. Link imported workflows

Once a workflow is imported, it has to be linked to a BIAFLOWS Problem. This can be performed in the Configuration panel of the Problem (Workflows tab) by toggling Enable for that workflow as illustrated below: