Skip to content

Gearify the BIDS app template

Introduction

Creating a BIDS app gear can be accomplished by modifying Flywheel's BIDS app gear template. This tutorial will walk through the key features of the template and the modifications that need to be completed to gearify a BIDS app. Additional instructions are available in the README.

The largest departure from previous BIDS gear templating is the introduction of the bids_app_command configuration field.
Configuration tab

This field allows the user to paste lab-validated, SOP BIDS commands directly into the gear configuration. By including this free-text field, users do not have to click individual configuration options to build the BIDS command. Additionally, development time for updates decreases, since specific keyword arguments do not have to be added or removed from the code and manifest.json to reflect the current state of the BIDS App. Furthermore, the template utilizes a new method to check the free-text input quickly in the gear run and return errors (fail), if the user gives keywords that are not allowed by the current version of the BIDS App.

Instruction Steps

  1. Fork the BIDS App Template repository

  2. Edit the Dockerfile to point to the BIDS app base Docker image.

    • The Dockerfile is the basis of the gear. This template uses two stages to build the gear.
    • The essential pieces of Stage 1 ("fw_base") are copied into Stage 2 ("bids_runner") to keep the Flywheel and BIDS App dependencies separate. This separation ensures that updating Flywheel dependencies does not affect the original BIDS App Docker image/algorithm/dependencies in any way, maintaining reproducibility. Other than those packages already there, do not install additional packages or update packages in Stage 2 unless you are sure it will not affect the algorithm.
    • Specifics:
    • Stage 1 ("fw_base") does not usually require editing as it is a python base image for all the Flywheel-specific dependencies to make the gear work. You may want to upgrade the python version of the "fw_base" image past python3.9. If you do so, make sure that the alternate python version in Stage 2 ("bids_runner") and here matches the version that you have chosen for Stage 1 ("fw_base").
    • Stage 2 is the official base image of the BIDS App. Enter the DockerHub location of the image to be the "bids_runner". (e.g., "nipreps/fmriprep" is the location for the DockerHub version of fMRIPrep)

  3. Edit the manifest.json.

    • Fields in the manifest have descriptions to help you decide what to keep and what to toss.
    • Areas marked "editme" are explicitly expected to be updated per the algorithm being gearified. (example)
    • The most important field and biggest shift in this template is the bids_app_command configuration option. Rather than adding and maintaining all the possible keyword arguments that the BIDS App will accept, use this field to enter the BIDS command as you would on a CLI. This field gives full flexibility to the user to generate a gear run with any valid keyword argument to the BIDS command.
    • Other inputs or configurations may be added as needed, but please keep in mind the power of the bids_app_command configuration field and limit explicit configuration options. Please refer to basic gear development documentation for further direction on editing these sections.
  4. Run fw-beta gear build in your local CLI (from the top level of the gear, where the Dockerfile and manifest.json live).
    • The updated Dockerfile will be used to build the gear locally. If the fw-beta build succeeds, then no more edits are necessary to the Dockerfile.
    • The manifest.json will be updated automatically to include environmental variables for future Docker (or Singularity) runs. Keep these edits by committing and pushing the manifest changes from fw-beta gear build. Having the correct environmental variables will save headaches when debugging gear runs as you are refining the code.
  5. Look at the pyproject.toml file.
    • The packages listed here are for making the gear run, not for the BIDS algorithm. The poetry.lock file will be turned into requirements.txt that are submitted to the "fw_base" Stage 1 of the Dockerfile to create the correct "flypy" environment.
    • Are all the dependencies that you specifically need listed there?
    • Poetry will automatically resolve and update dependencies for you as long as the absolutely necessary packages are outlined in the tool dependency section
    • To ensure all dependencies are up to date with any of your changes, run pre-commit install, then poetry update && pre-commit run poetry_export -a from the local command line. (Installing pre-commit only needs to happen once.)
  6. Check whether a FreeSurfer license is needed for the BIDS algorithm.
    • If FreeSurfer is part of the BIDS App, then a license IS needed and nothing needs to be modified in the template. (License finding methods are currently handled by the GearToolkit.)
    • If no license is needed, check that there is no FREESURFER_HOME defined in the manifest environment variables.
    • If no license is needed, add the kwarg, find_fs_license=False, to setup_bids_env in run.py
  7. Address CI/CD files.
    • Flywheel uses GitHub pipelines to ensure gears continuously pass tests and meet various standards. The set up for these pipelines are in .gitlab-ci.yml and .pre-commit-conf.yaml. Though we do not expect you to update and run the pipelines yourself, any gear submitted to the Exchange will need Flywheel staff to ensure the most recent versions of the pipelines are referenced and that the pipelines pass before publishing.
    • What you need to do: Uncomment the gearcheck lines and pytest lines
  8. Update the README.md
    • As you know the gear better than anyone, share the knowledge by updating the definitions, including/removing input and configuration fields, and documenting what is expected as output.
    • Adding details here helps all of us run and maintain the gear properly.
  9. Add custom processing options in main.customize_bids_command or special parsing conditions in the parser module.
    • Look at other BIDS App gears for examples of how this and other methods may be extended or changed for a particular BIDS App.
    • As an example, parser.parse_input_files was extended considerably for the BIDS fMRIPrep gear
    • Much of your developer power will come from modifying or adding methods in main and parser to address specific intricacies of the BIDS App you are wrapping.
  10. Add other methods (e.g., post-processing steps) in the run.py file in the appropriate sequence.
    • The run.py file is heavily commented to improve clarity and, hopefully, decrease development time.
    • Adding extra methods in "utils" or in the main.py or parser.py files and importing them to run.py is the cleanest way change pieces of the workflow.
  11. Re-run fw-beta gear build to update the Docker image that you are using to test, (upload), and debug gear runs.

Resources

The BIDS App Template relies heavily on the BIDS App Toolkit, where many of the commonly needed methods across BIDS App gears are stored. Of chief importance is the BIDSAppContext object, which stores pertinent filepaths, settings, and BIDS command options for the gear to use.

If you would like to see other examples of BIDS App gears available in the Exchange, check out the BIDS App Gear repo

FAQs