Gear Building Tutorial Part 2a: The Flywheel Environment
Introduction
This document provides information on the Flywheel environment.
Instruction Steps
The Flywheel Environment
When a gear is run in flywheel, the environment is set up in a very specific way that is the same for each gear. This allows us to make generalizations in our code that will always be true. Since these settings are expected, deviating from them can result in your gear failing. We'll discuss the main flywheel environment components that you'll need to know to develop your gear.
The Flywheel Directory
Flywheel handles all of its processing in a specific directory inside the docker image. This directory is ALWAYS
The flywheel directory will contain the following files and folders, along with any additional files you may have copied in when creating your docker image:
The work directory is where you should do all your gear processing - any intermediate files and processing results should initially go here. You can create any structure of subdirectories in here that you wish. After the processing is complete, you can move any files that you want to save as outputs from the work directory to the outputdirectory. Upon completion of gear execution, ONLY the files in the output directory will get saved back to flywheel. More on that below.
inputand outputare directories where files are stored as inputs or outputs to the gear. config.json is a file generated for your gear at startup. It has all the configuration settings and filenames of any inputs you set when you ran the gear. The config options are input options available to you and are determined by your manifest. For every setting you declare in your manifest, a value will appear in the config file.
Inputs
The "input" directory creates a folder for every input you specify for the gear (specified in the manifest, which we'll cover in the next session). For example, if you have three inputs: a NIfTI image, a text file, and a json file, you would specify to flywheel what you wanted to name each input. The "name" of the input is just what flywheel refers to the input file as. For example, you could name the NIfTI image "nifti_in", the text file "slice_timing", and the json file "subject_info". Flywheel will then create subdirectories in the "input" folder, one for each input, with the name you assigned. In this case, Flywheel will generate three subdirectories:
/flywheel/v0/
└── input/
├── nifti-in/
│ └── whatever_file_you_give.nii
├── slice_timing/
│ └── whatever_file_you_give.txt
└── subject_info/
└── whatever_file_you_give.json
This way, you always know exactly where to find your input files, as long as you know what you name them.
Output
When a docker container finishes running, it terminates itself, and in doing so, deletes all information and files inside. That doesn't sound very desirable for us, since we'll want the output from our code! Fortunately, flywheel has a way around this. Flywheel also creates an "output" directory in the flywheel directory. At startup, this means that you'll always have the following directory setup:
Any file that is saved in the "output" directory is then available in your flywheel instance, either listed as a file with the acquisitions if it's a utility gear, or in a separate "analysis" container if it's an analysis gear, which you can find in the "Analyses" tab in your Flywheel UI. There is no limit to the number of files you can place in the output directory.
Note that only files will be exported. Flywheel does not support the output of folders. If you would like your output to be stored in a folder, with an internal directory structure, then the resulting folder must be turned into a file, and placed in the "output" directory. The most obvious way to do this is to compress any folder you want to have as output (zip, tar, gzip, any kind will work).
Config Settings
Sometimes, you need to set values that aren't technically an input file. For example, maybe you're performing spatial smoothing, and you want to specify the kernel size. This is called a "config" setting, and you can also specify them in the manifest. They appear in the user interface as text or check boxes that you can fill out when you run the gear. These options are accessible through a python "config" dictionary. You name each config setting (e.g. "kernel size"), and then in your code, you access the input value through that key in the dictionary. (config["kernel size"] would have the value you input). This is discussed in more detail in later sections.
Environment
Unfortunately, environment variables set in the Dockerfile won't carry over to the gear environment when run on flywheel. For most basic gears, this isn't a problem, but for more advanced gears (for example, if you call FSL), these environment variables are important. We'll learn about how to re-initialize our environments in a later section.
Navigation
Part 2a: The Flywheel Environment