Gear Building Tutorial Part 2e: Testing, Debugging and Uploading
Introduction
From the previous sections, you should have a gear directory with a run script, a manifest, a Dockerfile, and a message.txt file. You can find these files in this repository for reference. In this section, we will show you how to locally run your gear using the flywheel CLI, and some debugging practices.
Instruction Steps
Local Gear Testing
The Flywheel CLI provides you with a tool to test your gear locally without uploading to flywheel. This tool requires that your directory has a manifest and a run script in your gear directory. A gear can be run locally with the command:
Manifest Inputs
Viewing Input Options
As discussed in the previous section, the manifest is used to generate a UI in the flywheel web instance when using a gear. Since we're running locally, we must enter our inputs a different way. First, we can call:
This displays all the possible input and config options we set in the manifest. This is a good way to debug your manifest to make sure that all the correct options are present. For our gear, calling this command shows us the following output:
Run your gear from the current folder
Usage:
fw gear local [flags]
Flags:
-h, --help help for local
--message_file string the custom message we'll print after our Hello statements
--my_name string Your name, for the program to say 'hello' to
--num_rep int The number of times to say 'hello'
As we can see, aside from the --help option, which is native to the command fw gear local, and is present every time --help is called, even though it's not strictly present in the manifest, we can see that there are three settings we can modify:
- message_file - From the manifest, this is the custom message we wish to have printed after our hello call. Notice that this input is a string, and there's a small description. These are taken from the type and description keys in the manifest
- my_name - This is our name, which will be echoed hello to.
- num_rep - the number of times to say hello
This is how you can verify that you've properly set the correct number of inputs, as well as their type, and description. If anything here is wrong or missing, modify your manifest.
Setting Input Options
To set the input options, we call each of them like a bash tag after fw gear local, with their value preceded by an equals sign. String values with spaces need to be surrounded by quotes:
In the case of our gear, we need to set every input that's not optional. Since we're overachievers, we'll set them even if they are optional.
Navigate to your gear directory and open a new terminal window. Since all the files we need are in this directory, we can simply provide the name for inputs. Any files not present in this directory will need their full path. Let’s say we want to say hello 5 times to the name Homer Simpson, and then print message.txt, so we enter:
Debugging
Didn't work, did it? That's on purpose. Building gears and creating docker images can be tricky, so we're going to walk you through a couple debugging steps to get things up and running. You probably got some message like:
Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container
process caused "exec: \"bash\": executable file not found in $PATH": unknown
This is a Docker error, and chances are you'll get a few of them as you're starting out. Being able to recognize and solve problems like this will be critical for your gear development ability. Let's break this down:
It looks like the program is having some problems running "bash". Could it have something to do with the Alpine operating system we're using? From a Google search, we can see that one of the top results is actually a stack overflow post about using Apline and bash in a Docker image! Reading this post, we see that Alpine does not come with bash. We can look for an existing Docker image with alpine, python, and bash, which do exist, such as in this repository, with the alpine tags, however we're going to install it ourselves to give you an idea of how to work with Dockerfiles.
Augmenting The Dockerfile
According to the info we found on stack exchange, we need to install bash ourselves. The post gives us the specific command to use:
We can add this as a separate RUN command, but given our discussion about not adding unnecessary layers, it might be best to just wrap it up with our existing RUN command.
As a result, our Dockerfile will now look like this:
FROM alpine:latest
RUN apk add --no-cache bash \
python3 \
py-pip \
&& pip install flywheel-sdk \
&& rm -rf /var/cache/apk/*
ENV FLYWHEEL=/flywheel/v0
RUN mkdir -p ${FLYWHEEL}
COPY run.py ${FLYWHEEL}/run.py
ENTRYPOINT ["python3 run.py"]
Compiling an Updated Docker Image
At this point, we now have a choice to make. We can either call the same build command, and overwrite our old image:
or we can increment the version tag, and create a new image for legacy’ sake:
As mentioned in the previous section, if we increment our docker image, we'll also need to increment our manifest version.
Generally, if your gear is already published, and you need to update your docker image because you're making functional changes to your code, it's a good idea to increment the version tag so that the old image is present and can still be used to run the old code. However, since we're just trying to get our first version of the unpublished gear to run, It's fine to overwrite the initial, failed image to avoid docker image clutter.
Build the new image, and let's try to run our gear again:
Success! If not, download this repository and make sure your files are identical. Or, use the google-fu examples shown here and attempt to fix it yourself!
Congratulations, you just created your first gear! Now this gear is ready to upload to your flywheel instance!
Gear Uploading
To upload the gear to your flywheel instance, you must make sure you're logged in to that instance using your API key from that site. Follow these steps to find your API key and login to your desired flywheel instance. If you have flywheel access for multiple sites/instances, this step is extremely important, as flywheel will upload the gear to whichever instance you're currently logged into.
Once you're ready, open a terminal window in the GearTutorial directory and type the following:
You should see a series of messages on screen indicating the status of the upload. Gears with large docker images may take a long time to upload, however for our case this should be relatively quick.
That's it! You should now see the gear under installed gearson the left-hand navigation bar, as well as in the drop-down menu when choosing a gear to run. (NOTE: you will need to refresh the flywheel instance page if you had it open before gear upload).
Here are a few things to note about flywheel gears on your instance:
- You cannot overwrite a gear. Every new upload must have a different version, no matter how small the change. It's strongly recommended that versions follow a logical progression, though this isn't enforced by flywheel.
- You cannot delete gears once uploaded, for the sake of provenance. However, Flywheel support can hide old gears on your instance if you so choose, to avoid gear clutter. All logs and outputs of a hidden gear are still visible on your instance.
Running Your Gear
To run this gear on the flywheel instance, you'll need to upload a "message.txt" file to use as input, and set your name and the number of reps in the config tab. The output of this gear will show up in the gear "logs", which you can view in the provenance tab for any session, as well as in the "Analysis" tab, if the gear is an analysis gear. A detailed description on how to run your gear, view its status, logs, and examine gear outputs can be found in this article.
Resources
Navigation
Part 2a: The Flywheel Environment