Skip to content

How to Run a Bulk Import

Instructions

If any sort of non-default mapping rules will be used, then the new (BETA) CLI will be needed to start the import.

The options for specifying user-defined mappings are extensive and described in more detail in the new (BETA) CLI documentation for the import run command.

0. Prerequisite -- CLI Installation

If the new (BETA) CLI is not already installed, follow the new (BETA) CLI installation instructions to install it.

An API key will be needed to sign in to the new (BETA) CLI and run commands. This can be the exact same API key used for the Legacy CLI.

If a new API key is needed, follow the documentation for creating a user API key.

1. Prepare source data

The first step is to decide how what information will be used to determine how the source will be mapped to the Flywheel hierarchy.

Refer to the documentation on Mapping to the Flywheel Hierarchy for more details of the various options for prepare your source data for mapping to the Flywheel Hierarchy.

2. (Optional) Register external storage

Importing from 'My Computer'

This step can be skipped if you are importing data from your local computer.

If you are importing data from an external storage device, that storage device must first be registered as an External Storage within Flywheel before the import can be started. This provides Flywheel with the necessary information about how to locate and access the storage device.

Follow the documentation explaining how to create a new "External Storage" registration in Flywheel.

3. Start the import

  1. Login using the new (BETA) CLI by running the following command:

    1
    fw-beta login

  2. The CLI will then prompt for the API key. Enter your API key when prompted.

  3. Start the import using the following command:

    1
    fw-beta import run -p <project> -s <storage>

    Where:

    • <project> is replaced with the Flywheel Hierarchy of the destination project (e.g., fw://demo/Alzheimers)
      • Make sure to wrap this value in double-quotation marks if it contains any spaces (e.g., "fw://flywheel/Brain Tumor Progression")
    • <storage> is replaced with either:
      • The full path to where the source data can be found on your local machine (e.g., /Users/jdoe/Desktop/dataset1), or
      • The ID of the External Storage containing the source data (e.g., 682e21f7487fc46c72718887).

  4. Once the import begins, the progress of the import will be displayed in the CLI output.

Tip

If you are importing from an External Storage device, then you can exit out of this progress display by pressing Ctrl+c or by simply closing the terminal window altogether. This has no effect on the export job itself -- the export will still continue to run.

However, if you are importing from your local computer, then the CLI process should not be terminated until all files are fully uploaded off of your computer.

Next Steps

References

  1. fw-beta admin storage
  2. fw-beta import run