How to Run a Bulk Import
Instructions
If any sort of non-default mapping rules will be used, then the new (BETA) CLI will be needed to start the import.
The options for specifying user-defined mappings are extensive and described in more detail in the new (BETA) CLI documentation for the import run
command.
0. Prerequisite -- CLI Installation
If the new (BETA) CLI is not already installed, follow the new (BETA) CLI installation instructions to install it.
An API key will be needed to sign in to the new (BETA) CLI and run commands. This can be the exact same API key used for the Legacy CLI.
If a new API key is needed, follow the documentation for creating a user API key.
1. Prepare source data
The first step is to decide how what information will be used to determine how the source will be mapped to the Flywheel hierarchy.
Refer to the documentation on Mapping to the Flywheel Hierarchy for more details of the various options for prepare your source data for mapping to the Flywheel Hierarchy.
2. (Optional) Register external storage
Importing from 'My Computer'
This step can be skipped if you are importing data from your local computer.
If you are importing data from an external storage device, that storage device must first be registered as an External Storage within Flywheel before the import can be started. This provides Flywheel with the necessary information about how to locate and access the storage device.
Follow the documentation explaining how to create a new "External Storage" registration in Flywheel.
3. Start the import
-
Login using the new (BETA) CLI by running the following command:
-
The CLI will then prompt for the API key. Enter your API key when prompted.
-
Start the import using the following command:
Where:
<project>
is replaced with the Flywheel Hierarchy of the destination project (e.g.,fw://demo/Alzheimers
)- Make sure to wrap this value in double-quotation marks if it contains any spaces (e.g.,
"fw://flywheel/Brain Tumor Progression"
)
- Make sure to wrap this value in double-quotation marks if it contains any spaces (e.g.,
<storage>
is replaced with either:- The full path to where the source data can be found on your local machine (e.g.,
/Users/jdoe/Desktop/dataset1
), or - The ID of the External Storage containing the source data (e.g.,
682e21f7487fc46c72718887
).- If you do not know the External Storage ID, see the section on finding the ID of an External Storage.
- The full path to where the source data can be found on your local machine (e.g.,
-
Once the import begins, the progress of the import will be displayed in the CLI output.
Tip
If you are importing from an External Storage device, then you can exit out of this progress display by pressing Ctrl+c
or by simply closing the terminal window altogether. This has no effect on the export job itself -- the export will still continue to run.
However, if you are importing from your local computer, then the CLI process should not be terminated until all files are fully uploaded off of your computer.