Skip to content

How to Run a Bulk Import

Instructions

The easiest and quickest option to get the data into Flywheel is to use the new uploader in the Flywheel Core Web App to upload the data.

The new uploader will temporarily stage the uploaded data into a pre-configured cloud object storage location and then trigger an import from that location.

The option for uploading data from your local machine is not currently available via the new CLI.

Tip

Only default mapping and filtering rules, are available when starting an import via the web app. If any sort of non-default mapping rules will be used, then the new (BETA) CLI will be needed to start the import.

Refer to the documentation on How to import data using the web app for more information.

1. Prepare source data

The first step is to decide how what information will be used to determine how the source will be mapped to the Flywheel hierarchy.

  • If your data consists of only DICOM files, the easiest option is to use the default DICOM header-based mapping rules and let Flywheel do the rest of the work. In this case,
    • Flywheel will derive the destination container labels from the DICOM headers,
    • Each source folder must contain exactly 1 DICOM Series, and
    • Each DICOM Series must be fully contained in exactly 1 source folder.
  • If your data consists of files other than DICOM, then you must must carefully organize your source dataset into a folder structure that matches the desired Flywheel Hierarchy. In this case,
    • Flywheel will derive the destination container labels from the source file paths (folder names).

Refer to the documentation on Mapping to the Flywheel Hierarchy for more details of the various options for prepare your source data for mapping to the Flywheel Hierarchy.

2. Register external storage

Before an Import can be started, Flywheel must first be configured with information about where to find the source data and how to access it. This is done by creating a new "External Storage" within Flywheel for the bucket location where the source data is stored.

Follow the documentation explaining how to create a new "External Storage" registration in Flywheel.

3. Start the import

The option to start a new import is located on the Imports tab of the details page of the destination project:

  1. Open the Flywheel Core Web App.
  2. Navigate to the destination Project to which the data should be imported.

  3. Navigate to the "Imports" tab of the destination Project (in the overflow menu).

    Project -> Imports Tab

  4. On the Imports tab, select the "Add Data" button to launch the "Add Data" dialog.

    Project -> Imports Tab -> Add Data Button

  5. Select whether you would like to "Import from External Storage" or "Upload and Import from my computer".

    Project -> Add Data -> From External Storage

  6. Select "Next".

  7. If uploading from your computer, follow this step. Otherwise, skip to step 8.

    1. Upload the data directly from your local machine (either by dragging-and-dropping or using the folder browser).

      Project -> Add Data -> Upload

    2. Expand the "Uploading Files" section to monitor the progress of the upload and confirm all files are uploaded successfully.

      Project -> Add Data -> Upload -> Successful Files

    3. If any file fails to upload, you may either:

    4. Use the "Retry Upload" option to retry uploading of that individual file, or

    5. Use the "Cancel Upload" option to skip uploading of that individual file.

      Project -> Add Data -> Upload -> Failed Files

  8. If importing from an external storage, follow this step. Otherwise, skip to step 9.

    1. Select the External Storage containing the source data to be imported.

      Project -> Add Data -> External Storage -> Select Storage

      Tip

      If the desired storage location is not available, refer to the "Register external storage" step.

  9. Choose your mapping rules, either "DICOM Headers" or "File Paths". Review the Mapping Rules documentation for more information.

    Applies to Flywheel version 19.1 or Earlier

    This choice of mapping rules is only available on sites running Flywheel version 19.1 or earlier where it was best to import DICOM and non-DICOM data separately. This option was removed in Flywheel version 19.2.

    With Flywheel 19.2, the default mapping rules have been combined so that both DICOM and non-DICOM data can be imported simultaneously, and the systems applies the appropriate mappings automatically based on the data type detected.

  10. Select "Start Import" (or "Complete Upload") to start importing the data into Flywheel.

Next Steps