Skip to content

How to Run a Bulk Import

Instructions

The easiest and quickest option to get the data into Flywheel is to use the new uploader in the Flywheel Core Web App to upload the data.

The new uploader will temporarily stage the uploaded data into a pre-configured cloud object storage location and then trigger an import from that location.

The option for uploading data from your local machine is not currently available via the new CLI.

Tip

Starting in version 21.5.0, you can select a managed import rule set when starting an import from the web app. For fully custom inline rules, use the New CLI (flyw).

1. Prepare source data

The first step is to decide how the source data will be organized into Flywheel's hierarchy.

Refer to the documentation on Bulk Import - Default Behavior for details about how Flywheel automatically detects and processes different file types.

2. Register external storage

Before an Import can be started, Flywheel must first be configured with information about where to find the source data and how to access it. This is done by creating a new "External Storage" within Flywheel for the bucket location where the source data is stored.

Follow the documentation explaining how to create a new "External Storage" registration in Flywheel.

3. Start the import

The option to start a new import is located on the Imports tab of the details page of the destination project:

  1. Open the Flywheel Core Web App.

  2. Select "Add Data" from the navigation panel on the left-hand side.

    Left Nav -> Add Data Button

  3. Select the Project to which you would like to import the data.

    Left Nav -> Add Data -> Select Project

  4. Select whether you would like to "Import from External Storage" or "Upload and Import from my computer".

    Project -> Add Data -> From External Storage

  5. Select "Next".

  6. If uploading from your computer, follow this step. Otherwise, skip to step 8.

    1. Upload the data directly from your local machine (either by dragging-and-dropping or using the folder browser).

      Project -> Add Data -> Upload

    2. Expand the "Uploading Files" section to monitor the progress of the upload and confirm all files are uploaded successfully.

      Project -> Add Data -> Upload -> Successful Files

    3. If any file fails to upload, you may either:

    4. Use the "Retry Upload" option to retry uploading of that individual file, or

    5. Use the "Cancel Upload" option to skip uploading of that individual file.

      Project -> Add Data -> Upload -> Failed Files

  7. If importing from an external storage, follow this step. Otherwise, skip to step 9.

    1. Select the External Storage containing the source data to be imported.

      Project -> Add Data -> External Storage -> Select Storage

      Tip

      If the desired storage location is not available, refer to the "Register external storage" step.

  8. (Optional) Select an import rule set to apply. If no rule set is selected, the default rule set for the project is used (if one has been configured). For more information, see Import and Export Rule Sets.

  9. Select "Start Import" (or "Complete Upload") to start importing the data into Flywheel.

Next Steps