Skip to content

De-identifying Data during Ingest

Introduction

Now that you have added a template, de-id profile, and any include/exclude settings to the config file, it's time to run the CLI command to upload your dataset to Flywheel.

The Flywheel CLI is an additional Flywheel program you download to use on your computer's Command Prompt (Windows) or Terminal (Linux, Mac) app. If you have not already, download and install the CLI before continuing with the steps below.

Instruction Steps

Upload Data from your Computer

  1. Open a text editor such as Sublime, TextEdit, or Notepad.
  2. Start with the following command:
fw ingest template -C MyConfig.yaml /path/to/my/Data --group [group.id] --project [project.label]

If you are using a de-id profile in your config file, use the --de-identify flag in your command.

  1. Replace the placeholders with information about your data and environment. For example:

Windows:

fw ingest template -C config.yaml C:\Users\ExampleUser\Desktop\flywheel\ImportData --group mygroup --project "Example Project" --de-identify

Mac and Linux:

fw ingest template -C config.yaml ~/Desktop/flywheel/ImportData --group mygroup --project "Example Project" --de-identify

You can find the group id and project label at the top of each project in the format fw://group.id/projectLabel

Projectpath-highlight.png

Tip: The Group ID is always one word. In the above image, you would use --group 612Lab.  Project labels with spaces require quotes around the label. For example, --project "Anxiety Study".

  1. Open the Terminal app (Linux, Mac) or Windows Command Prompt app.

  2. Copy and paste your command, and hit enter.

  3. Flywheel CLI displays the data it has found in the scan summary.

IngestTemplateHierarchySummary.png

  1. Review the hierarchy and scan summary to make sure it matches what you expect.
  2. Enter yes to begin importing. The Flywheel CLI displays its import progress.
  3. Once complete, sign in to Flywheel, and view your data.

Upload Data from an s3 Bucket

To upload data from an s3 bucket to Flywheel:

  1. Configure your AWS CLI credentials. The Flywheel CLI uses these credentials to access the data, so you must configure them before running the ingest template command. The Flywheel CLI does not support passing credential parameters to it.

See Amazon's documentation for more information on how to use the configure command to set up your credentials or learn more about creating a shared credentials file or using environmental variables to set up credentials.

  1. Start with the following command:
fw ingest template -C MyConfig.yaml s3://[bucket]/[optional-path-to-data] --group group.id --project "project label"
  1. Replace the placeholders with the relevant info for your data and environment.

If you are using the config file to de-id data, you must include the --de-identify flag.

Windows:

fw ingest template -C config.yaml s3://MyStudy/DataForUpload --group mygroup --project "Example Project" --de-identify

Mac and Linux:

fw ingest template s3://MyStudy/DataForUpload --group mygroup --project "Example Project" --de-identify

Tip: Project labels with spaces require quotes around the label. For example, "Example Project".

  1. Copy and paste your command into Terminal or Windows Command prompt and hit enter. Flywheel CLI displays the data it has found.

IngestTemplateHierarchySummary.png

  1. Review the hierarchy and scan summary to make sure it matches what you expect.

  2. Enter yes to begin importing. The Flywheel CLI displays its import progress.

  3. Once complete, sign in to Flywheel to view your data.