In version (14.4.0), Flywheel introduced a new way to copy an entire Flywheel project to a new location. The ingest project command gives users the ability to copy project data in a scalable and reliable way. This allows for streamlined multi-site collaboration as well helping workflows where data is curated on a pre-production Flywheel site before being moved to a production site.

The Ingest project CLI command can be used to copy data:

From one Flywheel site to a Flywheel different site
From one group to a different group within the same Flywheel site

Keywords

Source: The Flywheel site or project that contains the data you want to copy

Destination: The Flywheel site or project where you want to copy the data to

Steps

If you have a cluster, or are using Flywheel V3, we recommend using the cluster for a faster ingest. See this section for steps.

Sign in to the Source. You must have create container and upload data permissions on the source project.
The Source is the Flywheel site or Project that contains the data you want to copy.
From the Source, copy your API key. You will need this later on. To find your API Key, go to your Profile and copy the API Key.
Sign in to the Destination . You must have download permissions on the destination project.
If you are copying a project to a new group from within the same Flywheel site, your Destination and Source are the same.
In the Destination, create the Group and Project where Flywheel will place the data.
Open Terminal or Windows Command Prompt, and sign in to the Destination with the CLI. Learn more about how to download and install the CLI.
To see where you are signed in, enter the fw status command. To login, copy and paste the command from your Flywheel Profile page.
From the Flywheel CLI enter the following command:
```
fw ingest project [optional commands] SRC FW_WALKER_API_KEY
```
- Learn more about the optional flags for this command below
- SRC: The routing string from the Source Group and Project. For example: fw://alphagroup/alphaProject
- FW_WALKER_API_KEY: The API Key from the Source Flywheel site. For example: flywheel.io:yiLr8PHLtpe33IdYJ5
For example:
```
fw ingest project -group betagroup -project betaProject fw://alphagroup/alphaProject flywheel.io:yiLr8PHLtpe33IdYJ5
```
If you do not specify the group and project by adding the --group and --project flags, Flywheel will look for a Group and Project that matches the Source.
Flywheel begins scanning the Source and displays the results in the CLI.
Review the summary, and enter yes.
Flywheel ingest project command will copy the Source data and metadata, but not:
- Job history/provenance
- Analyses
- Change log history
- Project permissions
- Gear Rules
- Session templates
- Data Views
Once the scan is complete, sign in to the Destination to view.
Existing containers are not modified in destination project, and any existing files are not copied over.

Considerations

Empty containers in the Source will not be copied to destination
The File origins in the Destination show that the data was uploaded by a user
Only one project can be migrated per CLI command
The source project is copied to destination. The source project is not modified or deleted and remains on the Source site.

Use a cluster for a faster ingest

We recommend using a cluster if you have one available. Flywheel sites on V3 have can utilize the same cluster that the Flywheel Site is using. Follow the steps below for more information.

Steps

Sign in to the Source as a Site Admin.

The Source is the Flywheel site or Project that contains the data you want to copy.
From the Source, copy your API key. You will need this later on. To find your API Key, go to your Profile and copy the API Key.
Sign in to the Destination as a Site Admin.

If you are copying a project to a new group from within the same Flywheel site, your Destination and Source are the same.
In the Destination, create the Group and Project where Flywheel will place the data.
Open Terminal or Windows Command Prompt, and sign in to the Destination with the CLI. Learn more about how to download and install the CLI.

To see where you are signed in, enter the fw status command. To login, copy and paste the command from your Flywheel Profile page.
From the Flywheel CLI enter the following command:
```
fw ingest project --cluster URL --follow SRC FW_WALKER_API_KEY
```
- --cluster CLUSTER URL: Use the --cluster flag along with the cluster URL. If you are using a Flywheel V3 site, the URL to your cluster is https://yourflywheelURL.com/ingest.
  
  For example: https://university.flywheel.com/ingest
- --follow: Follows the progress of the ingest
- SRC: The routing string from the Source Group and Project. For example: fw://alphagroup/alphaProject
- FW_WALKER_API_KEY: The API Key from the Source Flywheel site. For example: flywheel.io:333333333333IdYJ5
For example:
```
fw ingest project --cluster https://yourflywheelURL.com/ingest --follow --group beta_group --project beta_project fw://alphagroup/alphaProject flywheel.io:333333333333IdYJ5
```
If you do not specify the group and project by adding the --group and --project flags, Flywheel will look for a Group and Project that matches the Source.
Flywheel begins scanning the Source and displays the results in the CLI.
Review the summary, and enter yes.
Note

Flywheel ingest project command will copy the Source data and metadata, but not:
- Job history/provenance
- Analyses
- Change log history
- Project permissions
- Gear Rules
- Session templates
- Data Views
Once the scan is complete, sign in to the Destination to view.
Existing containers are not modified in destination project, and any existing files are not copied over.

Reuse this command

You can reuse the command you created in these steps if you need to copy new data from the source in the future. Flywheel will only copy over the difference in the projects.

Command reference

Argument	Help Text
Positional arguments
SRC	The path to the folder to import (e.g.: fw://group-name/project-name/)
FW_WALKER_API_KEY	Your API key for the Source Flywheel site (e.g.: flywheel.io:yiLr8PHLtpe33IdYJ5)
Optional arguments
-h, --help	show this help message and exit
--compression-level COMPRESSION_LEVEL	The compression level to use for packfiles -1 by default. 0 for store. A higher compression level number means more compression.
--copy-duplicates	Upload duplicates found using --detect-duplicates to a sidecar project instead of skipping them (default: False)
--de-identify	De-identify DICOM files (default: False)
--deid-profile NAME	Use the De-identify profile by name
--detect-duplicates	Identify duplicate data conflicts within source data and duplicates between source data and data in Flywheel. Duplicates are skipped and noted in audit log (default: False)
--detect-duplicates-project DETECT_DUPLICATES_PROJECT [DETECT_DUPLICATES_PROJECT ...]	Specify one or multiple project paths to use for detecting duplicates
--enable-project-files	Enable file uploads to project container (default: False)
--encodings ENCODINGS [ENCODINGS ...]	Set character encoding aliases. E.g. win_1251=cp1251
--exclude PATTERN [PATTERN ...]	Patterns of filenames to exclude
--exclude-dirs PATTERN [PATTERN ...]	Patterns of directories to exclude
-g ID, --group ID	The id of the destination group (default comes from SRC)
--ignore-unknown-tags	Ignore unknown dicom tags when parsing dicom files (default: False)
--include PATTERN [PATTERN ...]	Patterns of filenames to include
--include-dirs PATTERN [PATTERN ...]	Patterns of directories to include
--load-subjects PATH	Load subjects from the specified file
--no-audit-log	Skip uploading audit log to the target projects (default: False)
-p LABEL, --project LABEL	The label of the destination project (default comes from SRC)
--require-project	Proceed with the ingest process only if the resolved group and project exists (default: False)
--skip-existing	Skip import of existing files (default: False)
--symlink	Follow symbolic links that resolve to directories (default: False)
General
-y, --yes	Assume the answer is yes to all prompts (default: False)
--ca-certs CA_CERTS	The file to use for SSL Certificate Validation
-C PATH, --config-file PATH	Specify configuration options via config file
-d, --debug	Turn on debug logging (default: False)
--no-config	Do NOT load the default configuration file (default: False)
-q, --quiet	Squelch log messages to the console (default: False)
--timezone TIMEZONE	Set the effective local timezone for imports
-v, --verbose	Get more detailed output (default: False)
Reporter These config options are only available when using cluster mode with the --follow argument or when using a local worker
--save-audit-logs PATH	Save audit log to the specified path on the current machine
--save-deid-logs PATH	Save deid log to the specified path on the current machine
--save-subjects PATH	Save subjects to the specified file
Cluster These config options apply when using a cluster to ingest data.
--cluster CLUSTER URL	Recruits resources from the cluster your V3 Flywheel site is installed on. Use when uploading a large amount of data. Your cluster URL is the URL you use to sign in followed by /ingest. For example: https://university.flywheel.io/cluster
-f, --follow	Follow the progress of the ingest (default: False)
Worker These config options are only available when using local worker (--cluster is not defined)
--jobs JOBS	The number of concurrent jobs to run (e.g. scan jobs)
--max-tempfile MAX_TEMPFILE	The max in-memory tempfile size, in MB, or 0 to always use disk
--sleep-time SECONDS	Number of seconds to wait before trying to get a task

{[{category.name}]}

Copy a Project using Ingest Project

Keywords

Steps

Considerations

Use a cluster for a faster ingest

Steps

Note

Reuse this command

Command reference