The ingest folder command should be used if you are importing data that you previously downloaded from Flywheel, or if you are importing data that was shared from another Flywheel site. The folder import command uses the folder structure to map your data to the Flywheel hierarchy. Learn how to run the ingest folder command.
Below is the basic Flywheel hierarchy:
Beta feature
CLI Bulk Ingest is currently in beta. This feature is designed to improve performance and stability when importing large volumes of historical data. CLI Bulk Ingest requires an upgrade to our new V3 infrastructure, which we will begin deploying to sites starting in April 2020.
We do not recommend using Bulk Ingest until your site is upgraded to V3 because this infrastructure change is required to the improved upload performance. If your site is on V2 infrastructure, continue to use the import command to upload historical data. Contact your site admin to see which version your site is on.
|
Required argument |
Description |
|---|---|
|
SRC |
The parent folder for the directory you want to import |
Tip
If you are using multiple optional arguments in your command, consider creating a config file. See our article for more information on creating and using a config file.
|
Optional argument |
Description |
|---|---|
|
-h, --help |
show this help message and exit |
|
-g ID, --group ID |
The id of the group, if not in folder structure |
|
-p LABEL, --project LABE |
The label of the project, if not in folder structure |
|
--dicom NAME |
The name of dicom subfolders to be zipped prior to upload |
|
--pack-acquisitions TYPE |
Acquisition folders only contain acquisitions of TYPE and are zipped prior to upload |
|
--root-dirs ROOT_DIRS |
The number of directories to discard before matching |
|
--no-subjects |
no subject level (create a subject for every session) |
|
--no-sessions |
no session level (create a session for every subject) |
|
--group-override ID |
Force using this group id |
|
--project-override LABEL |
Force using this project label |
|
--include-dirs PATTERN |
Patterns of directories to include |
|
--exclude-dirs PATTERN |
Patterns of directories to exclude |
|
--include PATTERN |
Patterns of filenames to include |
|
--exclude PATTERN |
Patterns of filenames to exclude |
|
--compression-level COMPRESSION_LEVEL |
The compression level to use for packfiles -1 by default. 0 for store. A higher compression level number means more compression |
|
--ignore-unknown-tags |
Ignore unknown dicom tags when parsing dicom files |
|
--encodings ENCODINGS |
Set character encoding aliases. E.g. win_1251=cp1251 |
|
--de-identify |
De-identify DICOM files |
|
--deid-profile NAME |
Use the De-identify profile by name. Learn more about how to create a de-id profile. |
|
--skip-existing |
Skip import of existing files |
|
--no-audit-log |
Skip uploading audit log to the target projects |
|
--load-subjects PATH |
Load subjects from the specified file |
|
--detect-duplicates |
Identify duplicate data conflicts within source data and duplicates between source data and data in Flywheel. Duplicates are skipped and noted in audit log |
These config options are only available when using cluster mode with the --follow argument or when using local worker.
|
Optional argument |
Description |
|---|---|
|
--save-audit-logs PATH |
Save audit log to the specified path on the current machine |
|
--save-deid-logs PATH |
Save deid log to the specified path on the current machine |
|
--save-subjects PATH |
Save subjects to the specified file |
These config options apply when using a cluster to ingest data.
|
Optional argument |
Description |
|---|---|
|
--cluster CLUSTER |
Ingest cluster url (default: None) |
|
-f, --follow |
Follow the progress of the ingest (default: False) |
These config options are only available when using local worker (--cluster is not defined)
|
Optional argument |
Description |
|---|---|
|
--jobs JOBS |
The number of concurrent jobs to run (e.g. scan jobs), ignored when using cluster (default: 4) |
|
--sleep-time SECONDS |
Number of seconds to wait before trying to get a task (default: 1) |
|
--max-tempfile MAX_TEMPFILE |
The max in-memory tempfile size, in MB, or 0 to always use disk (default: 50)The max in-memory tempfile size, in MB, or 0 to always use disk (default: 50) |
|
Optional argument |
Description |
|---|---|
|
-C PATH, --config-file |
Specify configuration options via config file. Learn more about how to create this file. |
|
--no-config |
Do NOT load the default configuration file |
|
-y, --yes |
Assume the answer is yes to all prompts |
|
--ca-certs CA_CERTS |
Set the effective local timezone for imports |
|
--timezone TIMEZONE |
Set the effective local timezone for imports |
|
-q, --quiet |
Squelch log messages to the console |
|
-d, --debug |
Turn on debug logging |
|
-v, --verbose |
Get more detailed output |