Smart Copy

Introduction

Flywheel recognizes the need to save storage costs and reuse data.  In order to prioritize this effort, we are introducing Smart Copy (released in version 16.5.0). Smart Copy is a way to share and reuse data and analysis in a consistent project framework while minimizing storage use. The feature allows users to create a new project by duplicating the original’s metadata and reusing its files. This new project can be shared with another set of users without having to duplicate and store those files.

You will see the following terminology used. Below are the definitions:

• Source or Original Projects are any projects in Flywheel that have been copied
• Copied or Dependent Projects are the projects that are copied and created from referencing files and copying metadata from the source project.

Instruction Steps

Allowing Copies of your Project

As an administrator of a project, you are able to control whether or not you want your project to be able to be copied. In order to provide the ability to copy your project, you will want to go to your Project level menu and select “Settings.”

Once in Project Settings, you will see a Project Sharing section.  Within this section, you will select “Allow Copies of Project” and hit the Save button.  

Upon adding a new project, the ability to copy projects be defaulted off in order to prevent unanticipated issues downstream.

You will get a notification that our project settings have been successfully saved as a green overlay in the upper right-hand corner.

This will provide other users a ‘Copy’ button within the individual project so other users can now copy your project.

Removing the Ability to Copy your Project

As an administrator on a project, you are more than welcome to remove the ability for other users to copy your project at any time. To do so, we will reverse the steps above by going back to the project menu, and going back to “Settings.”

Next, you will change this by selecting “No Copies Allowed (Default)” and hitting the Save button.

This will remove the copy button so users can no longer copy your project.  

Please note that the projects that have been copied from your project (dependent projects) will not be affected by this change and will continue to function as expected, utilizing your files.

Viewing Dependent Projects

You can view the Dependent Projects (those that were copied from your project) by looking at the Project Settings where you have turned on the ability for others to copy your project. Go to the Project menu (three dots) and select settings.

The Project Sharing settings will display.  Next to the Reset and Save buttons on the bottom right-hand side of the screen, there is a “View Dependent Projects” option. You will see a number on the right-hand side that indicates how many copies of your project there are that rely on your files for reference.

Any projects that have been created by copying your Source Project, will be listed on the flyover that displays when you have clicked the “View Dependent Projects” button. This Dependent Projects screen will display the name of the Copied Project, the group the project belongs to, as well as the project administrators who did the copy of your project.

There is no limit to the number of projects that can be copied from the Source Project.

If you hover over the avatar initials listed under administrator, it will provide you with the email address of each of the administrators of the dependent project which will allow you to reach out to them if you need anything.

For instance, if your organization is looking to delete projects you no longer use to save storage and costs, however, you’ve allowed other organizations to make copies of your projects, you can utilize the contact information to negotiate the transfer of the Source Project to a new owner.

Copying a Project

In order to copy a project, go into the project you want to copy from the Projects page on the left navigation. If the project administrator has given access, in the upper right-hand corner, you will see a copy button.  Select the copy button.

The Copy dialog opens, allowing you to enter the information for your new project and select the include and exclude criteria for what you’d like copied.

Destination

Under Destination, select the Group and enter the Project Name for the new project. Both fields are required.

Include and Exclude Filters

The remainder of the settings are optional. If you do not add any filters or change any of the checkbox options, the entire project (all metadata) is copied.

Two types of filters are available and can be combined:

• Include Filters: Files that match any of the conditions will be included in the copy. If no include filter is specified, all files are included unless they match an exclude filter.
• Exclude Filters: Files that match any of the conditions will not be included, regardless of the include filters.

Each filter consists of three parts:

• A Field Name (for example, acquisition.label, file.classification.Intent, or file.created)
• A matching operator
• A Value to compare against

Select Add Filter to add another condition, or the minus (−) icon to remove one.

Supported Fields

You can filter on standard hierarchy and file fields, including custom metadata under the info namespace and file classifications. The following fields are supported:

Matching Operators

Select the operator from the dropdown next to each field. The operators available depend on the data type of the selected field. Text fields, for example, do not support the greater-than and less-than comparisons, while date and numeric fields do.

Type-ahead is available during value selection but is not required. If you know the text is in the project, you do not have to wait to select from the dropdown; Smart Copy still copies what you have typed as long as it matches.

Metadata Options

The checkboxes below the filters control which metadata is copied for every container that is included. When checked, the associated metadata is copied:

• Analysis: Include all Analyses on every container (Project, Subject, Session, Acquisition) that is copied.
• Tags: Include all Tags on every container (Project, Subject, Session, Acquisition) that is copied.
• Annotations: Include all Annotations on every acquisition file that is copied.
• Notes: Include all User Notes on every container (Project, Subject, Session, Acquisition, Analysis) that is copied.
• Empty Containers: Include empty Subjects, Sessions, Acquisitions, and Analyses — ones that have no files matching the filters.

Analysis, Tags, Annotations, and Notes are included by default; Empty Containers is not. For annotations (measurement sets), only the most recent version of each annotation set is copied.

Once you’ve completed the criteria and you’ve hit the copy button, it will automatically bring you to the description page of the newly created, Copied Project that you've created.  

This will give the user the opportunity to update the Description page for the new project they've created (along with what else within the new project they'd like to update).

Within the new project, you will also notice a ‘linked’ icon on the respectively copied information. If a user hovers over the ‘link’ icon, it will explain that the file is being referenced from another project. This is how a user can be sure it's referencing the actual file in the Source Project (in this case Alzheimers project) and not owned by the Copied Project (in this case Alzheimers_Atrophy_Analysis project).

Within the Source Project (in our example case, the original source project is our Alzheimers project) settings screen, you will now see the newly copied project listed as a Dependent Project.

Source (Original) Project Behavior

Adding and Generating New Data to a Source Project

• When any new files are added or a new version is uploaded into the Source Project after a copy, they do not get updated in the Copied Project as they were not available at the time of the copy. When new metadata is added to the Copied Project, it does not impact the Source Project.
• When any new metadata is added or updated in the Source Project, it does not impact the Copied Project metadata once the copy has been completed.

Gears can be run on the Source Project with no impact on the Copied Projects.

Exporting and downloading the Source Project should continue with no impact on the Copied Projects.

Copied (Dependent) Project Behavior

Adding and Generating New Data to a Copied Project

• Adding and uploading new files to a Copied Project stores them locally to that project.  In this scenario, it is not referenced, therefore, the ‘referenced’ icon and tooltip do not display. This is the same if the user uploads a new version of a previously referenced file.
• Files copied from the Source Project will have their own version history with the first version linked to the version that was copied.

Removing and Deleting Data from a Copied Project

• When data is deleted or removed in the Copied Project, it does not impact the Source Project.

Gears can be run on the Copied Project with no impact on the Source Project.

Exporting and downloading the Copied Project should continue with no impact on the Source Projects.

Copying large projects will take more time.  In these instances, you can access the project while data is being populated.  You will see an indication that your project is being copied.

SDK

The Flywheel Python SDK supports these Smart Copy operations.

• Project Level project_copy() (identical to Flywheel Web UI)
• Subject Level subject_copy() (SDK only)
• Session Level session_copy() (SDK only)
• Acquisition Level acquisition_copy() (SDK only)