Smart Copy

Introduction

Flywheel recognizes the need to save storage costs and reuse data.  In order to prioritize this effort, we are introducing Smart Copy (released in version 16.5.0).  Smart Copy is a way to share and reuse data and analysis in a consistent project framework while minimizing storage use.  The feature will allow users to create a new project by duplicating the original’s metadata and reusing its files.  This new project can be shared with another set of users without having to duplicate and store those files.

ONE will see the following terminology used. Below are the definitions:

• Source or Original Projects are any projects in Flywheel that have been copied
• Copied or Dependent Projects are the projects that are copied and created from referencing files and copying metadata from the source project.

Instruction Steps

Special Notes

• Only the latest version of a file and only analyses where the inputs/outputs are the latest versions are eligible to be copied.
• Smart Copy cannot be used programmatically via the SDK or CLI.

Allowing Copies of your Project

As an administrator of a project, you are able to control whether or not you want your project to be able to be copied.  In order to provide the ability to copy your project, you will want to go to your Project level menu and select “Settings.”

Once in Project Settings, you will see a Project Sharing section.  Within this section, you will select “Allow Copies of Project” and hit the Save button.  

Upon adding a new project, the ability to copy projects be defaulted off in order to prevent unanticipated issues downstream.

You will get a notification that our project settings have been successfully saved as a green overlay in the upper right-hand corner.

This will provide other users a ‘Copy’ button within the individual project so other users can now copy your project.

Removing the Ability to Copy your Project

As an administrator on a project, you are more than welcome to remove the ability for other users to copy your project at any time.  To do so, we will reverse the steps above by going back to the project menu, and going back to “Settings.”

Next, you will change this by selecting “No Copies Allowed (Default)” and hitting the Save button.

This will remove the copy button so users can no longer copy your project.  

Please note that the projects that have been copied from your project (dependent projects) will not be affected by this change and will continue to function as expected, utilizing your files.

Viewing Dependent Projects

You can view the Dependent Projects (those that were copied from your project) by looking at the Project Settings where you have turned on the ability for others to copy your project.  Go to the Project menu (three dots) and select settings.

The Project Sharing settings will display.  Next to the Reset and Save buttons on the bottom right-hand side of the screen, there is a “View Dependent Projects” option.  You will see a number on the right-hand side that indicates how many copies of your project there are that rely on your files for reference.

Any projects that have been created by copying your Source Project, will be listed on the flyover that displays when you have clicked the “View Dependent Projects” button.  This Dependent Projects screen will display the name of the Copied Project, the group the project belongs to, as well as the project administrators who did the copy of your project.

There is no limit to the number of projects that can be copied from the Source Project.

If you hover over the avatar initials listed under administrator, it will provide you with the email address of each of the administrators of the dependent project which will allow you to reach out to them if you need anything.

For instance, if your organization is looking to delete projects you no longer use to save storage and costs, however, you’ve allowed other organizations to make copies of your projects, you can utilize the contact information to negotiate the transfer of the Source Project to a new owner.

Copying a Project

In order to copy a project, go into the project you want to copy from the Projects page on the left navigation. If the project administrator has given access, in the upper right-hand corner, you will see a copy button.  Select the copy button.

An overlay will populate allowing you to enter the information for your new project and select inclusion and exclusion criteria of what you’d like copied. Group (example: Flywheel group) and Project Names (example: Alzheimers_Atrophy_Analysis) are required.

The remainder of the settings listed are optional. If you don’t check any box, or list any include or exclude criteria, the entirety of the project (all metadata) will be copied.

If you check any of the checkbox options (exclude analyses, tags, or notes), it will exclude those actual items from the copy, however, you can still use those within the conditional include/exclude logic and it will filter by that type of data.  For example, if you include sessions with the tag ‘Passed_QC’, it will first pull all of the data for which a tag of ‘Passed_QC’ is present, but then will not include the actual tag of ‘Passed_QC’ or any other existing tag on the data.’ In your new project, you would have all data that originally had the ‘Passed_QC’ tag, but you will not have copied over the tag itself. This allows you to use the tagging feature to tag subjects (or other containers) as candidates to copy into a new dependent project, while not transferring the tags.

Type-ahead is available to the end-user during criteria selection but it is not required. If the user knows the text is in the project, you don't have to wait to select from the dropdown, it will still copy what you've typed in as long as it matches.

Note that the exclusion criteria will take priority over the inclusion criteria.

Once you’ve completed the criteria and you’ve hit the copy button, it will automatically bring you to the description page of the newly created, Copied Project that you've created.  

This will give the user the opportunity to update the Description page for the new project they've created (along with what else within the new project they'd like to update).

Within the new project, you will also notice a ‘linked’ icon on the respectively copied information.  If a user hovers over the ‘link’ icon, it will explain that the file is being referenced from another project.  This is how a user can be sure it's referencing the actual file in the Source Project (in this case Alzheimers project) and not owned by the Copied Project (in this case Alzheimers_Atrophy_Analysis project).

Within the Source Project (in our example case, the original source project is our Alzheimers project) settings screen, you will now see the newly copied project listed as a Dependent Project.

Source (Original) Project Behavior

Adding and Generating New Data to a Source Project

• When any new files are added or a new version is uploaded into the Source Project after a copy, they do not get updated in the Copied Project as they were not available at the time of the copy. When new metadata is added to the Copied Project, it does not impact the Source Project.
• When any new metadata is added or updated in the Source Project, it does not impact the Copied Project metadata once the copy has been completed.

Gears can be run on the Source Project with no impact on the Copied Projects.

Exporting and downloading the Source Project should continue with no impact on the Copied Projects.

Copied (Dependent) Project Behavior

Adding and Generating New Data to a Copied Project

• Adding and uploading new files to a Copied Project stores them locally to that project.  In this scenario, it is not referenced, therefore, the ‘referenced’ icon and tooltip do not display. This is the same if the user uploads a new version of a previously referenced file.
• Files copied from the Source Project will have their own version history with the first version linked to the version that was copied.

Removing and Deleting Data from a Copied Project

• When data is deleted or removed in the Copied Project, it does not impact the Source Project.

Gears can be run on the Copied Project with no impact on the Source Project.

Exporting and downloading the Copied Project should continue with no impact on the Source Projects.

Copying large projects will take more time.  In these instances, you can access the project while data is being populated.  You will see an indication that your project is being copied.

To read more about Smart Copy, visit our recent (April 7, 2022) blog post here.