Flywheel 18.2

April 2024

18.2.0

EARLY ACCESS

Flywheel + Azure Machine Learning (Limited Release)

Benefits

• A unified experience between our end-to-end data management and curation platform and Azure’s model training/production capabilities.
• Data Scientists can seamlessly access the Azure ML Ops solution to support clinical AI development and collaboration at Scale.

Features

• One-press Azure ML Workspace creation. Automation provides Flywheel project users an Azure ML workspace with user access, compute resources, and data integration with the press of a button.​
• Full ML Ops capabilities of Azure ML. Offers data scientists all the capabilities of Azure ML. Once onboarded, data scientists can access the Azure ML workspace directly.​
• Streamlined and Flexible data connectivity. Allows data scientists to pull Flywheel data into Azure ML or have Flywheel push data to Azure ML data storage. Either way the data access security is managed by Flywheel’s permissions.​
• ML Activity and Cost Visibility. Project leaders can view activities and assets created in Azure ML at a summary level and have visibility to projects cost, all within Flywheel.​
• Build Inference Gears. Gear developers can incorporate the Azure ML inference code in a gear, or leverage calls to Azure ML inference endpoints in their gears, allowing users to validate ML models on Flywheel projects and data.

DICOM Uploader with Large File Support (BETA)

A new web-based DICOM uploader supporting large files is available as a BETA.

To use the new uploader, navigate to the "Imports" tab in the overflow menu of the "Project" page, select the "Import Data" button, and choose the "Local Upload" option.

The new uploader has no fixed limit on the size of each individual file or the total volume of data that can be uploaded. However, each individual file must complete uploading within 24 hours.

Only DICOM files are supported in the new uploader at this time. Additional file types are planned to be supported in a later release.

After the files are uploaded, a new Project Import job is started, and the processing of the files can be monitored via the import job details.

The uploaded DICOM files will be placed into the Flywheel hierarchy based on the DICOM header information using system-defined mapping rules.

FEATURES

External Storages (Imports & Exports)

• Supplemental Groups for File-System Storages
  • If more than one group membership is necessary to grant sufficient permissions to read and write files to a file system storage, the additional groups may be specified as supplemental groups.
  • Supplemental groups may be specified as a comma-separated list of group IDs (GIDs) when configuring External Storages of type File System.

FIXES

Large File Support

• Resolved an issue with multi-part uploads being committed out of order in Azure.
• Resolved an issue where SDK and CLI fails to upload large files due to an SSL error.
• Resolved an issue where signed URLs could not be used outside of AWS, Azure, or GCP.

fw CLI

• Fixed an issue with the CLI where the fw download returned an invalid error message saying too many arguments were provided.
• Fixed an issue with fw ingest dicom when --copy-duplicates is enabled where the process would sometimes fail with an error saying, "User already has permission on the project."
• Fixed an issue where fw sync cannot be used if the --export-templates-file option is specified.

Project Imports, Exports, & Storages

• Fixed an issue where Project Imports started via the Flywheel Core Web App were defaulting to allow file conflicts to update existing data instead of rejecting the conflicts (and flagging for manual review).
• Fixed an issue with Project Imports where if more than one file is being imported to the same destination path simultaneously, then none of the affected files are visible in Flywheel Core even though the import audit report shows they were successfully imported.
• Fixed an issue where External Storages not assigned to the current Project, Group, or Site are available for selection when starting a new Project Import from the Flywheel Core Web App.
• Fixed an issue where retrying a Project Import job would sometimes cause the job to disappear entirely.
• Fixed an issue with Project Exports where the audit report is not available after an export job is cancelled.
• Fixed an issue where External Storages not enabled for Imports are available for selection when starting a new Project Import from the Flywheel Core Web App.

18.2.1

FIXES

• Resolved issues of viewer system slowness and lack of responsiveness.
• Resolved issue with viewing NIfTI files in the default viewer.
• Fixed issue with Flywheel Notebooks where the user's notebook files are not being populated into the server.

18.2.2

FIXES

• Fixed an issue with fw ingest dicom when --copy-duplicates is enabled where the process would sometimes fail with an error saying, "User already has permission on the project".

18.2.3

FIXES

• Ingest Cluster (fw ingest ... --cluster)
  • Fixed an issue where the Ingest Cluster service could freeze if multiple very large Ingest jobs are launched in parallel (>8 million concurrent Acquisition upload tasks).
  • Fixed an issue where Ingest job become stuck and never fully complete if certain processes are shut down while a related task is being processed.
  • Fixed an issue where operational metrics collection caused a heavy load on the Ingest database.
• Flywheel CLI
  • Fixed an issue where the Flywheel CLI would retry HTTP POST and DELETE requests in response to network errors, resulting in actions being duplicated on the server-side.
• Reader Tasks
  • Resolved an issue preventing users from cloning Read Tasks even if they have Administrator permissions.
• Viewer
  • Resolved issue displaying existing annotations on PNG, TIFF, and other non-DICOM images in specific cases.
• Gears & Compute
  • Resolved issue on Azure cloud instances where only the first output file of a gear would be stored in Flywheel instead of all output files.

18.2.4

FIXES

Ingest (fw ingest)

• Fixed an issue with fw ingest where data would be uploaded even without being de-identified if an invalid de-ID profile was specified via the --deid-profile option, resulting in a potential PHI risk.
  • Now, an error will returned and the data will not be uploaded if the specified de-ID profile does not exist.
• Fixed an issue with fw ingest where the entire Ingest job would fail if the source data included DICOMDIR files.
  • Now, the DICOMDIR files will be skipped and the Ingest job will proceed normally.
• Fixed a rare issue with fw ingest when the --detect-duplicates and --copy-duplicates options are enabled where the sidecar project could not be created if another sidecar project already existed with the same name.
  • This issue is exceedingly rare since the sidecar project name is derived from the current timestamp of when the sidecar project is created, so it would only occur if multiple ingest jobs targeting the same project reached the same step at the same time (<1 second apart).
  • Now, when this condition occurs, a suffix will be added to the new sidecar project name so that it can be created successfully without conflict.

Ingest Cluster (fw ingest --cluster)

• Fixed an issue with fw ingest --cluster introduced in version 18.2.3 which prevented the cluster Ingest system from functioning if very old Ingest jobs launched on a version prior to 17.8 (which used an older database schema) had not completed before the system was updated to version 18.2.3.
  • The issue of old, stuck Ingest jobs was already identified and fixed in the 18.2.3 patch to prevent such cases from reoccurring in the future. The 18.2.4 patch contains additional code fixes to handle the schema changes more gracefully.

18.2.5

FIXES

• Fixed an issue preventing exports of large projects from completing due to a timeout.
• Fixed an issue that was causing large Smart Copies to fail because of worker processes running out of memory.
• Fixed an issue that was preventing access to job data when the parent of the job was moved to a new group and the old group was deleted.