Flywheel Azure ML Integration

• Introduction

The Flywheel Azure ML integration (Flywheel AML) provides users a unified experience between Flywheel's end-to-end data management and curation platform and Microsoft Azure's Azure’s Artificial Intelligence (AI) and Machine Learning (ML) training/production capabilities. Data Scientists can seamlessly access the Azure ML solution to support clinical AI development and collaboration at Scale.

Features

• One-press Azure ML Workspace creation.  Automation provides Flywheel project users an Azure ML workspace with user access, compute resources, and data integration with the press of a button.​

• Full ML Ops capabilities of Azure ML. Offers data scientists all the capabilities of Azure ML. Once onboarded, data scientists can access the Azure ML workspace directly.​

• Streamlined and Flexible data connectivity.  Allows data scientists to pull Flywheel data into Azure ML or have Flywheel push data to Azure ML data storage.  Either way the data access security is managed by Flywheel’s permissions.​

• ML Activity and Cost Visibility. Project leaders can view activities and assets created in Azure ML at a summary level and have visibility to projects cost, all within Flywheel.​

Availability

Flywheel AML is in Limited Release for selected customers.

Using Azure ML

This user guide focuses on Flywheel's integration functionality with Microsoft Azure ML. Microsoft provides a documentation site: Azure Machine Learning documentation | Microsoft Learn that describes the usage of AML.

Permissions

There are two permissions that are required to use this feature:

Manage Workspaces - Under the Workspaces heading. Allows you to enable, create, and use the Azure ML workspace for a project. Manage Exports - Under the Data Transfer heading. Allows you to export Flywheel data to Azure ML storage.

Instruction Steps

Enabling the Azure ML Workspace

The Azure ML workspace is enabled in the Project settings under 'Workspaces'. Once enabled, project users with the correct permissions can start and access the workspace.

The Azure ML workspace can also be enabled on the Workspaces tab.

In either case, the user will require a role with the Manage Workspaces permission.

After sliding the toggle, or clicking enable, you will get a dialog to confirm this step. Disabling or deleting the workspace is not possible, within Flywheel, but can be handled within Azure ML via a support request.

On the Workspaces tab of the project you can see the progress being made to create the Azure ML workspace and resources needed for usage.

The following automates steps complete in about 3-5 minutes:

1. Creating Resources: Creating the Azure ML workspace itself in a Subscription tied to the Flywheel site, as well as the base resources required.
2. Syncing Users: Creating the users in the Azure Workspace based on the Flywheel project users having workspace access. Users will have the Azure ML role data scientist.
3. Workspace Ready: Making the workspace accessible, so that you can go into it before the remaining steps are complete.
4. Creating Compute: Creating compute instances for the Azure ML Notebook for each user.
5. Setup Transfer Storage: Setting up an Export storage destination for the Azure ML Workspace storage.

First time access to Azure ML Subscription

If this is the first time you are accessing the Azure ML Subscription, you will go through an onboarding process that will use the email contained in your Flywheel user profile to set up a user in the Azure ML subscription associated to the site. Setup requires the following steps:

1. Confirm your email in Flywheel.

   The email in your Flywheel profile will be receiving an email invitation to join the Azure ML subscription. This email will be used as your user account in Azure ML. Review the email and make any required edits.

2. Link your account to Azure ML.

   From your Flywheel Profile, click on the XYZ button. Alternatively, from the Workspaces tab, select the Link Account button to begin the request process.

   From Flywheel Profile:

   From Workspaces Tab:

   

   In either case you will see a dialog box that reminds you to check your email for an invitation and you can click Done at this point. If you happen to lose the invitation email, you can have another one resent by clicking Resend Invite.

   

3. Check your email for an invitation

   Within a few minutes you should receive an invitation email from invites@microsoft.com that looks like this:

   

   Click the Accept invitation button. To go through the new user enrollment. You may be asked to confirm the access and/or set up multifactor authentication (MFA). Be sure to retain your email address and password in order to log into Azure ML.

4. Complete

   You now have credentials for the Azure ML subscription that is associated with your Flywheel site.

Launching the Azure ML Workspace in a project

Once the workspace is enabled, you can click Launch Azure ML Workspaces from the Workspaces tab to access the workspace.

This will launch the Azure ML workspace associated with this project. You may be asked for authentication if you are not yet logged into Azure ML subscription. After you log in, you have the Azure ML role data scientist

You will notice the name of the workspace will match the group/project of the Flywheel Project.

Using Azure ML

Azure ML is a comprehensive tool for Machine Learning Operations and further instructions, tutorials, and examples can be found at the Microsoft site here, as well as other online sources.

Flywheel Workspaces tab

The Workspaces tab provides information about status and activities occurring in Azure ML. This allows the complete project team to collaborate around the Data Science work occurring in Azure ML.

Metrics provided: - Users Number of users in the Azure ML workspace. - Endpoints Deployed Number of Azure ML Batch Endpoints deployed. - Prior Month Cost Cost associated with the Azure ML workspace resources from last month. - Current Month Cost Month to date cost associated with the Azure ML workspace resources.

Azure ML Resource visibility

Models, Jobs, Experiments, and Compute Instances created in the workspace are listed and you can click to get a detailed view of the meta data kept with these resources in Azure ML. For Models and Jobs, the description can be edited using the markdown editor. Azure ML access is not required to view or edit this information.

Example Model details

Integration features in Azure ML workspace

Notebook Compute

When users in Flywheel are given permission to a particular Azure ML workspace an individual notebook compute server is provisioned for that user so that they can use it for ML development within Azure ML Notebooks. For more information on Azure ML Notebooks see Microsoft documentation here

Flywheel Data Access

Azure ML provides storage for ML training data alongside the compute resources. Flywheel's Export feature is used to snapshot Flywheel data and make it available to the Azure ML Workspace.

When an Azure ML Workspace is initially created, the Workspace Artifact Store is automatically registered in Flywheel as an external storage provider. This storage will be registered for Export, with the scope of the individual project, to limit data to the Flywheel project. The storage destination will be labelled by the Azure ML Resource ID of the workspace and the words workspace storage

For example: fw-66196643234e7098ae7df9ff workspacestorage

See the Flywheel documentation on external storage providers for more details: view-the-list-of-external-storage-providers for more information.

Within Azure ML the Workspace Artifact Store is reflected as a datastore called workspaceartifactstore.

Flywheel Data Export to Azure ML

Flywheel Export is used to snapshot and transfer data to the workspaceartifactstore of the workspace. For further information on using Export, see the Project Exports User Guide.

Once the export is complete, the data will be available in Azure ML datastore workspaceartifactstore in a directory called flywheel.

Additional compute requests or workspace actions

As data scientists use Azure ML workspaces they may request additional compute resources in the workspace. These should be directed to Flywheel Support to perform the Azure Subscription configurations. 

These may include:

• Compute clusters for training runs

• Inference endpoints

• Additional compute for Azure Notebooks (GPU support for example)

• Monitoring

• Budget monitoring alerts

• deprovisioning and deleting workspaces

Resources

• Azure ML site

• Azure Machine Learning documentation | Microsoft Learn

FAQs

Q: How is Azure ML Configured? A: A new Azure Subscription is created for the Flywheel site and the subscription is configured based on the requirements for the integration.

Q: How is the Azure ML Subscription managed? A: The integration manages IAM, creation of user compute, workspaces, and other assets. Flywheel support manages creation of shared compute within workspaces (compute clusters for training and inference endpoints).

Q: What’s the hierarchy of AML?

A: Each Flywheel project can have one AML workspace enabled. Flywheel project to AML workspace has a one to one relationship. Each AML workspace has its own Azure resource group and the resource groups all live in an Azure Subscription. Each Flywheel site has an Azure Subscription with a one to one relationship.