Flywheel Azure ML Integration

Introduction

The Flywheel Azure ML integration (Flywheel AML) provides users a unified experience between Flywheel's end-to-end data management and curation platform and Microsoft Azure's Azure’s Artificial Intelligence (AI) and Machine Learning (ML) training/production capabilities. Data Scientists can seamlessly access the Azure ML solution to support clinical AI development and collaboration at Scale.

Features

• One-press Azure ML Workspace creation.  Automation provides Flywheel project users an Azure ML workspace with user access, compute resources, and data integration with the press of a button.​

• Full ML Ops capabilities of Azure ML. Offers data scientists all the capabilities of Azure ML. Once onboarded, data scientists can access the Azure ML workspace directly.​

• Streamlined and Flexible data connectivity.  Allows data scientists to pull Flywheel data into Azure ML or have Flywheel push data to Azure ML data storage.  Either way the data access security is managed by Flywheel’s permissions.​

• ML Activity and Cost Visibility. Project leaders can view activities and assets created in Azure ML at a summary level and have visibility to projects cost, all within Flywheel.​

Availability

Flywheel AML is in Limited Release for selected customers.

Learning to use Azure ML

Microsoft provides a documentation site: Azure Machine Learning documentation | Microsoft Learn that describes the usage of AML.

Microsoft provides training, for example:

• Designing and Implementing a Data Science Solution on Azure.
• Introduction to Azure Machine Learning
• AML Workshop for Azure Machine Learning

Third party providers also supply classes for AzureML, for example Microsoft Azure Machine Learning for Data Scientists is offered from the Classcentral.com site.

Flywheel Permissions

There are 5 permissions that control the functionality of this feature:

Under the Project Settings heading:

Manage Project Settings Allows a user to enable the Azure ML integration feature for a project.

Under the Azure Machine Learning heading:

Read View resources and resource details displayed in Flywheel for the activities in Azure ML. Modify Modify the resource descriptions for the resources displayed in Flywheel. Access Access the Azure Machine Learning Studio

Under the Data Transfer heading:

Manage Exports - Allows you to export Flywheel data to Azure ML storage.

Instruction Steps

First time access to Azure ML Subscription

If this is the first time you are accessing the Azure ML Subscription, you will go through an onboarding process that will use the email contained in your Flywheel user profile to set up a user in the Azure ML subscription associated to the site. Setup requires the following steps:

1. Confirm your email in Flywheel.

   The email in your Flywheel profile will be receiving an email invitation to join the Azure ML subscription. This email will be used as your user account in Azure ML. Review the email and make any required edits.

2. Link your account to Azure ML.

   From your Flywheel Profile, click on the Link Azure Account button. Alternatively, from the Workspaces tab, select the Link Account button to begin the request process.

   From Flywheel Profile:

   From Workspaces Tab:

   

   In either case you will see a dialog box that reminds you to check your email for an invitation and you can click Done at this point. If you happen to lose the invitation email, you can have another one resent by clicking Resend Invite.

   

3. Check your email for an invitation

   Within a few minutes you should receive an invitation email from invites@microsoft.com that looks like this:

   

   Click the Accept invitation button. To go through the new user enrollment. You may be asked to confirm the access and/or set up multifactor authentication (MFA). Be sure to retain your email address and password in order to log into Azure ML.

4. Complete

   You now have credentials for the Azure ML subscription that is associated with your Flywheel site.

Enabling the Azure ML Workspace

The Azure ML workspace is enabled in the Project settings under 'Workspaces'. Once enabled, project users with the Azure Machine Learning Access permission can start and access the workspace.

The Azure ML workspace can also be enabled on the Workspaces tab.

In either case, the user will require a role with the Manage Project Settings permission.

After sliding the toggle, or clicking enable, you will get a dialog to confirm this step. Disabling or deleting the workspace is not possible, within Flywheel, but can be handled within Azure ML via a support request.

On the Workspaces tab of the project you can see the progress being made to create the Azure ML workspace and resources needed for usage.

The following automates steps complete in about 3-5 minutes:

1. Creating Resources: Creating the Azure ML workspace itself in a Subscription tied to the Flywheel site, as well as the base resources required.
2. Syncing Users: Creating the users in the Azure Workspace based on the Flywheel project users having workspace access. Users will have the Azure ML role data scientist.
3. Workspace Ready: Making the workspace accessible, so that you can go into it before the remaining steps are complete.
4. Creating Compute: Creating compute instances for the Azure ML Notebook for each user.
5. Setup Transfer Storage: Setting up an Export storage destination for the Azure ML Workspace storage.

Launching the Azure ML Workspace in a project

Once the workspace is enabled, you can click Launch Azure ML Workspaces from the Workspaces tab to access the workspace. This will require the Azure Machine Learning Access permission.

This will launch the Azure ML workspace associated with this project. You may be asked for authentication if you are not yet logged into Azure ML subscription. After you log in, you have the Azure ML role data scientist

You will notice the name of the workspace will match the group/project of the Flywheel Project.

Using Azure ML

Azure ML is a comprehensive tool for Machine Learning Operations and further instructions, tutorials, and examples can be found at the Microsoft site here, as well as other online sources.

Flywheel Workspaces tab

The Workspaces tab provides information about status and activities occurring in Azure ML. This allows the complete project team to collaborate around the Data Science work occurring in Azure ML. Users with the Azure Machine Learning Integration Permissions of Read or Modify can view or make changes to the information.

Metrics provided: - Users Number of users in the Azure ML workspace. - Endpoints Deployed Number of Azure ML Batch Endpoints deployed. - Prior Month Cost Cost associated with the Azure ML workspace resources from last month. - Current Month Cost Month to date cost associated with the Azure ML workspace resources.

Azure ML Resource visibility

Models, Jobs, Experiments, and Compute Instances created in the workspace are listed and you can click to get a detailed view of the meta data kept with these resources in Azure ML. For Models and Jobs, the description can be edited using the markdown editor. Azure ML access is not required to view or edit this information.

Example Model details

Integration features in Azure ML workspace

Notebook Compute

When users in Flywheel are given permission to a particular Azure ML workspace an individual notebook compute server is provisioned for that user so that they can use it for ML development within Azure ML Notebooks. For more information on Azure ML Notebooks see Microsoft documentation here

Flywheel Data Access

Azure ML provides storage for ML training data alongside the compute resources. Flywheel's Export feature is used to snapshot Flywheel data and make it available to the Azure ML Workspace.

When an Azure ML Workspace is initially created, the Workspace Artifact Store is automatically registered in Flywheel as an external storage provider. This storage will be registered for Export, with the scope of the individual project, to limit data access to the Flywheel project. The storage destination will be labelled by the Azure ML Resource ID of the workspace and the words workspace storage

For example: fw-66196643234e7098ae7df9ff workspacestorage

See the Flywheel documentation on external storage providers for more details: view-the-list-of-external-storage-providers has more information on storage providers registered in the Flywheel Site.

Within Azure ML the Workspace Artifact Store is reflected as a datastore called workspaceartifactstore.

Flywheel Data Export to Azure ML

Flywheel Export is used to snapshot and transfer data to the workspaceartifactstore of the workspace. For further information on using Export, see the Project Exports User Guide.

Once the export is complete, the data will be available in Azure ML datastore workspaceartifactstore in a directory called flywheel.

Additional compute requests or workspace actions

As data scientists use Azure ML workspaces they may request additional compute resources in the workspace. These should be directed to Flywheel Support to perform the Azure Subscription configurations. 

These may include:

• Compute clusters for training runs

• Inference endpoints

• Additional compute for Azure Notebooks (GPU support for example)

• Monitoring

• Budget monitoring alerts

• deprovisioning and deleting workspaces

Resources

• Azure ML site

• Training Class: Designing and Implementing a Data Science Solution on Azure.

• Introduction to Azure Machine Learning

• AML Workshop for Azure Machine Learning

• Azure Machine Learning documentation | Microsoft Learn

• Azure WorkshopPLUS Training

FAQs

Q: How is Azure ML Configured? A: A new Azure Subscription is created for the Flywheel site and the subscription is configured based on the requirements for the integration.

Q: How is the Azure ML Subscription managed? A: The integration manages IAM, creation of user compute, workspaces, and other assets. Flywheel support manages creation of shared compute within workspaces (compute clusters for training and inference endpoints).

Q: What’s the hierarchy of AML?

A: Each Flywheel project can have one AML workspace enabled. Flywheel project to AML workspace has a one to one relationship. Each AML workspace has its own Azure resource group and the resource groups all live in an Azure Subscription. Each Flywheel site has an Azure Subscription with a one to one relationship.

Q: How do I remove user access from AzureML when I wish to remove a user from Flywheel? A: Before removing the user from Flywheel, you must remove the user from the project template of each group and select the option to remove from all projects.