Video Annotation - Overview

Introduction

Flywheel provides turn-key annotation capabilities for video data with its Video Annotation opt-in module. With this module, Flywheel deploys and maintains a private instance of the Computer Vision Annotation Tool (CVAT), and the components needed to integrate it with Flywheel. Video data managed by the Flywheel service can be ready to annotate within minutes with only a couple Project settings.

Features

• Simple Administration: All configuration is managed with Flywheel Project Settings. The CVAT services and integration points are fully deployed and maintained by Flywheel.
• Single Sign-On: No separate user account or login needed with CVAT.
• Automatic Data Coordination: Data is transferred automatically between Flywheel and CVAT.
• CVAT Annotation Tools: All the manual CVAT annotation tools and shapes are available.
• Video Formats: All the CVAT supported 2D video and still image file formats are supported.
• Validated: The integration capabilities between Flywheel and CVAT are fully validated by Flywheel.
• Audit Trail: Annotation files exported to Flywheel are tied to the user exporting them.
• Access Logging: The Flywheel Access Log records whenever users launch a file in CVAT.

Best Practice

• Use the available Project Settings configuration options to limit the availability of CVAT to only the projects and acquisitions that require it. Automated background data transfer, processing, and storage is consumed to make data ready to annotate. This can affect the performance of Core Flywheel processes, and increase cost.

Integration Notes

• When video data in Flywheel is deleted, the video and annotation data stored by the CVAT service is automatically pruned according to the Flywheel Site's "Cleanup Deleted" retention schedule.

• A given video file in Flywheel will only ever match to a single video file and annotations within CVAT. That link between Flywheel and CVAT cannot be modified.

  • Replacing a video file in Flywheel with a new version will reset the file's annotations.
  • New copies of a video file resulting from Smart Copy will start with no CVAT annotations.
  • Restoring a prior version of a video file in Flywheel will restore that file's CVAT annotations.

• When performing a Project Smart Copy, the CVAT related project settings will not be copied from the source Project.

Limitations

The video annotation module being offered with the 20.4.0 Flywheel release provides the core foundation for video annotation workflows on data managed by Flywheel. Some native Flywheel and CVAT features are not currently available with the Video Annotation Module.

The following Flywheel-native features are not available with CVAT:

• Reader Tasks

The following CVAT-native features are not available with the Flywheel integration:

• Personal Workspaces
• Standalone Projects/Tasks/Jobs
  • The Flywheel integration will automatically create and manage the CVAT Projects/Tasks/Jobs, such that data relationships between Flywheel and CVAT are maintained, and that Flywheel Project Roles govern access to video and annotation data in CVAT.
• Semi-automatic and Automatic Annotation tools
  • The integrated CVAT service does not include a interface for managing the models, services, and computing infrastructure required for automatic annotation by CVAT.
  • If the annotations for an existing dataset already exists, and is in a format supported by CVAT, the annotations can be uploaded and available for manipulation.

Other integration related capabilities not currently available:

• Fine Grained resource management: The ability to prune the video and annotation data from CVAT without deleting it from Flywheel.
• Automation:
  • Bulk and programmatic Export of annotations from CVAT (via Flywheel Gear, Jupyter, etc.)
  • Bulk and programmatic Import of annotations into CVAT (via Flywheel Gear, Jupyter, etc.)
    • Useful when compatible annotation data is already available, from outside FW, from a FW gear, or previously exported and Smart Copied.

Resources

• Video Annotation - Settings
• Video Annotation - Annotating
• Video Annotation - Persisting Annotations in Flywheel