Skip to content

fw-curation

Introduction

fw-curation is a Python package designed to simplify the process of running curation tasks on the Flywheel hierarchy. It provides a set of classes and methods that make bulk curation actions more efficient and streamlined, with tools for traversing the hierarchy, handling multiprocessing, and aggregating reports. This package is maintained by Flywheel and requires Python 3.10 or higher.

The library operates at a higher abstraction level than the Flywheel SDK. While the SDK provides generalized API interaction, fw-curation focuses specifically on data curation tasks for data managers and developers who need to perform bulk operations across containers.

Key Features

  • Configurable Hierarchy Traversal: Walk through the Flywheel hierarchy with customizable traversal patterns
  • Parallelizable Operations: Built-in support for multiprocessing to handle large-scale curation tasks efficiently
  • Curator Classes: Extensible base classes for implementing custom curation logic at different hierarchy levels
  • File-Specific Curation: Dedicated curator class for operations focused on File objects
  • Integrated Reporting: Reporter objects that attach to curators for easy tracking of visited containers
  • Reduced Boilerplate: Standardized framework that minimizes repetitive code in curation workflows

Installation

The package can be installed using pip:

1
pip install fw-curation

For optional Flywheel SDK integration:

1
pip install 'fw-curation[sdk]'

Basic Usage

Here's a simple example of using fw-curation to log acquisition names during hierarchy traversal:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
from fw_curation.curator import HierarchyCurator
from fw_curation.walker import Walker
import flywheel
import logging

log = logging.getLogger(__name__)

class Curator(HierarchyCurator):
    def curate_acquisition(self, acq):
        log.info(acq.label)

if __name__ == '__main__':
    fw = flywheel.Client()
    project = fw.lookup('example/test')
    walker = Walker.from_container(project)
    curator = Curator()
    curator.curate(walker)

Core Components

Walker

A configurable, parallelizable traversal tool for the Flywheel hierarchy that can be instantiated using Walker.from_container(). The Walker handles the complexity of navigating through projects, subjects, sessions, acquisitions, and files.

Curator Classes

  • Curator: Abstract base class exposing customizable methods for implementing curation logic
  • HierarchyCurator: Subclass with methods for each hierarchy level (e.g., curate_project, curate_subject, curate_session, curate_acquisition)
  • FileCurator: Subclass focused exclusively on File object handling

Reporters

Reporter objects attach to curators to provide easy reporting on visited containers during curation operations, helping track progress and results.

Resources