Skip to content

Release Notes

1.10.0 [2026-03-11]

Enhancements:

  • Added support for project.* Jinja2 template variables in de-identification profiles, allowing project-level metadata (e.g., project.info.site_id) to be used as substitution values.

Fixes:

  • Fixed mapping_csv input type declaration from source code to tabular data in manifest.json.
  • Disabled DICOM reading and writing validation warnings via fw_file.dicom get_config() to reduce noise during gear execution.

Maintenance:

  • Added ruamel.yaml>=0.18 dependency.
  • Updated PYTHON_VERSION from 3.12.12 to 3.12.13.
  • Added CLAUDE.md with codebase guidance for Claude Code.
  • Integrated claude-code CI pipeline for automated release notes generation.
  • Removed unused .env file containing COVER=30.

Documentation:

  • Updated README.md environment setup instructions to reference uv instead of poetry.

1.9.1 [2025-12-09]

Fixes:

  • Fixed session filtering when session_level_profile is True and subject exists in CSV but not all sessions are mapped, ensuring unmapped sessions are correctly skipped during export

Maintenance:

  • Updated flywheel-sdk from ~=20.3 to >=20.4.0
  • Updated python from 3.12.11 to 3.12.12

1.9.0 [2025-08-07]

Enhancements:

  • Added capability for session-level deid templating when session_level_profile is True
  • Added session_level_profile config option to enable session-level deid profiles instead of subject-level
  • Renamed csv input from subject_csv to mapping_csv for clarity
  • Added match_by_id config option for backwards compatibility with workflows that depend on matching destination session or acquisition containers by info.export.origin_id
  • Adjusted Flywheel metadata lookup to skip subject/session if expected metadata is missing (aligned with CSV lookup behavior)

Fixes:

  • Added delete_reason config option for validated instance compatibility when overwrite_files is "Cleanup" or "Cleanup_force"
  • Fixed regex to allow multiple jinja variables on the same line

Documentation:

  • Updated README.md to reflect mapping_csv input rename and session-level customization workflow
  • Added documentation for session_level_profile, delete_reason, and match_by_id config options
  • Added example table showing session-level mapping CSV format with session.label column

1.8.6 [2025-07-17]

Maintenance:

  • Updated python from 3.12.10 to 3.12.11
  • Migrated from poetry to uv for dependency management

Documentation:

  • Added compatibility note for Flywheel instances on version 19.5.x or earlier

1.8.5 [2025-05-31]

Fixes:

  • Updated deid and migration-toolkit to fix bug in logic of pixel scrubbing rules evaluation

1.8.4 [2025-05-23]

Enhancements:

  • Added non-root user flywheel to run the gear container, improving security posture

Fixes:

  • Fixed file upload method to use fw_client.upload_file_to_container instead of passing file objects directly to uploader
  • Fixed template file handling to ensure writable copy is made to work directory before processing
  • Fixed working directory to be gear's work dir during export to enable migration-toolkit pixel actions writing

Maintenance:

  • Updated flywheel-sdk from 16.19.2 to ^20.3
  • Updated pytest-cov from ^3.0 to ^6.1
  • Updated pytest-mock from ^3.8 to ^3.14
  • Updated joblib from ^1.0 to ^1.5
  • Updated ipykernel from ^6.29.4 to ^6.29
  • Simplified Uploader class by delegating file upload method selection to SDK
  • Increased minimum test coverage requirement from 40 to 44
  • Improved type hints in container_export.py, file_exporter.py, and retry_utils.py
  • Refactored upload_file_with_retry to accept file path instead of FileSpec object
  • Removed obsolete signed URL upload implementation from Uploader class
  • Added comprehensive unit tests for file_exporter.py, run.py, and uploader.py

Documentation:

  • Fixed line length formatting in README encryption/decryption sections to comply with formatting standards

1.8.2 [2025-05-09]

Enhancements:

  • Added Docker build caching support via DOCKER_BUILD_CACHE variable
  • Added classification validation via VALIDATE_CLASSIFICATION configuration
  • Implemented multi-stage Docker build for optimized image layers

Fixes:

  • Fixed label sanitization for numeric strings and strings containing commas
  • Fixed duplicate file upload prevention logic in session export
  • Improved error messages for file upload conflicts to be more descriptive

Maintenance:

  • Increased test coverage requirement from 35% to 40%
  • Refactored Dockerfile to use multi-stage builds with base, build, and dev stages
  • Updated flywheel-migration from ^13.10.1 to ^13.10.2
  • Replaced pip with uv pip for faster dependency installation
  • Added requirements-dev.txt to .dockerignore allowlist
  • Renamed function quote_numeric_string() to sanitize_label() for clarity
  • Added empty tests/__init__.py file for proper test package structure
  • Applied code formatting fixes to comply with ruff linter rules
  • Reordered pre-commit hooks: ruff_format now runs before ruff
  • Added noqa comments to suppress PLR0913 (too many arguments) and PLR0912 (too many branches) warnings where appropriate
  • Simplified conditional logic in multiple functions for better readability

1.8.1 [2025-05-01]

Enhancements:

  • Included the original filename and parent container information in the error log when a filename conflict occurs during export

Maintenance:

  • Updated migration-toolkit to 13.10.1
  • Updated python from 3.12.9 to 3.12.10

Documentation:

  • Added information in the README about the filename profile

1.8.0 [2025-04-11]

Enhancements:

  • Added support for updating Flywheel metadata fields using subject_csv data via new flywheel configuration block in deid profiles
  • Files skipped due to no matching profile are now logged in warning messages and included in output CSV entries

Fixes:

  • Fixed handling of files without matching templates to properly record skipped status instead of silently ignoring them

Maintenance:

  • Updated flywheel-migration to ^13.10 with pixel extras
  • Updated python from 3.12.7 to 3.12.9

Documentation:

  • Added example configuration showing how to update subject.label and other Flywheel metadata fields using flywheel block in deid profiles

1.7.0 [2025-01-16]

Enhancements:

  • Added capability to pull values from Flywheel subject metadata for subject-specific deid profile templates using subject.* Jinja variables

Fixes:

  • Fixed typo in log message output

Maintenance:

  • Split parser functions from run.py into dedicated fw_gear_deid_export/parser.py module
  • Added fw_gear_deid_export/var_utils.py module for handling Jinja variable utilities
  • Refactored container_export.export_container() function signature to accept jinja_var_df and key_dict parameters directly
  • Refactored Jinja variable validation and dataframe creation logic
  • Added openssl system package to Dockerfile
  • Migrated from flywheel.GearContext to flywheel_gear_toolkit.GearToolkitContext
  • Updated flywheel-migration dependency to 13.9.1
  • Added comprehensive test coverage for parser functions in tests/test_parser.py
  • Added comprehensive test coverage for variable utilities in tests/test_var_utils.py
  • Updated existing tests to reflect refactored deid template validation logic

Documentation:

  • Added section on subject metadata as option for subject-level customization
  • Updated examples to demonstrate using subject.info metadata with deid profiles
  • Fixed markdown formatting for gear level section
  • Reorganized subject-level customization sections with proper heading hierarchy

1.6.1 [2024-10-21]

Maintenance:

  • Migrated CI configuration from flywheel-io/scientific-solutions/etc/sse-qa-ci to flywheel-io/tools/etc/qa-ci
  • Updated python from 3.11 to 3.12
  • Updated base Docker image from python:3.11-slim-bullseye to flywheel/python:3.12-alpine
  • Replaced system package installation (apt-get) with Alpine package manager (apk)
  • Updated flywheel-sdk from 16.16.2 to 16.19.2
  • Updated flywheel-migration dependency to ^13.9.0
  • Added flywheel-gear-toolkit dependency ^0.6.18
  • Refactored retry logic into dedicated utility module
  • Added smart retry handlers for file uploads and container creation operations
  • Added comprehensive test coverage for retry utilities and uploader
  • Updated pre-commit hooks configuration and linting rules
  • Updated Docker environment variables for python 3.12 and virtual environment
  • Reformatted README markdown syntax for consistency
  • Reformatted Jupyter notebook whitespace

Fixes:

  • Fixed gear failure when subject.date_of_birth is empty string or None

1.6.0 [2024-08-28]

Enhancements:

  • Upgraded python from 3.8 to 3.11
  • Updated flywheel-migration dependency to support encrypt and decrypt actions for DICOM and non-DICOM files
  • Added support for DICOM pixel data masking based on pre-defined coordinates
  • Added public_key, private_key, and secret_key configuration options for encryption/decryption workflows
  • Changed overwrite_files configuration from boolean to enum with options: Skip, Cleanup, Cleanup_force, and Replace
  • Added date_of_birth to subject metadata allow list
  • Enhanced README with comprehensive encryption/decryption usage instructions, workflow documentation, and tutorials
  • Updated container matching to use label instead of info.export.origin_id

Fixes:

  • Files that fail de-identification are now skipped and reported in the export CSV with job state marked as failed
  • Fixed subject label truncation when length exceeds 64 characters
  • Fixed file.info.header removal behavior - now removed by default unless include-info-header: true is set
  • Fixed timestamp field formatting to use correct datetime format %Y-%m-%d %H:%M:%S.%f
  • Fixed error handling for smart-copied files with improved deletion strategies

Maintenance:

  • Migrated from poetry export to pyproject_export in pre-commit hooks
  • Updated pre-commit hooks from black and isort to ruff and ruff_format
  • Added gearcheck hook to pre-commit configuration
  • Updated flywheel-sdk to 16.16.2
  • Removed pinned version for git package in Dockerfile
  • Switched from ruamel.yaml.Loader to YAML(typ="safe", pure=True) for safer YAML parsing
  • Updated Flywheel classification metadata in manifest
  • Renamed identity action to keep throughout profiles and documentation

Documentation:

  • Added comprehensive encryption/decryption workflow examples
  • Added blank export profile example with file.info.header preservation
  • Expanded README with detailed input/config descriptions, usage instructions, and development setup
  • Added tutorial notebook reference for batch subject-level processing
  • Improved documentation formatting and organization with table of contents

1.5.1 [2023-04-12]

Fixes:

  • Fixed file metadata update to conditionally include type parameter only when present, preventing errors when updating file modality without type information

1.5.0 [2023-03-20]

Maintenance:

  • Migrated CI/CD configuration to use SSE QA CI templates from flywheel-apps/utils/ci-templates
  • Updated .pre-commit-config.yaml to reference SSE QA CI hooks
  • Migrated from poetry to requirements.txt for Docker dependency management
  • Upgraded base Docker image from python:3.8-slim-buster to pinned python@sha256:a076... (python 3.8.16)
  • Added .dockerignore file to optimize Docker build context by excluding unnecessary files
  • Updated Docker entrypoint from poetry run python to direct python execution
  • Pinned git package version to 1:2.30.2-1+deb11u2 in Dockerfile
  • Updated repository URLs from flywheel-io/flywheel-apps/deid-export to flywheel-io/scientific-solutions/gears/deid-export
  • Added setuptools>=65.5.0 and MarkupSafe<2.1 dependencies
  • Removed requirements.txt from .gitignore to allow version control
  • Added environment variables to manifest: PATH, GPG_KEY, PYTHON_VERSION, PYTHON_PIP_VERSION, PYTHON_SETUPTOOLS_VERSION, PYTHON_GET_PIP_URL, PYTHON_GET_PIP_SHA256, PWD
  • Set PYTEST_COV_FAIL_UNDER to 35% and enabled PUBLISH_POETRY in CI variables
  • Removed extra blank lines throughout codebase for consistency

Documentation:

  • Added whitespace formatting improvements to README for better readability

1.3.0 [2022-01-18]

Enhancements:

  • Added support to export file's zip_member_count and file tags
  • Added .pre-commit-config.yaml with hooks for code formatting (isort, black), manifest validation, and pytest

Fixes:

  • Updated flywheel-migration reference to make de-identified UID DICOM compliant

Maintenance:

  • Migrated CI/CD configuration to use shared CI templates from flywheel-io/flywheel-apps/utils/ci-templates
  • Added --no-cache-dir flag to pip install in Dockerfile to reduce image size
  • Copied README.md to Docker image and run poetry install --no-dev twice to ensure proper installation
  • Updated test Dockerfile to include .pre-commit-config.yaml, .env, and .git directory
  • Installed pre-commit in test container
  • Added .env file with COVER=30 configuration
  • Removed .env from .gitignore to track environment configuration

1.2.3 [2021-07-29]

Enhancements:

  • Added mlset metadata field for subject containers with values Training, Test, and Validation

Fixes:

  • Fixed cohort field capitalization to use Control and Study instead of lowercase variants
  • Removed constraint on species field to allow any string value

Maintenance:

  • Added test coverage for metadata validation with correct capitalization

1.2.2 [2021-06-09]

Fixes:

  • Fixed error when processing sessions without a state column in the export dataframe
  • Fixed handling of empty session exports by returning the dataframe instead of None
  • Fixed parsing of CSV lookup data by using to_dict('records')[0] instead of deprecated squeeze("rows")

Maintenance:

  • Updated README URL reference in manifest.json from master to main branch
  • Removed support for deid-log configuration in deid profiles with warning message
  • Reordered YAML fields in deid-dicom.yml example to place regex and name before action fields

1.2.1 [2021-03-30]

Enhancements:

  • Added blank-export.yml template that preserves all original filenames and metadata

Fixes:

  • Fixed error when updating file type for files without modality when type is None

Maintenance:

  • Updated flywheel-migration dependency to zip_file_profile branch

1.2.0 [2021-03-18]

Enhancements:

  • Updated flywheel-sdk from ^14.5.0 to ^15.3

Fixes:

  • Fixed typo in log message from "two errors" to "errors" in run.py:131
  • Fixed file type not being set when modality is not present, working around FLYW-6472
  • Improved upload retry logic to properly close failed responses before retrying
  • Fixed incorrect handling of file metadata updates by extracting type and modality before processing other metadata

Maintenance:

  • Updated flywheel-migration dependency to commit b67680b2
  • Removed unused _upload_session instance variable from Uploader class
  • Refactored signed_url_upload to accept optional upload_session parameter instead of using instance variable
  • Removed verbose logging statement from export_container function
  • Migrated from get_safe_filename to sanitize_filename function
  • Added test coverage for pluralize and get_upload_ticket_suggested_headers functions
  • Added upload_session parameter to upload methods for improved testability

1.1.0 [2021-03-15]

Enhancements:

  • Added --debug flag to container_export.py script for enabling debug logging when running standalone
  • Added uploader.py module implementing signed URL uploads with automatic retry logic for improved reliability

Fixes:

  • Fixed YAML loading security issue by replacing yaml.load() with yaml.safe_load() in container_export.py
  • Fixed parent container lookups by implementing caching for project and subject containers and LRU caching for session and acquisition containers to improve performance
  • Updated deid_template.py to use logger.debug() instead of logger.warning() for informational messages about unused CSV columns
  • Added validation for PatientIdentityRemoved field in example de-identification profile

Maintenance:

  • Updated project repository references from fw-gear-deid-export to deid-export in manifest.json
  • Migrated from yaml to ruamel.yaml library for improved YAML handling
  • Applied black code formatter across all Python files for consistent code style
  • Reorganized imports across all modules to follow standard Python conventions (stdlib, third-party, local)
  • Added comprehensive test coverage for uploader.py module including retry logic and signed URL support
  • Updated example de-identification profile validation in tests

Documentation:

  • Updated README with corrected YAML formatting for zip profile configuration showing proper field ordering and validation options

1.0.0 [2020-12-23]

Enhancements:

  • Added profile-based de-identification and export functionality for Flywheel containers
  • Added support for de-identifying DICOM, JPG, PNG, TIFF, XML, JSON, CSV, TSV, and text files
  • Added support for flywheel-migration de-identification profiles
  • Added subject-specific configuration support via CSV mapping file
  • Added Flywheel metadata de-identification with container-level transformations
  • Added automated container hierarchy creation in destination project
  • Added file overwrite protection with configurable override option
  • Added export status tracking with CSV output reporting
  • Added Jupyter notebook tutorial for batch processing at subject level

Documentation:

  • Added comprehensive README with usage instructions and workflow examples
  • Added YAML profile examples for DICOM, non-DICOM, table/text, and MHD files
  • Added LICENSE file (MIT)