Skip to content

Webinar: BIDS Templating

Introduction

Learn more about the BIDS standard, how to upload your data in to Flywheel using BIDS, and an example of how to extend the default BIDS specification. Follow along with the webinar using the interactive Jupyter notebook.

Examples to Follow in the Webinar

  1. BIDS 101
  2. Flywheel BIDS Overview
  3. Demo: Flywheel BIDS Curation Overview in the UI
  4. BIDS Curation Gear
  5. Project Curation Template Components
  6. Demo: Extend the BIDS template JSON file

Instruction Steps

1. BIDS 101

Brain Imaging Data Structure ( BIDS):

A filename convention and directory layout for organizing and describing neuroimaging datasets.

⏱ PRO TIP ⏱: Download a PDF of the latest BIDS specification supported by Flywheel and take an hour to read through, paying particular attention to the Common Principles section.

Example of a BIDS formatted dataset file tree

BIDSformatexample.png

BIDS Building Blocks

type - a functional group of different types of data.

  • func, dwi, fmap, anat, meg, eeg, ieeg, beh

<index> - a numeric value, possibly prefixed with arbitrary number of 0s for consistent indentation

  • e.g., 01 in run-01 following run-<index> specification

<label> - an alphanumeric value

  • e.g., rest in task-rest following task-<label> specification

suffix - an alphanumeric value, located after the key-value pairs and before the file extension

extension - a portion of the filename after the left-most period (.) preceded by any other alphanumeric

  • e.g., .nii.gz

Filename Structure

The BIDS filename convention consists of a chain of entities or key-value pairs, a suffix and an extension.

One entitysub-<label> corresponds to the subject entity because it has the sub- " key" and <label> "value", where <label> would in a real data file correspond to a unique identifier of that subject, such as 42. For example, sub-42.

Anatomy imaging data

sub-<label>[_ses-<label>][_acq-<label>][_ce-<label>][_rec-<label>][_run-<index>]_<modality_label>.nii[.gz]

entitiessub-<label>, ses-<label>, acq-<label>, etc.

  • Brackets [ ] indicate the entity is optional

type - anat

suffix - <modality_label>

  • Options: T1w, T2w, T1rho, T1map, T2map, T2star, FLAIR, FLASH, PD, PDmap, PDT2, inplaneT1, inplaceT2, angio

extension - .nii.gz

  1. Fill in the above, along with <index> and <label> for each entity to get the filename:

sub-<label> -> sub-42

acq-<label> -> acq-mprage

<modality_label> -> T1w

.nii[.gz] -> .nii.gz

  1. Stitch together with _ (underscores)

sub-<label>_acq-<label>_<modality_label>.nii[.gz] -> sub-42_acq-mprage_T1w.nii.gz

OHBM 2020: Automating BIDS Workflow with Flywheel

  • Talk on BIDS support in Flywheel, including import, curate, export, & analyze

BIDS Overview

  • Documentation introducing BIDS in Flywheel

Curation is the process of producing metadata on an existing Flywheel Project so the project can then be utilized as a BIDS formatted dataset. BIDS metadata is stored in the open-ended info.BIDS attribute that is associated on every Project, Session, Acquisition, Analysis, and File Container in the Flywheel system.

2. Flywheel BIDS Overview

  • Curation:

  • The process of producing BIDS metadata that maps your Flywheel Project to the BIDS standard

  • Metadata:

  • Stored in the info.BIDS attribute on each Container

  • Template: A JSON file used during curation to populate metadata

FlywheelBIDSfigure

3. Demo: Flywheel BIDS Curation Overview in the UI

Flywheel BIDS
Flywheeldata BIDSFlywheelData

4. BIDS Curation Gear

curate-bids repository

  • Input:

  • Project Curation Template

  • Process:

  • Producing BIDS metadata that maps your Flywheel Project to the BIDS standard. I.e., given a Flywheel Container, fill in and stitch together the BIDS building blocks.

  • Output:info.BIDS.<METADATA>

Flywheel BIDS Curation Overview

BIDS curation in Flywheel is built on a JSON template that dictates a metadata mapping. The JSON template provides flexibility for adapting to the continuously evolving standards of BIDS; however, this process involves project-specific modifications in order to ensure the process runs smoothly. The Flywheel implementation of BIDS empowers users to achieve BIDS compliant datasets; however, with the flexibility and constantly changing specifications, there will inevitably be a setup cost. With Flywheel, this setup cost is the construction of a Project Curation Template.

Project Curation Templates Overview

There is one default template maintained in the bids-client repo, which is called by the curate-bids Gear if no project-template.json is provided as an attachment on the project. Any project can have a project-template.json that provides the specific metadata mapping needed for BIDS curation on that project. This project-template.json may be reformulated in its entirety or extend the default template. In any of these cases, when referring to the BIDS curation process, these templates are referred to as the Project Curation JSON Templates.

Definition

Project Curation Template: the master template defining the BIDS standard for the project and used by the curate-bids Gear for BIDS metadata mapping.

⏱ PRO TIP ⏱: Some knowledge of JSON Schemas and regular expressions will be useful.

Default Project Curation Template

bids-client repository

  • bids-v1.json is stored in the bids-client repository which contains the core functionality of all Flywheel BIDS (i.e., import, curation, export)

  • curate-bids Gear will default to this template if:

  • A custom template is not provided as project attachment

  • The custom template is not JSON Schema compliant

Custom Project Curation Templates

bids-templates repository

  • Attached at the project-level
  • Name of file must end in project-template.json

e.g., custom-project-template.json

  • Automating BIDS Curation achieved via ReproIn Project Curation Template

  • ReproIn is part of the ReproNim Center suite of tools and frameworks

  • ReproIn repository -> A setup for automatic generation of shareable, version-controlled BIDS datasets from MR scanners

  • ReproNim -> A center for reproducible neuroimaging computation

  • Extend bids-v1.json - the default Project Curation Template

  • Modify only components of the Template that are relevant for your project

⏱ PRO TIP ⏱: Check the log for curate-bids Gear to make sure your Custom Project Curation Template was selected.

Default Template Custom Metadata
DefaultLogs Custommetadatalogs

5. Project Curation Template Components

Within the Project Curation Template are sections, most commonly, definitions and rules, along with other, more optional, sections (e.g., initializers, resolvers, etc.).

In [1]:

import json
import uuid
from IPython.display import display_javascript, display_html, display

class RenderJSON(object):
    def __init__(self, json_data):
        if isinstance(json_data, dict) or isinstance(json_data, list):
            self.json_str = json.dumps(json_data)
        else:
            self.json_str = json_data
        self.uuid = str(uuid.uuid4())

    def _ipython_display_(self):
        display_html('<div id="{}" style="height: 600px; width:100%;font: 22px/22px monospace !important;"></div>'.format(self.uuid), raw=True)
        display_javascript("""
        require(["https://rawgit.com/caldwell/renderjson/master/renderjson.js"], function() {
            renderjson.set_show_to_level(2);
            document.getElementById('%s').appendChild(renderjson(%s))
        });
      """ % (self.uuid, self.json_str), raw=True)

In [2]:

with open('assets/bids-v1.json') as jfile: bids_v1_template=json.load(jfile)

In [3]:

RenderJSON(bids_v1_template)

Within the definitions are Container Template definitions, which refer to the pattern template applied to a particular Flywheel Container.

Definition

Container Template: a portion of the Project Curation Template that dictates the information needed to structure one container, such as an anatomy file.

In [4]:

with open('assets/container_template_example.json') as jfile:

container_template = json.load(jfile)

In [5]:

with open('assets/container_template_example.json') as jfile: 
 container_template=json.load(jfile)

bids_v1_template = json.load(jfile)

BIDS Metadata: the set of metadata attached to a Flywheel Container (e.g., file, acquisition, session) that contains information for the BIDS standard stored as info.BIDS.<METADATA>. Defined via the Project Curation Template.

Container Template BIDS Metadata
icontainer_template_properties-1-1 bids_metadata_example-1

To apply the Container Template definitions, the rules are applied as follows:

  1. Determine if an object encountered should have that Container Template applied ( i.e., the where clause of the rule definition)
  2. If so, initialize the <label> and <index> sections for the chain of entities, or key-value pairs, relevant to the data type of the object encountered to capture the BIDS metadata necessary to stitch together the BIDS filename.

In [6]:

with open('assets/rules_example.json') as jfile:
    rules_example=json.load(jfile)

In [7]:

RenderJSON(rules_example)

6. Demo: Extend BIDS template

In [8]:

with open('assets/bids-v1-extension-project-template.json') as jfile:
    extension_example=json.load(jfile)

‍Debug Pro Tip ‍

Check your Project Curation Template with a JSON linter to ensure proper syntax. For example: https://jsoneditoronline.org/

‍Debug Pro Tip ‍

Check your regular expressions against the strings you are parsing (e.g., acquisition.label). For example: https://regex101.com/

In [9]:

RenderJSON(extension_example)