How the BIDS Template Engine Processes Data

Introduction

At the core of BIDS uploading and curation is a template processor that is capable of matching objects and files to templates, and automatically updating fields (such as Filename and Path. Learn more about what these fields do in the BIDS overview article.)

This article gives an overview of how the Flywheel BIDS template processes data BIDS.

It is important to understand how the templating engine processes data to build or modify templates. The template engine stores these special properties in the BIDS metadata:

• template - The name of the matched BIDS template is stored here after the first run of the BIDS template engine.
• valid - Whether or not the metadata is currently valid, as of the last BIDS template engine run. A metadata object is valid if there are no JSON schema errors, and all required fields are populated.
• error_message - If valid is false then this will be a string of error messages, one per line, indicating why the metadata is invalid.
• ignore - If set to “true”, then this file will be ignored by the exporter.

When the templating engine encounters an object or a file to process, it checks the BIDS metadata object to see if the template field is set. If the template field is not set, then the engine will process the object in initial mode, otherwise if a template is specified, then the engine will process the object in update mode.

• Initial mode: The engine processes all of the rules defined in the template JSON file against the current context object, until a match is found. If a match is found, then the BIDS metadata object will be initialized using the default properties defined in the matched template. Next, any initializers defined in the rule will be run. In addition, the template property will be set to the name of the matched template definition. If no match is found, then the BIDS property on the info object will be set to the string literal: “NA”.
• Update mode: the engine will use the current context to populate any auto_update fields specified in the matched template (Learn more about Auto Updates in the BIDS template file)

When running the curate-bids gear, the option, "reset" will clear out all previous BIDS information so that it is running in "Initial mode". If "reset" is not chosen, it will be in "Update mode".

The templating engine walks down the entire project hierarchy, building a context object as it goes. The context object stores each object in the hierarchy as a named property, and will include the current Flywheel container type (project, session, acquisition, or file) as well as the parent container type. You can see an example of this context object below.

Instruction Steps

Example Templating Context

Here is an example of what a context object might look like for an acquisition file:

{
  "project": {
    "files": [], 
    "info": {
      "BIDS": {
        "Funding": "", 
        "Name": "bids", 
        "License": "", 
        "HowToAcknowledge": "", 
        "BIDSVersion": "1.0.2", 
        "ReferencesAndLinks": "", 
        "DatasetDOI": "", 
        "template": "project", 
        "Authors": [], 
        "Acknowledgements": ""
      }
    }, 
    "group": "scitran", 
    "created": "2018-01-04T21:44:39.927Z", 
    "modified": "2018-01-25T21:59:34.362Z", 
    "label": "bids", 
    "_id": "5a4ea0472081c60014082965", 
    "permissions": [
      {
        "access": "admin", 
        "_id": "justinehlert@flywheel.io"
      }
    ]
  }, 
  "parent_container_type": "acquisition", 
  "session": {
    "info": {
      "BIDS": {}
    }, 
    "group": "scitran", 
    "created": "2018-01-04T21:44:40.088Z", 
    "timestamp": "2017-11-07T12:25:05Z", 
    "modified": "2018-01-23T21:45:44.013Z", 
    "label": "session01", 
    "project": "5a4ea0472081c60014082965", 
    "permissions": [
      {
        "access": "admin", 
        "_id": "justinehlert@flywheel.io"
      }
    ], 
    "_id": "5a4ea0482081c60017082969", 
    "subject": {
      "code": "wl_subj045_20171107", 
      "_id": "5a4ea0482081c60017082968"
    }
  }, 
  "acquisition": {
    "files": [
      {
        "origin": {
          "type": "user", 
          "id": "justinehlert@flywheel.io"
        }, 
        "mimetype": "text/plain", 
        "name": "wl_subj045_20171107+01+AAHead_Scout_64ch-head-coil_01-header.txt", 
        "created": "2018-01-04T21:44:40.185Z", 
        "modified": "2018-01-04T21:44:40.185Z", 
        "info": {}, 
        "type": "text", 
        "size": 12199
      }
    ], 
    "created": "2018-01-04T21:44:40.125Z", 
    "modified": "2018-01-25T21:59:34.588Z", 
    "label": "01+AAHead_Scout_64ch-head-coil", 
    "session": "5a4ea0482081c60017082969", 
    "_id": "5a4ea0482081c60018082964", 
    "permissions": [
      {
        "access": "admin", 
        "_id": "justinehlert@flywheel.io"
      }
    ]
  }, 
  "file": {
    "origin": {
      "type": "user", 
      "id": "justinehlert@flywheel.io"
    }, 
    "mimetype": "text/plain", 
    "name": "wl_subj045_20171107+01+AAHead_Scout_64ch-head-coil_01-header.txt", 
    "created": "2018-01-04T21:44:40.185Z", 
    "modified": "2018-01-04T21:44:40.185Z", 
    "info": {}, 
    "type": "text", 
    "size": 12199
  }, 
  "ext": ".txt", 
  "container_type": "file", 
  "subject": {
    "code": "wl_subj045_20171107", 
    "_id": "5a4ea0482081c60017082968"
  }
}

Click Next to learn more about the BIDS template file.