Brain Imaging Data Structure (BIDS) is a codified way of organizing neuroimaging and behavioral experimental data for the purpose of sharing. If you are new to the BIDS standard, we strongly recommend reviewing the Common Principles on the BIDS website or downloading a PDF of the BIDS specification.
There are two options for getting BIDS-formatted data on Flywheel: upload data that is already in the BIDS format, or curate data on Flywheel to fit the specification. This article gives an overview of how to complete BIDS curation as smoothly as possible.
Note
Note: While the BIDS curation process is not difficult, setting up the process will take time, attention to detail, and will require input from someone who understands how and why each scan was acquired (for example, someone who knows the study design.)To begin, you'll want to take a look at the BIDS Study Design Spreadsheet. This will help you plan out how to get your data into BIDS.
Next, you will use 3 gears for BIDS curation:
DICOM MR Classifier: Interprets DICOM tags for a basic understanding of the type of scan
dcm2niix: Converts DICOM files into NIfTI files and set Flywheel metadata with DICOM tag values
curate-bids: Uses a template to set BIDS metadata on Flywheel
The first two gears can be run using gear rules so that they are executed automatically whenever new scans appear. The third, curate-bids, can be run after all scans have been turned into NIfTI files in dcm2niix.
However, before you can successfully run the curate-bids gear, your data must be labeled in the way that the curate-bids gear expects. How you update the labels depends on if you are obtaining new data uploaded directly from the scanner or if you are working with Retrospective data.
For new data that will be acquired, the best way to begin is by setting the proper names at the scanner console. For this, we highly recommend the ReproIn naming convention.
When the ReproIn naming convention is used, the BIDS Curation gear can be used with the default ReproIn template, and BIDS curation will be almost automatic.
Learn more about the ReproIn naming convention or take a look at a scanner walkthrough .
For retrospective data that has already been acquired, use the BIDS Pre-Curation gear to rename Flywheel acquisition, session, and subject code/labels so that they match the ReproIn convention. Then you can run the BIDS Curation gear with the ReproIn template (learn more about the ReproIn naming convention.)
There is often hesitation at relabeling data because it might obscure provenance, but rest assured that relabeling is safe on Flywheel because the raw information is not changed. The original acquisition name can be retrieved from the SeriesDescription DICOM tag. The session label can be retrieved from the StudyDescription and the subject label/code is typically the Patient ID or Additional Info fields (check with the Flywheel Site Admin at your institution for the specific fields configured for your site.) The BIDS Pre-Curation gear also allows you to mark specific files to be ignored by adding “_ignore-BIDS” to the end of an acquisition label. Flywheel will then skip all files in that acquisition when exporting data in BIDS format.
Note
NoteNote
If you only want to change a small number of labels, this can be done manually in Flywheel, but if a large number of changes need to be made, the BIDS Pre-Curation gear should be used.Once you have relabeled your data, you will run the curate-bids gear to start determining where your files fit into the BIDS specification. The curate-bids gear (also known as the BIDS Curation gear) walks the Flywheel hierarchy of a project (subjects, sessions, acquisitions) and matches files with specific parts of the BIDS specification using rules and definitions in a project curation template. The definitions in this template establish the structure of the BIDS path and file names, while the rules are used to recognize files and then extract parts of names to determine each file's complete BIDS path and name. You can learn more about Flywheel's project curation template in our article.
After running the BIDS Curation gear, the next step is to check the results. The latest version of the gear produces spreadsheets that summarize the mapping between the original names to the final BIDS paths and filenames. If you are not using the latest gear version, you can run a script to generate these reports. The gear (and script) can take a list of pairs of regular expressions for fine-tuning the mapping between field maps and the files that they will modify. Initial processing using the project curation template in the gear produces a list of all possible files for each field map to modify. The regular expressions provide a more specific correspondence between the field map (matching the first regex) and the scans to modify (matching the second regex).
The spreadsheets produced are:
<group>_<project>_niftis.csv: A list of the original information (acquisition name, file name, series number, etc.) to the final BIDS path/filename. A column indicates if the path/filename is duplicated, which will result in an error for the BIDS Validator. BIDS Apps run the BIDS Validator and will error-out if there is a problem so the gear won’t run. This spreadsheet should be checked to see if all of the files have been properly recognized or ignored.
<group>_<project_acquisitions.csv: is useful when there are multiple subjects because it shows the “usual count” of the acquisitions for all subjects and then lists the subjects that have these expected acquisitions and the ones that do not. It lists warnings for unexpected numbers of specific acquisitions and errors for subjects that do not have the expected number of the usual acquisitions.
<group>_<project>_acquisitions_details_1.csv (_2.csv): lists all of the unique acquisition labels along with the number of times they have been seen. It also provides additional details that should help understand which subjects have missing or additional acquisitions.
<group>_<project>_intendedfors.csv: lists the field maps and then the paths to the files that those maps are going to be used to correct. If the IntendedFor regular expression pairs are provided, it will list the mapping provided by processing using the project curation template as the “before” results and also the after using the regexes to trim down those results.
BIDS Curation is usually an iterative process of editing names, running the BIDS Curation gear, and then checking the above spreadsheets. However, it can also involve editing the project curation template. Learn more about editing the project curation template below as an alternative method to renaming the files.
Note
NoteThe curate-bids gear does not actually save data in BIDS format. Instead, it sets the Flywheel metadata that will be used when BIDS-formatted data is exported using the CLI and when running a BIDS App gear such as BIDS-fMRIPrep.If you do not want to rename files, it is possible to edit a BIDS project curation template so that it can recognize and process arbitrary file names. Typically, this is not recommended because the template is a large complicated json file and exactly what is happening during the processing accomplished by the “where” and “initialization” sections is best understood by stepping through it using a debugger. See an example of these sections in the BIDS template file article.
Crafting particular regular expressions in the template to recognize and extract the proper strings from arbitrary file names makes the template processing brittle whereas changing the names of acquisitions, subjects, and sessions to use the ReproIn convention that is expected by the existing template not only allows the template to work, but also makes the purpose of each acquisition clear on the platform.
Getting data into BIDS format makes processing and sharing data easier. Using the ReproIn naming convention starting at the scanner console or else by renaming acquisitions makes BIDS curation easier because it makes the link between each acquisition and where it fits into the BIDS Specification explicit.
Take a look at our webinar on how the CMU-Pitt BRIDGE Center standardizes on BIDS using Flywheel. You will also learn more about how Flywheel's centralized platform streamlines BIDS in an end-to-end solution for data management and collaboration.
Once you have an understanding of BIDS in Flywheel, start planning your curation by using the BIDS Study Design Spreadsheet or take a look at our BIDS curation tutorial to start curating your own data.