Data Views

Introduction

Flywheel captures a great deal of data and metadata from imported data sets, processing results, and information added by users. All of that data is indexed and can be used in a Data View report.

Example: Create a demographics report for all of the subjects in a study or aggregate data from the exams in a longitudinal study to assess changes over time.

Instruction Steps

View a Site Template

Site admins have the ability to create site templates. Site templates will automatically appear in each project. Users in that project can run the report against their own project data to generate a Data View. This allows you to standardize reporting throughout the site.

1. Navigate to your project.

2. Click Data View from the top menu.

1. Click the name of the Site Template. The template pulls data from this project to create the report.

2. Click Export to download the report in CSV, TSV, JSON, JSON-Flat, or in JSON row-column format

Create a Data View for a Project

Besides the Site Template, a Data View can be defined at the project level. This allows you to standardize reporting within a project.

1. Navigate to your project.

2. Click Data Views from the top menu.

1. Click Create New Data View.

2. Enter a name and a short description so other users know what type of report they are generating.

3. Begin typing in the Available Columns to filter field names. Note that the available columns list includes only Flywheel's default fields in this view. Additional fields, such as custom metadata, are available by using the 'Filter Available Columns' search. Note that Project level data can not be included in a Data View.

4. Select the arrow to add the field to Selected Columns.

5. Click on a field in Selected Columns to edit the alias. The alias will be the column header in the report.

1. Optional: preview the tabular report by clicking Preview.

2. Press Save to save the Data View.

Generating a Tabular Report

Once a Data View is defined, it can be used to create tabular reports for viewing or export.  This report will include the latest project data at the time of execution.

1. To view data in the report, navigate to your project and click the name of the template from the Data Views tab.

1. This initiates report generation and adds the report job to the Queue. 

2. For smaller reports, the report will be displayed within seconds. For larger reports, you may have to wait, but you can leave the page and check the Queue tab to find the latest status of the report.
   

3. Reports are saved in the queue for 30 days. For each report you can view, save to the project, download, or delete the report.

If you want the report to remain in long term storage, then you should save the report to the project or download the report. Download supports CSV, TSV, JSON, JSON-Flat, or JSON row-column format

Data View Filtering

Data Views can be defined with a filtering step that limits the data in the report to reflect specific criteria.  Click the Edit Filters button to define the inclusion criteria.

Filtering is straightforward on containers and metadata. 

Filtering on file attributes can lead to unexpected results or errors because files can exist at all levels of the hierarchy. File filtering depends on appropriately setting the container level of the file.  See the next section for details.

File Level Settings and Filename Filtering

When columns beginning with 'file' are selected in the Data View, they have special settings for setting the container level of the file (subject, session, or acquisition) and file name filtering.

Since files can exist at multiple levels of the hierarchy, specifying the level is required.

The level is also constrained by the other columns you selected in the Data View. The container level must be equal to or lower than the column you select.  

Example: If you include a session level column, then the container level you select could be either session or acquisition, but not subject. In other words, if you want to select files from the session level of your project, then you may select session, subject or project level columns, but not acquisition level columns.

Filename filtering can be done via direct match (with * as wildcard) or as a regular expression.

Data Views - Grouping and Aggregation

In some cases you may want to group data and provide a numerical summary of the groups (for example, the count of subjects by subject.sex).  You would first select the grouping variable(s), which must reflect text or integers. Then the aggregation method (for example 'count') and variables (reflecting numeric columns).  This will result in a summarized data set.

1. Select the grouping variable(s), which must reflect text or integers. To select a column for grouping, click on the Group by this column checkbox.

1. To select a column for aggregation, apply an Aggregation Method to the column.

Optional - Changing the data type for Grouping or Aggregation

Flywheel allows flexible data typing when creating metadata.  For example, it is possible to define subject.info.status data for two subjects, each having different data types. This can be a challenge when creating tabular data from mixed type data.

Using the Coerce Type setting allows you to change how the data view interprets the data.  Coercion will only work if the data can be retyped.  For example, a String of '1.0' can be coerced to a Float of 1, but a String of 'one' can not be coerced to a Float.  In this situation, that row would show an error in the error column.   When data with type Float is coerced to an Integer, it will be truncated after the decimal point.

If you want to apply different aggregation methods to the same data column (Min, Max, Mean), you can do that by adding the same column multiple times to the Data View and using the Alias field on the General tab to give them different aliases. Then set the aggregation method for each of these aliased columns. If you want to apply aggregation to the same column used for grouping, you need to create an alias for the aggregation.

Limitation: When using Grouping, the fields used for Filtering must be ones that are higher in hierarchy than the lowest level Aggregated, or must be exactly the fields Aggregated, otherwise an error will result.

Advanced Options

• By default, rows with missing data are shown in the report. Toggle Hide rows with missing data to remove these rows.

• By default, reports include an error column. This column is used to show any numerical warnings or errors that occured as part of the aggregation.  For example, if missing values are encountered in the data, the aggregation will show those cases in the error column.  Toggle Display error column to remove this column.

SDK

The Flywheel SDK provides some additional features for Data Views, including incorporating data files and analysis results files into the Data View alongside the Flywheel metadata.

• MATLAB
• Python