Troubleshooting Bulk Export
Audit Report
After an export job terminates for any reason (whether successful, cancelled, or failed), an audit report is made available.
The audit report contains details of what action was taken for every single source file selected for export (based on the specified export rules), including any errors that may have occurred. The audit report can be extremely helpful in troubleshooting issues, and it can also be useful for maintaining a record of exactly which files were exported and where each file was placed in the destination.
The audit report will be automatically placed at the root of the export destination and named after the export job ID, e.g., export-61a4edde0bd9c95dbd561230-logs.csv.gz
. In this case, the audit report is in the CSV format and compressed using Gzip (*.csv.gz
).
Downloading the Audit Report
Via the Web App
The export audit report can be downloaded via the Flywheel Core Web App by navigating to the Bulk Export page, locating the export job line, and then either:
-
Selecting the "Download Report" option in the overflow menu on the export job line, or
-
Opening the Export job details dialog from the overflow menu on the export job line and then selecting the "Download Report" button.
Currently, only the CSV format is available when downloading the audit report via the Web App. To download the audit report in JSON Lines format, use the new (BETA) CLI.
Via the New (BETA) CLI
The export audit report can be downloaded via the new (BETA) CLI using the following command:
Where:
<id>
is replaced with the ID of the export job to be monitored, and<format>
is eithercsv
orjsonl
If you do not know the export job ID, see the article locating the export job ID.
Tip
The output will be written to the terminal (stdout), so you may want to redirect the output to a file for storage instead. E.g., fw-beta export get 682ccd2079fed7cd5c0cd46e --report jsonl > ~/Downloads/export-audit-report-682ccd2079fed7cd5c0cd46e.jsonl
For more information, see the new CLI documentation for export get
.
Formats for the Audit Report
The audit report is available in either CSV or JSON Lines (JSONL) format.
CSV
The CSV version of the audit report contains the following columns:
status
-- Indicates whether the file was successfully exported, skipped, or if an error was encounteredreason
-- Reason for why the file was not exported (empty if the file was successfully exported)src_path
-- Path (including name) of the file in the export Destinationdst_path
-- Path to where this file should be placed in the Flywheel Hierarchy
Export Audit Report -- CSV Example
The following table shows an entire CSV-formatted Export Audit Report.
status | reason | src_path | dst_path |
---|---|---|---|
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-01.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-01.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-02.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-02.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-03.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-03.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-04.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-04.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-05.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-05.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-07.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-07.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-06.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-06.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-08.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-08.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-09.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-09.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-11.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-11.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-10.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-10.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-14.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-14.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-12.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-12.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-13.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-13.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-15.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-15.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-16.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-16.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-17.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-17.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-18.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-18.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-20.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-20.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-21.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-21.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-19.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-19.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-22.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-22.dcm |
created | nan | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-23.dcm | Enigma Biosciences/PGBM-017/{StudyDate}/5361 - Mask_Tumor/1-23.dcm |
created | nan | Enigma Biosciences/PGBM-017/19970917/32602 - dT1/32602 - dT1.dicom.zip | Enigma Biosciences/PGBM-017/19970917/32602 - dT1/32602 - dT1.dicom.zip |
created | nan | Enigma Biosciences/PGBM-017/19970917/37614 - FLYW_reg/37614 - FLYW_reg.dicom.zip | Enigma Biosciences/PGBM-017/19970917/37614 - FLYW_reg/37614 - FLYW_reg.dicom.zip |
created | nan | Enigma Biosciences/PGBM-017/19970917/37616 - T1pre_reg/37616 - T1pre_reg.dicom.zip | Enigma Biosciences/PGBM-017/19970917/37616 - T1pre_reg/37616 - T1pre_reg.dicom.zip |
created | nan | Enigma Biosciences/PGBM-017/19970917/37619 - nCBF_reg/37619 - nCBF_reg.dicom.zip | Enigma Biosciences/PGBM-017/19970917/37619 - nCBF_reg/37619 - nCBF_reg.dicom.zip |
created | nan | Enigma Biosciences/PGBM-017/19970917/37620 - sRCBV_reg/37620 - sRCBV_reg.dicom.zip | Enigma Biosciences/PGBM-017/19970917/37620 - sRCBV_reg/37620 - sRCBV_reg.dicom.zip |
created | nan | Enigma Biosciences/PGBM-017/19970917/37751 - T2_reg/37751 - T2_reg.dicom.zip | Enigma Biosciences/PGBM-017/19970917/37751 - T2_reg/37751 - T2_reg.dicom.zip |
created | nan | Enigma Biosciences/PGBM-017/19970917/37621 - nRCBV_reg/37621 - nRCBV_reg.dicom.zip | Enigma Biosciences/PGBM-017/19970917/37621 - nRCBV_reg/37621 - nRCBV_reg.dicom.zip |
created | nan | Enigma Biosciences/PGBM-017/19970917/5361 - Mask_Tumor/5361 - Mask_Tumor.dicom.zip | Enigma Biosciences/PGBM-017/19970917/5361 - Mask_Tumor/5361 - Mask_Tumor.dicom.zip |
The following table shows a single line from a CSV-formatted Export Audit Report. The line has been transposed for easier viewing.
column | row value |
---|---|
status | created |
reason | nan |
src_path | Enigma Biosciences/PGBM-017/19970917/5361 - Mask_Tumor/5361 - Mask_Tumor.dicom.zip |
dst_path | Enigma Biosciences/PGBM-017/19970917/5361 - Mask_Tumor/5361 - Mask_Tumor.dicom.zip |
JSON Lines
The JSON Lines version of the audit report contains far more technical details about how the system processed each file.
Here is an example of just one line from an export audit report in JSON Lines format describing how the system processed a single source file:
Export Audit Report -- JSONLines Example
The following is an entire JSONLines-formatted Export Audit Report.
The following is a single line from a JSONLines-formatted Import Audit Report. The line has been reformatted for easier viewing.