Discovery Admin is built on open source XNAT and provides the ability to configure connections between PACS or VNA, index the DICOM metadata, which will expose a high level view of the imaging data. Discovery Admin also enables management of data requests that come from Discovery Dashboard and allows uploading of imaging study identifiers. Data requests can then be pulled as full imaging studies from your clinical imaging system and staged for further processing and analysis.
This document provides instructions on how to index your PACS and create data requests, as well as monitor both indexing and requests.
Indexing the PACS into Discovery
In order to get a cohort-centric view of DICOM metadata in Discovery Dashboard, it is necessary to index PACS. This is the first step in populating the data that will be used in your visualizations. Once the Discovery plugin has been configured and the appropriate PACS and SCP receivers are in place, Discovery indexing can be executed.
Executing an Indexing Job
Indexing jobs are handled on a per date basis that correspond to the DICOM StudyDate tag. Execute for all days desired to visualize and build imaging cohorts from your PACS.
To execute an index job:
1. Begin by selecting the Index button at the top of the Daily Summary Index page and choose the date range to index
2. Select Index. The time taken for indexing will be very dependent on the pacs system accessed, network connectivity and the daily number of studies that require indexing. During the indexing process, the daily index number will be updated and the progress can be monitored.
Viewing and Canceling Indexing Jobs
Indexing jobs can be monitored on a per job basis. This will display how many days are complete out of days requested. Indexing jobs can also be canceled in this section.
To view or cancel indexing jobs, under Site Administration in the top navigation menu:
1. Select Plugin Settings
2. Select IQ Plugin
3. Select Indexing Jobs
Previous and running indexing jobs can be reviewed and canceled on this screen. Also, daily progress per job will be displayed.
Viewing the Daily Indexing Status
Indexing progress can also be viewed based on DICOM Study Date. This provides a general idea of what has been indexed and what is missing. Under Site Administration in the top navigation menu:
1. Select Plugin Settings
2. Select IQ Plugin
3. Select Daily Indexing Status
A summary of the indexing status based on a daily count is displayed. As indexing occurs, this table will be populated.
A data request is simply a collection of Study Instances or Accession Numbers that identifies a cohort of interest. There are currently two methods to submit a data request to Discovery Admin, (1) as a CSV upload, or (2) as a submission from Discovery Dashboard. Once a request is fulfilled, further analysis can be performed.
For Data Requests, please follow the below steps:
1. Begin by selecting the Under Administer
2. Then select Plugin Settings
3. Select IQ Plugin
4. Then select Data Requests
Once a cohort has been selected via the Discovery user dashboard, there is a Send to XNAT button on the Study Instance UID visualization. A CSV containing Study Instance UIDs Or Accession Numbers can also be submitted through the Discovery Plugin UI.
More information about the request can be viewed using the Info action including the requesting user, comment, date, Discovery query details, and the requested UIDs.
A data request involves two (2) distinct steps in the Discovery plugin:
- Submit a data request from the Discovery dashboard to request a new dataset from the PACS. This will import the required imaging data from the currently configured pacs systems and receive data via the configured DICOM receivers on the XNAT system. On arrival in the pre-archive images will be labeled according to the appropriate naming convention applied by the Discovery plugin. Additionally, all images will be assigned a date based project based on the year and month of their Study Date (200910). This will provide the master projects for every image pulled from the PACS system.
- Once data has been received and stored into these master date based projects, they need to be linked to a data ‘request project’. This will allow individual labs or researchers to have their own copy of the data without physical duplication of the original DICOM data files. This is achieved by duplicating the associated metadata of an imaging session and linking the underlying files to the original resources stored on disk.
Submit a Data Request: Option 1 – Discovery Dashboard Button
From the Discovery Dashboard, a user can select the Request Cohort button on the Study Instance UID visualization. This will send the list of filtered UIDs to the Discovery Plugin data request service. The request will remain in PENDING state until a site admin selects the Download action from the table. More detailed information on filtering is available in the Discovery Dashboard Documentation.
Submit a Data Request: Option 2 – Upload CSV File
1. To submit a Data Request, begin by using the file downloaded from the Discovery dashboard or elsewhere
2. Then choose New Request from the Data Requests screen
Once a request has been submitted it will be placed in a DICOM Query Retrieve queue and imported using the same mechanism used by the DQR Plugin.
The CSV upload currently supports the following formats:
Study Instance UID
Downloading a Data Request
A data request will be in PENDING state until an admin starts the download process. Select Download from the table for the desired data request to start the process. Once the request is passed to the DQR plugin, the status should change to SUBMITTED.
Monitoring a Data Request
The DQR Plugin gives you the ability to view a history of previously requested sessions, as well as a view of current requests that have been queued for import. This is available at Upload > Import Queue/Historyin the top navigation of your XNAT.
Any DQR-enabled user in XNAT has the ability to view their own import queue. All site administrators have the ability to toggle between viewing their own items and viewing import requests for the entire site. Here is an example of a current user's view of queued imports.
Import Queue / History Status Codes
The status field for items in the DQR Import Queue and Import History can be any of these:
- QUEUED – request is in the Import Queue and has not been submitted to PACS
- PROCESSING – request is being processed for submission to the PACS (this is typically a very ephemeral status)
- ISSUED – request has been issued to the PACS and XNAT is waiting for a response
- FAILED – request has been denied by the PACS, or the PACS was offline and could not accept the request
- RECEIVED – request has been approved by the PACS and data has been received
Viewing the PACS Import History
The display of a user's history table is very similar to the user's queue table. Click on the "History" tab on the PACS Import Queue and History page to bring up this table. See the DQR documentation for more details on how to monitor your download request.
Viewing Pre-Archive Status
Once the download requests are picked up by the PACS, the imaging sessions should start to appear in the prearchive, available under Upload > Go to Prearchive in the top navigation menu. Sessions will start to appear in the prearchive table as the PACS fulfills the requests, and the table entries will disappear as the data is moved into the master date based projects.
Note: More details on the prearchive are available in the XNAT wiki.
Sessions will start to appear in the Recent Data Activity section on the main XNAT page.
Linking a Data Request
Prior to linking a data request, one will need to create a new project for this data request if it does not already exist. To create a project, go to New -> Project in the top navigation menu.
Using the data requests screen, choose a previously completed data request and choose the Link button. Select the project from the drop-down list and click the link button. An email confirmation will be sent out once the data linking is complete.
Data Request Linked Project
This project provides a view into the cohort requested by the researcher. Data can be analyzed and processed further in this project, or it can be sent to Flywheel Enterprise using the Flywheel Integration tool.
A linked project now exists for the data request which will behave just like a normal XNAT project from the user’s perspective.