Bulk Importing Dose Readings

Navigation: Dosimetry Data > Bulk Importing Dose Readings

Dosimetry Data

Bulk Importing of Dose Readings can save the RSO significant time and is a highly efficient method of importing large volumes of legacy data. The Exposure Data panel within the Dosimetry tab includes an Import Button which displays a dropdown menu with options for Importing Dose Reports; they are the Dose Report (File) and the Dose Reports (Bulk/Folder) options;

Dosimetry Data Import Button

Dose Reports (Bulk/Folder) Import Option

The Main Steps in Bulk Importing

The Bulk import Process runs the same sequence of tasks as the Individual Dose Report Import which are described under Import Process Steps, The main differences are that the Bulk Importer applies the steps to multiple Dose Report Files and any that have import problems or anomalies are skipped.

Operating the Bulk Data Import tool involves the following steps;

1.Select Dose Reports (Bulk/Folder) from the Import dropdown button in the Dosimetry tab.

2.Select the required top-level folder containing Dose Reports a nested sub folder structure.

3.Immediately after selecting the top-level folder Dose Report files within the tree are examined.

4.Historion prompts for Centre Names for any Dose Report files containing unknown Centres.

5.When file examination is complete the list of files found can be viewed and Import is ready.

6.Bulk Importing of Dose Readings via Dose Reports found is started by clicking [Start Import].

7.The import is processed, and an import log is generated indicating which of the found files were successfully imported and those that weren’t. Any Dose Reports not successfully imported will have an import failure reason summary in the import log.

8.For large Bulk Import processes and for future follow up in case needed, the user selects copy to clipboard and saves the Bulk Import Log in a new text file, Word document or Excel sheet.

9.For any files not successfully imported; if the reason for import failure is unclear or if the log suggests individual importing for further detail, the user individually imports the file for more detailed problem diagnostic information.

For troubleshooting Dose Reports that fail to Bulk Import, individual Dose Reports should be imported using the Import Dose Report (File) option, not the Bulk Dose Report Import option.

Select the Required Folder

To import Dose Reports contained within a Windows folder structure (a folder including all subordinate subfolders in any folder tree beneath it) into Historion, select Dose Reports (Bulk/Folder)

A Windows file system navigation dialog will be displayed prompting for selection of the required top-level folder. Navigate to and select the required top-level folder then click the [OK] button to start the Dose Reports Bulk Import Process;

Historion Checks the Selected Files

If the required top level folder has been selected and when the [OK] button is clicked the following Bulk Import Utility tab is displayed and an initial file checking process commences;

Each file found within the folder selected and any sub folders beneath it is examined. The file formats are checked and those matching known Dose Report File Formats (.CSV, .XLS, .XLSX, .XML) are all compared to active Dose Report Type Definitions.

Once Historion starts file checking the process must run to completion. During this process the [Cancel Import] and [Start Import] buttons are deactivated. This is to ensure that all files involved, which are being sequentially opened and examined, are properly closed and the task, which is automated, completes without ambiguous results.

Add Centre Names if Prompted

The field representing Centre Number within files matching known Dose Report Types are checked against the Centre entries already recorded in the Historion database. The first time (only) that Centres are introduced to Historion (via a Centre, Series, Account, Area, Site or Department code) in a Dose Report file, a tab will request the Centre Names (their labels).

Adding Centre Names when prompted is covered above under Record Centre Names. Recording Centre Names is not mandatory for the Bulk Import Process to continue. This is one of several ways in which the Bulk Import is different to the single file Import.

If multiple Dose Report files confirming to known Dose Report Types are present within the selected folder, any with unknown Centres will be skipped by the Import Process if their respective Centre Names are not recorded when prompted during import. The Importer will attempt to import remaining files that do not have any unidentified Centres for their Dose Readings.

File Checking Completes

When file checking has completed and Centre Names provided if prompted the following confirmation message is displayed and the Bulk Import process is ready to start;

Check Files in Path

When all Dose Report files found within the selected folder structure have been examined Historion displays the [Cancel Import] and [Start Import] buttons and the Files in path link is displayed.

Files in path link

Selecting the Files in path link provides a list of all files found having known file formats;

Tip: paste the list of files found into a text file, word document or Excel worksheet by right mouse clicking the list of files and using Select All followed by right-click and copy;

For a large folder structure where large numbers of Dose Report files are found copying the list of detected files to a new text file, Word document of Excel spreadsheet can be helpful for reviewing and import preparation purposes. The list allows for confirmation that the correct folder was selected and the correct set of Dose Report Files have been selected for Bulk Importing.

Tip: if a tall folder structure is selected and some file names are very long it is helpful to maximise the File List tab for easier viewing and to see more files listed on tab;

Maximise File List tab

Review Files for Import

If the File list is open close it by selecting the Close File List link. This return the view to Bulk Dose import tab ready for importing.

The Bulk Import can be cancelled from the File List Tab by clicking the [X] close button instead of the Close File List link. This may be needed, such as in the event that the review of the list of files found results in the realisation that the wrong folder has been selected.

If closing the File List Tab from the [X] close button a prompt is displayed for confirmation as the Bulk import will be aborted if the close is continued;

Confirm the Import Mode Required

There are two import mode options available at the lower middle area of the Bulk Import Utility tab;

The default mode is Check Reading Duplication After Import (Faster). This option results in Duplicates not being detected whilst importing, between Dose Reports, this check waits until all Dose Reports selected have been imported in bulk. Duplicate detection runs once only at the end of the Bulk Import process. Duplicate checking is time expensive as the full set of Dose Reading are reloaded each check.

In the default mode, if the import process is interrupted it is strongly recommended the User immediately run a manual Duplicate Detection scan to find any duplicates imported and to set the instance numbers of all imported readings (which is done as part of the same process).

Check Reading Duplication After Import (Faster) Bulk Import Mode

This option is frequently used at the outset of a new Historion implementation whilst importing large numbers of historical or legacy Dose Reports into Historion, especially during user acceptance testing.

This option is significantly faster, but not always preferred in established implementations. This option is used, for performance purposes, by the Online Data Transfer interface (participating providers) in Historion versions 6.0.1.7 onward.

Check Reading Duplication During Import (Slower) Bulk Import Mode

This option instructs Historion to check the whole system for duplicated readings between each Dose Report imported. This provides a fail-safe if the process stops, is interrupted or does not complete for any reason. A Duplicate Scan is suggested if this occurs but shouldn’t be strictly necessary as the database is constantly kept up to date with reading instance numbers and duplicate detection.

This option requires re-loading of the locally cached Dose Readings data, from the Historion database, after each Dose Report has been imported hence it can take in the order of ten times longer (or longer) to Bulk Import Dose Readings with this option set.

Checking for potential duplication between imports is significantly slower but may be preferred by those RSO’s with well-established Historion implementations and clean data and if there is a suspected or potential high risk of duplication of Dose Readings in a new Bulk Import set.

Complete the Bulk Import

After reviewing the files for import and confirming the most suitable Import Mode the Bulk Import can commence by clicking [Start Import]. The lower progress meter indicates that the Bulk Import is now running and how far along it is, in working through the set of files to be imported.

Once Historion starts a Bulk Import the process must run to completion. During this process the [Cancel Import] and [Start Import] buttons are removed. This is to ensure that all files involved, which are being sequentially opened and examined, are properly closed and the task, which is automated, operates without any incomplete updates.

When the Bulk Import Utility has completed processing all of the files the following message is displayed;

Click on [OK] to display the final Bulk Import Utility log window as described below.

Bulk Import Utility Log

The Bulk Import Utility Log window is displayed on completion of processing all applicable files within the selected top level folder and any subordinate folders. The files are processed sequentially and the result of each import attempted is displayed;

Bulk Import Failure Messages

The Bulk Import process will skip past files that, for any reason, cannot be imported. The Individual Dose Report (File) Import function provides more diagnostic detail for a file that does not import. The Bulk Import Tool Log (above) indicates the first reason for skipping each file, for any files not imported. In the example above the following file import outcomes can be observed and are typical;

IMPORTED OK. (Two of the files in the visible log area shown were successful imported.)

Not imported. The Dose Report format is unknown. Import this file separately for more information,

Not imported. Unknown Centres in Import file.

Not imported. This Dose Report has already been imported.

Not imported. The file is missing data for start date.

There are numerous reasons that can lead to a Dose Report not being imported successfully. Some Dose Reports may not import due to causes that are Dose Report Type and structure specific, so a sample of failure reasons, only, is provided here. The log detail provided should provide an indication of the issue. Alternatively Historion will provide more troubleshooting diagnostics if files that will not Bulk Import are individually imported instead.

The Bulk Import Log can be kept for further analysis, by selecting the Copy to Clipboard link at the top right corner of the Bulk Import Log tab and pasting the log into a new text file, Word document or Excel workbook. It is recommended these log files be retained.