Data Quality
Overlapping Dose Readings
Overlapping Data in Historion specifically refers to Dose Readings for the same wearer, with wearing periods that cross over each other at the start or end (or both) of their wearing date ranges.
Duplicated data and unexpected or errant Overlaps in Historion are highly undesirable under most circumstances. Historion safeguards against duplication rigorously including the use of Overlap detection as an additional data validation tool.
Detection of Overlapping Readings in Historion is an important Data Integrity Control. Overlapping Readings, when detected, can be indicators of partial or full Data Duplication, not otherwise apparent when overlaps remain unchecked.
Overlapping Readings can be indicators of data errors, but can also be indicators of inaccurate record keeping, associated with badge distribution and administration; dates of return and other events.
Sometimes overlapping data is difficult to avoid, such as when a Wearer is “double-badged” (wearing multiple dosimeters) and multiple results are sought and expected for the same wearing period.
The important task for Historion is to make the presence of Overlapping Readings visible to the RSO, so that the RSO can investigate and assess the legitimacy of results for badge wearing periods that appear to cross over and whether the data is unusual and needs further investigation.
Common examples of Dose Reading Overlap causes include;
Re-issued or “versioned” Dose Readings replaced by the Dosimetry Provider after review
Provider Laboratory Error due to procedural fault or failure
Provider Data Error; software fault or failure of data reporting or interpretation
Provider Human error
Client Organisation Human error in manual Dose Report changes
Special purpose testing and research
Pregnancy Monitoring (monitoring of mother and foetal monitoring at the same time)
Client Organisation badge identification or administration or returns procedural mistakes
Double Badging (wearing multiple dosimeters)
Double Badging is where individuals wear multiple monitors at the same time, such as for pregnancy monitoring or extremity monitoring with left and right hand (or other body) sides.
In some scenarios a monitor might be worn on the outside of a lead apron and another monitor might be worn on the inside of the lead apron. A monitor might be worn on the body and concurrently another at the upper thoracic region, in various procedures and applications.
Sometimes Provider data will not fully explain the Dosimetry Application context. As a result, legitimate extra Dose Readings, that are expected, may appear as overlapping or duplicated when first imported and not yet assessed.
Handling Overlapping Readings
There are three steps in handling Overlapping Reading, described further below;
1.Scan for Overlapping Dose Reading occurrences
2.Decide on a course of action (4 scenarios are described here)
3.Replace, Allow, Delete or Ignore overlapping Dose Readings
in future we will include void here
How to Check for Dose Reading Overlaps
Historion does not automatically check for Dose Reading Overlaps.
Checking for Overlaps can be time consuming as dates for Dose Reading Wearing Periods are thoroughly cross-checked against the corresponding date ranges of other Dose Readings for the same Wearer and Provider.
Checking for Overlaps after importing Dose Readings is recommended and requires opening the Dose Reading Overlaps listing page from the Overlap feature icon in the Data Quality panel then executing an overlap scan by selecting the rescan icon;
.png)
Overlaps Data Quality Utility
If a scan for Overlaps has never been done the scan icon will be labelled “Scan” otherwise “Re-Scan”
To commence scanning for Dose Reading Overlaps simply requires selecting the Scan icon shown;
.png)
Scan/Re-Scan to Start Process
.png)
After confirmation the scan process will run, and the number of Overlaps detected advised.
.png)
Dose Reading Overlap Groups and Instances
The rows in the Overlap Readings Listing Page represent individual Dose Readings that Historion has detected as overlapping. The Dose Reading Overlaps are sorted and presented in sets in the list known as Overlap Groups.
The Overlapping Readings List includes a mixture of Dose Reading columns and some columns normally associated with Dose Reports, such as the [Dose Report Readings] column which shows the number of Dose Readings in the parent Dose Report that each overlapping Dose Reading belongs to.
The layout of the Overlapping Readings List and the default columns displayed were selected to assist in identifying groups of Overlapping Reads and to quickly identify whether different Dose Reports are involved with the overlaps. Historion highlights rows belonging to alternate Overlap Groups in different Colour Bands for clarity.
Overlap Group “Set” Grey/White/Grey Colour Banding
.png)
Overlap Instance
The Grey-White-Grey colour banding emphasizes the sets of Dose Readings, the Overlap Groups, making them easier to see in the Overlaps list. Alternating bands are shown with green arrows above.
Each Overlap Group is assigned a non-permanent Group Number when Historion runs its Overlap scanning process. The Overlap Group number is a simple numeric sequence and a new number may be assigned to a group if Dose Reports are imported or deleted and Overlap Detection re-run. The Overlap Group 2 is emphasized in the red box in the example above.
Each Dose Reading that is a Detected Overlap is assigned a simple numeric sequence within its parent Overlap Group termed Overlap Instance. The Overlap Instance number is a simple numeric sequence and a new number may be assigned to a Dose Reading Overlap if Dose Reports are imported or deleted and Overlap Scanning is re-run.
The highest Overlap Instance within an Overlap Group indicates the number of times Dose Readings for the same Wearer have been detected in Historion for Wearing Periods that intersect. In the example above the blue checked line emphasizes a set of two overlapping Dose Reading instances.
Dose Reading Overlaps may appear in the same Dose Report or across multiple Dose Reports. This is important information when trying to decide how to handle duplicated Dose Readings because it will lead to further identification of the cause of the duplication.
The Dose Readings column shows the number of Dose Readings present in the parent Dose Reports and is useful as a quick check to see if the Overlapping Dose Readings are from the same or different Dose Reports.
This highlights the importance of the Dose Report Readings column in identifying where different Dose Reports are involved and the circumstances around overlapping Dose Readings. Similar comparison information can be gained from the Dose Report Date/Time imported columns.
When trying to understand the circumstances around detected Overlapping Dose Readings observe the Duplicate Grouping and Instance details and the numbers of Dose Readings column, to determine whether multiple Dose Reports contributed to any duplication.
Overlap Indicator Symbol
Historion’s Visual Communication Standard provides for immediate recognition of Dose Readings flagged as Overlapping Dose Readings, as shown in figure 143 below.
.png)
Overlap indicator symbol
Dose Reading Overlap Information Columns
In addition to the Overlap Indicator Symbol outlined above, Historion provides a set of Overlap Information columns for each Dose Reading (in Dose Reports as distinct from the Overlap lists);
.png)
Overlap Group and Instance values
None of the column values shown above, described below, can be changed directly in their respective cells within the Dose Report Details page. These values show the results of calculations and actions in managing Overlapping Readings in the Overlap lists described above.
Sorting these columns within a large Dose Report can be helpful to show all duplicates together. By default, the Overlap Groups and Instance values will appear ordered by their numeric sequence.
The definitions of these columns are as follows;
Is Overlap
This column provides an additional visual indication as to whether the Dose Reading has been detected as an Overlapping Dose Reading.
Resolved Overlap
This column provides a visual indication as to the whether the Dose Reading overlap has been classified by the RSO as a Resolved Overlap Reading.
Overlap Group
This column provides a visual indication as to the numeric group number assigned to a set of Overlap Dose Readings detected as further described above.
Overlap Instance
This column provides a visual indication as to the numeric instance or sequence number assigned to a specific Dose Reading within a set of Dose overlaps detected, as further described above.
Deciding on a course of action (4 scenarios)
Replace
Before replacing a Dose Report, ensure a new version of the Dose Report has been received from the Dosimetry Provider. A Dose Reports can then be replaced by first deleting it from Historion and then importing the new version.
Resolve
Before Resolving an Overlapping Dose Reading it is strongly recommended to review and where required change the Dose Reading Classification for the Overlapping Dose Reading to be resolved. Overlapping Dose Readings can be set to Resolved (kept) in Historion if the RSO has determined that the Dose Reading overlaps are legitimate, given the circumstances of monitoring and data involved.
A Resolved Overlapping Dose Reading is moved to the Resolved overlapping Dose Readings list and is then hidden from view by default. This is a basic Historion Workflow concept intended to assist in managing large volumes of Dose Readings that have been flagged for further assessment and management by the RSO.
Overlapping Dose Readings can be Resolved in three ways; by ticking the [Is Resolved Overlap] cell for a single selected Overlapping Dose Reading in the Overlap Dose Readings list as shown below or as a group by selecting multiple Overlap Dose Readings in the list and clicking the Resolve button at the lower right corner of the listing page as shown next page.
Box to tick to Resolve a selected Overlap Dose Reading
.png)
After selecting [Yes] when prompted for confirmation the selected Overlap Dose Reading is filtered out of the list. Resolved Overlap Readings can still be viewed, or the Resolved flag removed, by using the Show All or the Resolved feature options displayed at the lower left corner of the Overlapping Dose Readings list page;
.png)
Resolved Overlap Reading Visibility Options
Multiple Overlapping Dose Readings can be Resolved (with caution!) by selecting multiple rows in the Overlap Dose Readings list and using the bulk [Resolve] item (lower right corner of the listing page).
A prompt is shown for confirmation indicating the number of Overlap Dose Readings that have been selected to be Resolved as shown below;
Six Duplicate Dose Readings have been selected
.png)
Bulk Allow Confirmation prompt
Allowing Duplicate Dose Readings will not change anything related to the Dose Reading itself and has no implications for the Dose Reading other than hiding the duplication record from the default view in the Duplicate Dose Readings list as a basic Historion Workflow concept.
In order to change the effect of the Duplicate Dose Reading in reporting areas of Historion the Dose Reading will need its classification details changed as described under Dose Reading Classification.
Deleting Duplicate Dose Readings
s
There are very limited circumstances under which any Dose Readings should be deleted. It is always preferable to seek a replacement Dose Report from the Provider, remove the imported version and import the corrected Dose Report as an official, legal Provider originated record.
The following circumstances would seem to justify deleting Duplicate Dose Readings, however in each case this is a local decision and policy of the Historion users and their management.
These are not recommendations but scenarios in which removing duplicates may be understandable;
A data migration exercise, from a legacy system involving large numbers of scripted Dose Reading transactions (added to the Historion database by an IT Department or a contractor) has resulted in data duplication that is identified and can be conveniently managed in the Duplicates Data Quality page.
In this instance there may be no Original Dose Reports available and data cleansing and duplicate removal is desirable and is done as a once-off step, as part of the data migration.
A Duplicate Dose Reading which is present and determined to be a Data Quality or other record error and the Provider of the result may be no longer available or unwilling to change and reissue the Dose Report as a new version, to address the Duplicate Dose Reading result.
The Historion User’s organisation has decided to, and takes complete responsibility for, (and has documented the fact of) removing the Duplicate Dose Reading.
A comment recording the removal of the Duplicate Dose Reading is made in the Notes field of the applicable Dose Report, explaining the difference between the Stored Original and the Dose Readings available for reporting that are still contained in the transacted Dose Report’s Readings appearing as rows in Historion. A File Note is registered against the applicable Wearer’s details explaining the circumstances and any record action(s) taken.
A Legacy Body or Legacy Extremity Dose Report Type has been imported from a manually authored (typed) Dose Report and the User is the author of and is directly responsible for generating the content of the Dose Report produced.
An external Provider is not involved or represented. Duplicated Dose Readings are present due to human error and removal is permissible and more convenient than generating a new Dose Report version.
In this case the new Dose Report version, to replace the one with duplicates is still preferable, so that stored Original Dose Report content matches Dose Readings retained for reporting.There are very limited circumstances under which any Dose Readings should be deleted. It is always preferable to seek a replacement Dose Report from the Provider, remove the imported version and import the corrected Dose Report as an official, legal Provider originated record.
The following circumstances would seem to justify deleting Duplicate Dose Readings, however in each case this is a local decision and policy of the Historion users and their management.
These are not recommendations but scenarios in which removing duplicates may be understandable;
A data migration exercise, from a legacy system involving large numbers of scripted Dose Reading transactions (added to the Historion database by an IT Department or a contractor) has resulted in data duplication that is identified and can be conveniently managed in the Duplicates Data Quality page.
In this instance there may be no Original Dose Reports available and data cleansing and duplicate removal is desirable and is done as a once-off step, as part of the data migration.
A Duplicate Dose Reading which is present and determined to be a Data Quality or other record error and the Provider of the result may be no longer available or unwilling to change and reissue the Dose Report as a new version, to address the Duplicate Dose Reading result.
The Historion User’s organisation has decided to, and takes complete responsibility for, (and has documented the fact of) removing the Duplicate Dose Reading.
A comment recording the removal of the Duplicate Dose Reading is made in the Notes field of the applicable Dose Report, explaining the difference between the Stored Original and the Dose Readings available for reporting that are still contained in the transacted Dose Report’s Readings appearing as rows in Historion. A File Note is registered against the applicable Wearer’s details explaining the circumstances and any record action(s) taken.
A Legacy Body or Legacy Extremity Dose Report Type has been imported from a manually authored (typed) Dose Report and the User is the author of and is directly responsible for generating the content of the Dose Report produced.
An external Provider is not involved or represented. Duplicated Dose Readings are present due to human error and removal is permissible and more convenient than generating a new Dose Report version.
In this case the new Dose Report version, to replace the one with duplicates is still preferable, so that stored Original Dose Report content matches Dose Readings retained for reporting.
Be certain to only delete one Dose Reading Duplicate at a time for any Duplicate Group. The Groups include Dose Readings that are to be kept; the whole set it shown. Any accidental removal will require deleting and reimporting the parent Dose Report and starting again with Duplicate detection and removal.
To delete a Duplicate Dose Reading only select the row representing the reading to be removed then click the Delete icon at the lower left corner or the Duplicates Listing Page or press <Delete>;
.png)
Delete Duplicate Reading(s)
A confirmation prompt requires a Yes response to continue with the Duplicate Dose Reading delete and a dialogue is displayed after the delete, showing the number of Duplicate Dose Readings remaining;
.png)
.png)
After deleting Duplicate Dose Reading(s) Historion prompts for a rescan against the Dose Readings that remain in the database;
.png)
Re-scanning after deleting a Duplicate Dose Reading is recommended. Rescanning recreates the Duplicate Groups and removes the IsDuplicate flags from Dose Readings that are no longer in a state of duplication as a result of the deleting one or more Dose Readings in a duplicate set.
.png)
At any stage after rescanning for Duplicate Dose Readings Historion will provide an indication on completion as to the number of Duplicate Dose Readings retained and how many have been flagged as Allowed Duplicate Dose Readings.
Ignore
Ignoring Duplicate Dose Readings is not recommended. Ignoring Duplicate Dose Readings means not managing or reclassifying duplications at all and this exposes the organisation to data related risk and problems as described under Risks Associated with Poor Data Quality.
Even zero value, less than detectable or less than minimum reportable dose (MRD) Duplicate Dose Readings should be investigated as they may represent assignment of the Dose Reading to the wrong Wearer, the wrong wearing period or other data recording or reporting failure.
|