PRAiS v4.1

PLEASE NOTE that licensees are NOT permitted to further distribute the software nor allow the software to be used by, on behalf of or for the benefit of any third parties.

Version 4 of the PRAiS software is now available for download. We have recalibrated the PRAiS v2 [1] logistic regression model and updated the inclusion criteria and one risk factor. This is the PRAiS v4 risk model and is implemented PRAiS v4 of the Excel software. The model was calibrated on UK National Congenital Heart Disease Audit (NCHDA) data from April 2015 – March 2022.

NOTE: there is no PRAiS3 risk model – we chose to call both the risk model and the Excel software by the same version number to avoid confusion.

Product specification

This software can be used to apply the PRAiS risk model for death within 30 days following cardiac surgery in children using routinely collected audit data and produce a Variable Life Adjusted Display (VLAD) chart to facilitate routine monitoring of programme-level outcomes. 

Change History:

Changes from v4 to v4.1

• The software was incorrectly not allowing “ICD surgical” as a valid Specific Procedure. This has now been fixed so that “ICD surgical” procedures are correctly allocated and included when running the risk model. The mapping spreadsheet has also been updated to reflect the inclusion of “ICD surgical”.

Changes from v3.0.2 to v4

• Age: We now include all children under 18 years of age (instead of under 16)

• We have added a new additional risk factor (Prematurity).

• We have implemented the latest NCHDA Specific Procedure algorithm as of July 2024 (v8.5). Note, that following discussions with NICOR, there are slight some changes to allocations in terms of not using the “allowed only with” codes and using separate groupings for HLHS Hybrid and Tracheal procedures.

• Procedure types “11 – EP surgery” and “7. Hybrid” are now included within PRAiS, and are assigned a transformed procedure type of “Non-bypass” for the model.

• We have updated the allowed procedure, diagnosis and comorbidity codes in line with latest NCHDA code lists (as of July 2024) and their allocation to one of more of: broad procedure groups, broad diagnosis groups and additional risk factors. 

 Changes from v3.0.1 to v3.0.2

• fixed a typo in the specific procedure "Tricuspid Valve Repair" (from Tricupid to Tricuspid) in the PRAiS software
• Removed 2 out-of-date codes from mappings for Additional Risk factors in the “Important_Mappings_Used_For_PRAiS_v3_0_2.xls” spreadsheet.

Changes from v3.0 to v3.0.1:

• fixed a problem with rare procedure codes entered in diagnosis fields

Product update (v2.2 to v3.0.1) 

We have updated the PRAiS software to version 3.0. Main changes include:

a.         Risk factors:

i.          Age: we no longer use age bands together with continuous age. Instead we use a non-linear function of age in our model (the square root of age in years + age).

ii.          Weight: we now use a non-linear function of weight to capture the additional risk of small babies (the square root of weight in Kg + weight)

iii.         Specific procedures are now grouped into one of 16 broader procedure groups

iv.         The diagnosis codes for a record are now used to allocated that record to one of 11 broad diagnosis groups

v.         Instead of having a single “yes/no” comorbidity field, we now have four separate yes/no indicators of other health problems: “Congenital comorbidity”; “Acquired comorbidity”; “Severity of Illness indicator” and “Additional cardiac risk factor”. A child can have any combination of these additional health problems.

b.         We have updated the Case Mix Summary worksheet to reflect these new risk factors and now include the procedure groupings as well.

c.         New option on the Dashboard – we have added a new option to the Dashboard that will clear the “YourData” sheet for you, to make it easier to replace old data with new data.

d.         We now include hybrid type procedures that are used treat children with Hypoplastic Left Heart Syndrome (HLHS). Specifically this only includes records with EPCC procedure codes: 122021; 122020; 121004 OR both 121014 AND 121419 (as implemented in the NCHDA May 2016 specific procedure algorithm). You do not have to do anything different – the software will automatically include these records.

e.         We have added the NCHDA field “Optional – unique procedure ID” to the “YourData” worksheet to make it easy for you to link episodes back to your own hospital procedure records. PLEASE NOTE – this means any automatic procedures you have for extracting columns for use in this software will need to be updated 

f.          We have added “Actual number of 30-day deaths “ and  “expected number of deaths for your unit using PRAiS” to the “Prediction Interval” sheet as further potentially useful information.

Product description

The Excel file PRAiSv4 is designed to allow UK paediatric cardiac centres to routinely monitor their short term surgical outcomes using the data that they collect for the National Congenital Heart Disease Audit (NCHDA). Outcomes are partially risk adjusted using a model that estimates the risk of death within 30 days of a surgical procedure based on procedure, age, weight and a patient's recorded diagnoses and additional health problems.

Outcomes over time are displayed using a bespoke version of the Variable Life Adjusted Display (VLAD) technique which is intended to facilitate the monitoring of clinical outcomes. The VLAD chart is a plot of the difference between the expected and actual tally of deaths. The plotted trace will rise for each survival and fall for each death, the height of the fall or rise depending on the estimated probability of death for that episode. The survival of a high risk episode results in a larger rise than the survival of a low risk episode and a death of a low risk episode results in a larger fall than a death of a high risk episode. In this way, the VLAD chart can help to identify runs of outcomes that are better or worse than expected, based on partial risk adjustment. If the VLAD plot goes down over time, there have been more deaths than expected and if it climbs, there have been more survivors than expected. If it fluctuates around the horizontal, then outcomes are broadly in line with expectations. We additionally highlight episodes where there was a reintervention within 30 days as part of the VLAD chart.

Example output of a VLAD chart for 1000 hypothetical episodes is shown below. This is just an illustrative example and does not correspond to actual data.

Example of a VLAD chart. This hypothetical unit had a run of survivors in early 2011 followed by a run of deaths in the spring of 2011. There were also clusters of reinterventions in the summer and autumn. Note that this is an illustrative example and does NOT correspond to actual data.

Users can enter data directly into the software in the format it is submitted to the National Institute of Cardiovascular Outcomes (NICOR) National Congenital Heart Disease Audit (NCHDA). All pre-processing is performed as part of the software, including basic error and consistency checking.

After data are pre-processed, an estimated risk of death within 30 days following surgery is calculated for each surgical episode and a VLAD chart generated over the time period of the included data. To facilitate discussion of any deaths during the period, information on these patients is copied to a separate worksheet.

The software includes additional, optional, functionality for units to examine their recent case mix by bands of estimated risk of death and by mix of patients with different clinical features, along with a comparison to historical national data for interest. The software also includes a function to generate prediction limits for observed 30-day survival among surgical episodes.

Generating new VLADs is very easy – simply put the new data in the relevant worksheet and the software will reset all the other sheets as necessary when rerunning the tool. 

Further information can be found on the UCL research group's webpage at: https://www.ucl.ac.uk/clinical-operational-research-unit/research-domains/congenital-heart-disease-children-and-adults

Product requirements

Users need to have Microsoft Excel for Microsoft Windows.

Note that PRAiS v4 was developed and tested using Microsoft Excel for Microsoft 365 MSO (Version 2406 Build 16.0.17726.20078) in Windows 11. While we have tried to produce robust software, we cannot guarantee that this software will be bug-free in all versions of Excel or across all Windows platforms. Note that we have not tested it in earlier versions of Excel and cannot guarantee support for earlier versions. Macros must be enabled for the software to run.

The software does NOT work on Excel for Mac.

PLEASE NOTE that the software was developed assuming that data corresponding to UK national audit is available. Required data are:

• Unique patient identifier
• Date of birth
• Date of procedure
• Date of death (for patients that have died)
• Procedure type (bypass, non-bypass, catheter, hybrid (but only for HLHS, see user manual))
• Procedure codes (in EPCC short code format)
• Diagnosis codes (in EPCC short code format)
• Comorbidity codes if applicable (in EPCC short code format)
• Weight (in kg)