Library of clinical studies
6.2.2. Library of clinical studies
When the purpose of a nuclear medicine procedure is to provide quantified physiological parameters, different software should produce the same and reproducible results within a defined margin of error. The same is true of the same software used by different users. A particular software package that offers fully automatic processing (without operator intervention) must produce identical results for the same clinical study. In order to test the clinical software, a library or database of clinical studies is required, one that will allow the studies to be processed with different clinical software or versions of software, their results compared and a reference set of results obtained for each study. This bypasses the data acquisition part of the clinical procedure and When the purpose of a nuclear medicine procedure is to provide quantified physiological parameters, different software should produce the same and reproducible results within a defined margin of error. The same is true of the same software used by different users. A particular software package that offers fully automatic processing (without operator intervention) must produce identical results for the same clinical study. In order to test the clinical software, a library or database of clinical studies is required, one that will allow the studies to be processed with different clinical software or versions of software, their results compared and a reference set of results obtained for each study. This bypasses the data acquisition part of the clinical procedure and
An example is the gated blood pool study analysed with clinical software that determines the left ventricular ejection fraction. The user establishes the ejection fraction value as determined by the clinical software installed on their system and compares that value to the value expected by the same version of software or by different software. Reproducibility is an important check. It should be noted that the ejection fraction might vary up to ±50% between different clinical software packages. Modification of a software package may produce a change in ejection fraction values. The user must always be aware of possible differences or changes in values and must always establish and/or confirm the normal range of values to be adopted. Several levels of clinical studies can be considered for different purposes.
6.2.2.1. Sample clinical data from the manufacturer or supplier of software The manufacturer or software supplier must provide sample clinical data
with reference results for each clinical software package. These studies are required to check that the clinical software has been installed properly and that it is being used properly. The sample of clinical data must include at least one normal and one abnormal study. When an updated version of software is supplied, a new set of reference results for the same sample of clinical data must be provided.
6.2.2.2. Reference clinical data It is recommended that a library of studies with reference data be
established for each clinical procedure in regular use. These serve as an extensive set of reference studies that are available for the (re-)evaluation of the software. The studies may be clinical studies provided by other departments, or they may be the results of clinical studies collected from one’s own department. Normal and abnormal studies must be included.
A set of studies could be obtained through a computer users’ group or through others using the same clinical software. Pooling studies may help with the set-up of normal and abnormal reference results, when the studies are clinically validated according to strict criteria characterizing normal and pathological conditions. However, pooling of data also requires strict adherence to standard procedural methods.
6.2.2.3. International databases Validated clinical databases of studies, such as those promoted under the
COST B2 program, are valuable for comparing software results against known data. The advantage of such databases is that they have been tested on a wide basis in a number of departments and with different software packages, which may include the one the user is using or is about to use.
Such studies will most likely be in a standard file format, such as Interfile or DICOM. Before using such studies, the user must be able to ‘read’ this format and be able to convert it into the native file format of their own computer. The user is cautioned though that conversion may not be effortless and a study file header may require some editing.
6.2.2.4. Database results that define normal ranges Some clinical software packages include a database of normal results that
have been developed at one or more institutes. Care must be exercised in using these databases.
It is necessary to be certain that the results from the clinical software conform to those expected from the software. This can be checked by processing a sample set of ‘normal’ studies used for the database. This can only partially test specificity but gives no indication of sensitivity. A sample set of such data should be supplied with the clinical software.
It is necessary to know details of the patient population that contributed to the normal database and to determine whether the patients examined in one’s own department are matched to the database population (e.g. in age, size, weight). Details of the data population must be supplied.
If the decision is taken to use the database of normal results, care is required to use the same clinical procedure (e.g. for SPECT myocardial perfusion studies, a database supplied for supine patient positioning cannot be used with prone patient positioning).
6.2.2.5. Paediatric databases Quantifying image data acquired from children may be difficult and may
even fail, for example, owing to small ROIs, small organ size that affects automatic ROI edge detection and patient movement. The user is cautioned in using clinical software designed for adults for processing paediatric studies. If a large percentage of patients are paediatric, then special efforts must be made to obtain software that has also been tested on a paediatric population. This area needs more attention.
6.2.2.6. Mathematical simulations Data from mathematical simulations are data of known form and data on
statistical noise distribution generated by a computer. Such data are useful in verifying specific algorithms because the exact result is known. For example, a simulation based on the physiological model of the left ventricle can be helpful for testing specific algorithms of the cardiac software that determines the ejection fraction. Simulated data can be used in conjunction with, but not to replace, actual in vivo clinical data.