Hospital factors associated with clinical data quality

  

Health Policy 91 (2009) 321–326

Contents lists available at

  

Health Policy

  0168-8510/$ – see front matter © 2009 Elsevier Ireland Ltd. All rights reserved. doi:

  Motivated by the desire to characterize true case mix differences, many countries have examined the accuracy of diagnostic codes and the issue is important to payors and hospitals alike n general, there are two methods for examining quality of clinical data: clinical data audits and statistical analyses, though these methods are often used in tandem data audits provide a high standard against which to measure accuracy and completeness of

  system, including disease surveillance area variations research combination of factors that con- tribute to inaccurate and incomplete reporting of chronic conditions is complex; whereas case mix-based hospital payment systems embed incentives to report payment- increasing comorbidities, the role and significance of other factors are not well understood.

  E-mail addresses: (O. Steinum).

  ∗ Corresponding author. Tel.: +1 603 653 0817.

  All significant clinical events during a patients’ hos- pitalization are assumed to be recorded in the patient record. Since chronic conditions affect the evaluation, treat- ment and possible clinical outcomes of patients, accurate reporting of chronic diseases into the patient record is expected. After discharge from hospital, the clinical infor- mation abstracted from the patient record is used for an array of purposes and is vital to many aspects of the health

  © 2009 Elsevier Ireland Ltd. All rights reserved.

  fying incompletely reported clinical data. Using these results, coding quality initiatives can be focused in a directed manner.

  Conclusions: Longitudinally analyzing chronically ill patients is a novel approach to identi-

  Patients discharged from community or small hospitals, discharged alive, or transferred to another acute inpatient hospital tend to have less complete comorbidity reporting. For some chronic diseases, very old age affects chronic disease reporting.

  Results: There are a multitude of factors associated with incomplete clinical data reporting.

  repeatedly hospitalized over a determined period and identifies characteristics associated with accurate and complete reporting of chronic conditions. These methods leverage the high prevalence of chronic conditions amongst patients with multiple hospitalizations. The study is based on retrospective analysis of longitudinal hospital discharge data from a cohort of Ontario (Canada) patients.

  Methods: In a two-step process, the method links hospitalizations of patients who are

  In some countries, the reported magnitude of comorbidity inaccuracy and incompleteness is compelling. Beyond incentives provided in payment systems, the role and significance of other factors that contribute to inaccurate and incomplete reporting of chronic conditions is not well understood. A complementary approach that identifies factors associated with inaccurate and incomplete data is proposed.

  DiaQualos AB, Jordfall 420, Uddevalla SE-45197, Sweden a r t i c l e i n f o Keywords: Clinical coding Data quality DRG Hospital payment a b s t r a c t Objectives: As chronic conditions affect the evaluation, treatment, and possible clinical out- comes of patients, accurate reporting of chronic diseases into the patient record is expected.

  The Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth College, Hanover, NH 03766, USA b

   a

  , Olafr Steinum ^,

  Hospital factors associated with clinical data quality Jason M. Sutherland

  j o u r n a l h o m e p a g e :

1. Introduction

1 Tel.: +46 707 698 545.

  322 J.M. Sutherland, O. Steinum / Health Policy 91 (2009) 321–326

  originally abstracted data. Clinical data audits consist of identifying the frame of hospitals and patients’ charts to sample, selecting a sample of hospitals and charts, reabstracting sampled charts and then analyzing discrep- ancies (between the originally abstracted information and the reabstracted information). Clinical data audits have been associated with discouraging ‘upcoding’ and should be routine when case mix adjusted activity forms a basis of hospital payment data audits are time-consuming, expensive to conduct and are limited by their cross-sectional nature. Conversely, statistical analysis requires only that the clinical data be aggregated such that analyses can be applied to searching for non-random variation in case mix or payment amounts. Statistical analysis is less costly than clinical data audits and can be partially automated; however, the identification of unusual patterns of case mix does not necessarily confer a status of poor clinical data quality within a hospital and clinical data audits are still required to confirm results.

  International reabstraction studies reveal that the accu- racy and completeness of clinical data are variable within and between countries some countries, the mag- nitude of comorbidity inaccuracy and incompleteness is compelling; clinical data audit results in Canada have revealed that in excess of 20% of comorbidities are coded inaccurately or incompletely Ontario is Canada’s most populous province, home to over 12 million residents and over 100 acute inpatient hospitals. Ontario is a single payor system, in which the Ontario Min- istry of Health and Long-Term Care (MOHLTC) directly, and indirectly, funds acute inpatient hospitals. In this setting, clinical data is used for a variety of purposes, such as public hospital reporting targeted hospital funding initiatives. In a relatively minor role (relative to the amount of funding received by hospitals), case mix adjusted (weighted) hos- pitalizations form the basis of the integrated population- based allocation (IPBA) hospital funding formula

  In Ontario, patient hospitalizations are case mix adjusted using the case mix groups plus (CMG+) system, similar to diagnosis-related groups (DRG). In the CMG+ case mix system, based on abstracted diagnoses and procedures, each patient is assigned to a unique CMG group. Within CMG, there are five comorbidity levels, each corresponding to comorbidities of increasing severity beyond the main diagnosis. Each patient assigned to the same CMG (and comorbidity level) is assigned the same cost weight which, ostensibly, describes the costliness of the patient’s care rel- ative to that of all patients (a weighted case). In a hospital payment system based on case mix weighted episodes, such as Medicare’s DRG, each patient’s cost weight corre- sponds to a monetary value. Consequently, inaccurate and incomplete diagnosis reporting affects comorbidity level and, potentially, CMG case mix-based payment.

  Due to the importance of accurate clinical data and the limited effectiveness of existing methods to evaluate it, a complementary approach that identifies factors associated with inaccurate and incomplete data and whose results can be applied to improving the quality of clinical data links hospitalizations of patients who are repeatedly hos- pitalized over a determined period. Secondly, multivariate analysis of comorbidity reporting in this subset of patients’ hospitalizations identifies characteristics associated with accurate and complete reporting of chronic conditions, including examining the relationship between comorbid- ity reporting and hospital case mix (payment) measures. This paper aims to fill the gap in available methodologies by leveraging the high prevalence of chronic conditions amongst patients with multiple hospitalizations. The study is based on analysis of retrospective longitudinal hospital discharge data of a cohort of Ontario patients. The method is applied to several conditions to draw attention to hos- pital factors associated with chronic disease reporting and the inference is that incomplete reporting extends beyond those patients with multiple hospitalizations.

  2. Data

  All acute inpatient hospitals in Ontario are required to submit an electronic discharge summary to the discharge abstract database (DAD). There are 1.149 million acute inpa- tient discharges from over 120 Ontario hospitals for the period between April 1, 2005 and March 31, 2006 available for analysis. All potential patient and hospital identifiers are encrypted. In creating the cohort of patients for analysis, only those patients for whom there were at least four acute inpatient hospitalizations during the year and for which there exists a valid (anonymized) identifier are included. In this cohort of patients, there are 7986 patients represent- ing 49,352 acute inpatient discharges (in the population of discharges, 17.8% of patients had more than one admission).

  Diagnostic and procedural classification systems are standardized across all Ontario hospitals; diagnoses are coded in International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10-CA, Canada) while procedures are coded in Canadian Classifi- cation of Interventions (CCI). Each discharge summary can accommodate up to 25 diagnoses and 25 procedures. The average number of diagnoses on discharge summaries (last discharge) is 4.8 (see Guidelines informing clinical abstraction are uniform across all hospitals; for example, diabetes mellitus should always be recorded even if it does not affect treatment during the hospitalization.

  Chronic diseases are identified by the presence of ICD- 10 codes. Hospitals are identified by a unique identifier. Hospital size is stratified by: teaching hospital, community hospital, and small hospital, as per the IPBA hospital funding formula. Each hospitalization is labeled a medical

  Table 1 Summary of characteristics of cohort of patients with at least four dis- charges in 1 year, representing 3.4% of all hospital discharges in Ontario.

  Characteristic Value Number of patients 7,916 Number of discharges 49,352 Female (%)

  49.5 On last discharge Discharge alive (%)

  85.9 Age (years) 57.6 (S.D. 23.5) Number of diagnoses

  4.8

  J.M. Sutherland, O. Steinum / Health Policy 91 (2009) 321–326 323 or surgical hospitalization according to the presence of surgical codes and surgical case mix group assignment. Indicator variables representing inpatient death, transfer out of hospital and transfer into hospital are created. Patient age is stratified into three groups: less than 65 years, 65–80, and greater than 80.

  1

  abstracted in every hospitalization when it is noted in the patient chart.

  Analysis finds that DM is inconsistently coded for many patients’ hospitalizations, as 59.11% of hospitalizations include a diagnosis of DM. One patient’s hospitalizations are summarized in an example. DM was recorded for 2 of 5 hospitalizations. Hospital B does not record DM even though the condition was recorded on a previous 8- day hospitalization in the same hospital. Hospital C, for a 5-day cardiac admission, does not record DM.

  The results of the multivariate logistic regression analy- ses associating hospital factors to DM diagnoses are shown in several factors are identified as being statis- tically significantly related to the probability to reporting DM. DM is less likely to be reported consistently for hospi- tals’ oldest patients (estimated odds ratio, OR = 0.83). The results also show that DM is less likely to be reported for surgical hospitalizations (relative to medical hospi-

  Table 2 Hospitalizations and diabetes recording; diabetes is recorded in 2 of the patients 5 hospitalizations.

  

Admission Hospital CMG+ from main diagnosis or principal procedure Diabetes diagnoses Length of stay

  1 A 464 partial excision/destruction of prostate, closed approach None

  2 B 149 symptom/sign of respiratory system Type 2 DM; no complications

  4. Results

  8

  3 C 195 heart failure with cardiac catheter None

  6

  4 A 139 chronic obstructive pulmonary disease DM; not otherwise specified with polyneuropathy

  6

  5 B 139 chronic obstructive pulmonary disease None

  4.1. Diabetes mellitus (DM) In the cohort of chronically ill patients, the ICD-10 codes used to identify patients with diabetes are: E10–E14. There are 2113 patients with four or more hospitalizations, at least one of which includes a diagnosis of diabetes mellitus. The analysis assumes that a single hospitalization with a diag- nosis of DM infers a positive diagnosis of DM for the patient (this assumption is discussed more later). Canadian coding standards that diabetes mellitus should be

  0.61 are re-assigned. Differences between the original case mix payment amount and the newly assigned case mix payment amount are summarized.

  3. Methods The methods outlined are used to identify hospital characteristics associated with inaccurate or incomplete chronic disease reporting. Hospitalizations for the same patient are linked such that all diagnoses and procedures for the same patient are observable across hospitaliza- tions. The basis for linking hospitalizations is a universal encrypted personal identifier. Clinical data audits have shown these fields are completely and very accurately reported For each hospitalization, indicator vari-

  0.36 80 plus

  ables are derived, representing presence or absence of select chronic conditions in each of the repeated hos- pitalizations. Three chronic conditions are examined in detail; diabetes mellitus, congestive heart failure (CHF), and chronic liver disease. These conditions are selected for several reasons: there is specificity in the coding sys- tem (ICD-10) to unambiguously identify these conditions, the conditions do not resolve, nor are they likely to be newly diagnosed over the course of the repeated hospital- izations. In addition, these conditions are chosen to contrast the differences in completeness of chronic disease report- ing between those conditions that affect payment weights (such as CHF) and those that do not (diabetes mellitus) in the CMG case mix system.

  For each condition, a multivariate logistic regression model is used to associate hospital level factors to the probability of accurate comorbidity reporting during a hospitalization. The dependent variable is presence or absence of the chronic condition. The independent vari- ables are: hospital, hospital size stratum, patient age stratum, medical or surgical hospitalization, transfers in, transfers out and interaction effects. The statistical model is fit using generalized estimating equations (GEE) to con- trol for patient clustering effects (multiple hospitalizations for each patient). All effects are included in the model (even if they are not statistically significant).

  Case mix payment effects are ascertained by imputing chronic conditions not reported on select hospitalizations. For example, if presence of CHF is documented in many hospitalizations, except does not appear in one, existence of the condition is imputed to the errant hospitalization. Using the imputed comorbidity, the comorbidity level is recalculated and the cost weight and monetary equivalent

  Results of the multivariate logistic analyses of the probability of reporting DM for each hospitalization in a cohort of 2113 patients with at least four hospitalizations and for whom DM is assumed.

  Variable Estimated odds ratio P-value Age <65 years 1.00 – 65–79

  0.95

  0.83

  1.03

  0.01 Medical hospitalization 1.00 – Surgical 0.71 <0.01 Teaching hospital 1.00 – Community 0.65 <0.01 Small

  0.89

  0.62 Discharged alive

  0.82

  0.11 Transferred into hospital

  1.19

  0.08 Transferred out of hospital

  5

  324 J.M. Sutherland, O. Steinum / Health Policy 91 (2009) 321–326

Hospitalizations and CHF reporting for a patient hospitalized six times during the year at the same hospital; four hospitalizations do not include a diagnosis

of CHF. The impact of under coding CHF on cost weights and hospital reimbursement is $13,610.

Admission CMG+ from main diagnosis or principal procedure Original cost weight Comorbidity adjusted cost weight Cost difference

1 196 heart failure without cardiac catheter 0.9561 NA NA 2 196 heart failure without cardiac catheter 0.9561 NA NA 3 248 severe enteritis 0.8746 1.4318 $3984 4 249 enteritis 0.5544 0.7985 $1745 5 653 septicemia due to S. aureus 1.3288 1.8739 $3898 6 248 severe enteritis 0.8746 1.4318 $3984 talizations) (OR = 0.71). Community hospitals are much less likely to report DM (relative to teaching hospitals) (OR = 0.65). Discharge and transfer status are not found to affect DM reporting. Comorbid diagnosis of DM does not affect case mix measurement in CMG+ for respira- tory or circulatory conditions (representing the majority hospitalizations) although DM does affect case mix for less- frequently occurring conditions (such as nervous system and eye conditions).

  Table 5 Results of the multivariate logistic analyses of the probability of reporting CHF for each hospitalization in a cohort of 2044 patients with at least four hospitalizations and for whom CHF is assumed.

  Variable Estimated odds ratio P-value Age <65 years 1.00 – 65–79 1.27 <0.01 80 plus 1.41 <0.01 Medical hospitalization 1.00 – Surgical 1.45 <0.01 Teaching hospital 1.00 – Community

4.2. Congestive heart failure (CHF)

  Hospital and patient level factors associated with CHF coding are shown in multivariate results show that older patients are more likely to have CHF reported during more of their hospitalizations (OR = 1.41). Surgi- cal patients are more likely than medical patients to have CHF reported consistently (OR = 1.45). Patients discharged alive are significantly less likely to have CHF reported (OR = 0.54). Also, patients transferred to another acute inpa-

  0.54 Medical hospitalization 1.00 – Surgical

  0.89

  0.98

  0.07 Transferred out of hospital

  1.70

  0.08 Transferred into hospital

  0.53

  0.14 Small 0.24 <0.01 Discharged alive

  0.64

  0.29 Teaching hospital 1.00 – Community

  0.81

  1.19

  Canadian coding standards not require CHF to be reported unless the condition affects treatment dur- ing the hospitalization. However, unlike DM, CHF diagnosis does affect case mix payment weights; reporting CHF increases the comorbidity level (and cost weight) provid- ing an incentive to report CHF when the condition exists. It is assumed that a cost weight of 1.0000 represents a value of $7150 (Canadian dollars). The original case mix payment amount is compared to the payment amounts hypotheti- cally accrued if CHF had been reported (shown in If hospitalization 3 had included CHF as a comorbidity, the value of the hospitalization would have increased by $3984. Summing the four hospitalizations that do not include CHF as a comorbidity, the difference between the actual and hypothetical payment amount is $13,610. The difference in payment levels suggests that maximizing case mix pay- ment is not the lone factor in CHF reporting.

  0.29 80 plus

  There are 2044 unique patients with at least four hos- pitalizations and at least one of the patients’ discharges includes a diagnosis of CHF. The prevalence of CHF diag- nosis amongst these hospitalizations is 44.45%. The ICD-10 codes used to identify patients with CHF are I50–I50.9. A common pattern in CHF reporting is shown in

  1.00 – 65–79

  4.3. Chronic liver disease There are 372 unique patients diagnosed with chronic liver disease with at least four hospitalizations, of which at least one hospitalization includes a diagnosis of chronic liver failure. Of these hospitalizations, 54.12% include the diagnosis of chronic liver disease. The ICD-10 codes used to identify these patients are: B18.0–B18.2, B18.8, B18.9, K70.2, K70.3, K71.3–K71.5, K71.7, K72.1, K73–K73.2, K73.8, K73.9, K74–K74.6 and Q44.7. Amongst these ICD-10 codes listed, several affect case mix payment by increasing the Table 6 Results of the multivariate logistic analyses of the probability of reporting chronic liver failure for each hospitalization in a cohort of 372 patients with at least four hospitalizations and for whom chronic liver failure is assumed. Variable Estimated odds ratio P-value Age <65 years

  0.35 Transferred out of hospital 0.82 <0.01 to those whose entire episode is in the same hospital (OR = 0.82).

  0.90

  0.38 Discharged alive 0.54 <0.01 Transferred into hospital

  1.30

  0.34 Small

  0.90

  All hospitalizations are at the same hospital. The first two hospitalizations are for cardiac conditions without surgi- cal treatment (the CMG+ is heart failure without cardiac catheter). Subsequent hospitalizations are for: enterocoli- tis, diarrhea and septicaemia. Hospitalizations 3–6 do not list CHF as a comorbidity.

  0.84

  J.M. Sutherland, O. Steinum / Health Policy 91 (2009) 321–326 325 comorbidity level. Chronic liver disease is required to be reported only when the diagnosis affects treatment.

  As shown in small hospitals are much less

  likely to report chronic liver disease (OR = 0.24). Although the relationship is weak, those patients discharged alive are less likely to have had the diagnosis reported (OR = 0.53), though transfer patients admitted are more likely (OR = 1.70). The inability to detect smaller effects may be due to a lack of statistical power owing to the limited sample size.

5. Discussion

  The importance of high quality clinical data is difficult to overemphasize given its myriad of high profile uses. Although significant effort is put toward monitoring and improving the quality of clinical data, there are limitations to the methods currently in use. Longitudinally analyzing chronically ill patients is a novel approach to identifying problematic clinical data. The results show that improving hospital case mix payment is not the only factor related to accurate and complete reporting of chronic conditions. The strategy and results shown in this analysis identify other factors, such as transfer status and mortality, that play an important role in chronic disease reporting. If the results generalize beyond the chronically ill, hospitals have a strong incentive to apply these results; more accurate clinical data often increases case mix payment. In the long term, increasingly accurate clinical data will, in turn, lead to case mix payment systems less affected by measurement error might reasonably be expected to result in more confident use of clinical data for hospital level public reporting.

  Inferring from the results that hospital coding practices can contribute to incomplete clinical data, hospital level coding quality initiatives can be focused in a directed man- ner. For instance, community and small hospitals are less likely than large teaching hospitals to completely report comorbidities, bringing clinical data quality issues to the forefront of discussions with small and community hos- pitals. The same is true in instances where patients are discharged alive or transferred out. Surgical cases imply a behavioral response to case mix payment; when comorbid- ity reporting affects payment amounts for surgical patients, comorbidities tend to be well reported (relative to medical cases).

  Disease management, surveillance or epidemiologic research applications that employ cross-sectional analyses of hospital discharge data will under report disease burden. One possible strategy to mitigate the effect of under report- ing bias is to develop a look-back period (of undetermined duration), during which hospitalizations preceding the index hospitalization(s) are similarly examined to detect presence of comorbidities. This point highlights an impor- tant limitation of this study: missing comorbidities are assumed to be coding omissions or inaccuracies. In fact, incidence of comorbidity codes may indicate over coding or the possibility of ‘false positives,’ which has not been examined. However, the logistic models discussed may also be used to isolate factors that affect the incidence of tors related to workload and training programs of hospital abstractors may affect disease reporting but are not con- sidered in these analyses (since they are unavailable).

  These methods underscore an important issue: in a hospital reporting system, who bears responsibil- ity for completeness and accuracy of reporting chronic conditions? Within the context of a hospital payment applications considered in this analysis, hospitals have the financial incentive to report only those conditions that affect payment. However, there are typically many other users of clinical data. In other countries where acute hospitalizations are used as one input into regional population-based funding models, which are susceptible to these reporting biases, the incentives for improving clinical data quality rest with a regional coalition of stakeholders.

  Consider the setting of many Canadian provinces in which clinical data does not form a basis for hospital pay- ment; does some portions of the responsibility for accurate and complete clinical data lie with non-hospital users of the abstracted information (often a government agency)? For example, if a patient is diagnosed with Type 2 DM and a hospitalization 1 year later omits a diagnosis of diabetes, should the agency either ‘query’ the hospital or ‘impute’ a diagnosis to improve completeness? If clinical data qual- ity is a shared responsibility between users, a formalized feedback mechanism to improve on clinical data quality would need to be developed. In a possible model, electronic abstracts could be rejected and returned to the hospital for correction and resubmission. In another model, elec- tronic messaging could query previously existing abstracts to identify chronic conditions for inclusion on the patient chart. In the future, the methods described may be less applicable when an electronic medical record is broadly available. In these environments, medical history will cross between hospital systems, providers, and sectors of the health care system, resulting in accurate and complete dis- ease reporting.

  Already, significant effort is invested in monitoring and improving the quality of clinical data in many countries. However, there are limitations to the reabstraction and sta- tistical methods currently in use and a dearth of new meth- ods being proposed. The methods proposed in this study attempt to address some of these shortcomings. Nonethe- less, there are still opportunities to achieve improvements in clinical data quality by continued investigation into clin- ical and data-intensive methods and techniques.

  Acknowledgement

  The authors would like to acknowledge the Ontario Min- istry of Health and Long-Term Care for providing access to the anonymous clinical data and for supporting data quality improvement initiatives.

  References [1] Penberthy L, McClish D, Pugh A, Smith W, Manning C, Retchin S. Using Hospital Discharge Files to Enhance Cancer Surveillance. American

  Journal of Epidemiology 2003;158(1):27–34. [2] Bright RA, Avorn J, Everitt DE. Medicaid data as a resource for epi- demiologic studies: strengths and limitations. Journal of Clinical

  Epidemiology 1989;42:937–45.

  326 J.M. Sutherland, O. Steinum / Health Policy 91 (2009) 321–326 [3] Busse R, Schreyögg J, Smith PC. Editorial: hospital case payment systems in Europe. Health Care Management Science 2006;9(3):

  211–3. [4] Shwartz M, Peköz EA, Ash AS, Posner MA, Restuccia JD, Iezzoni LI. Do variations in disease prevalence limit the usefulness of population-based hospitalization rates for studying variations in hos- pital admissions? Medical Care 2005;43:4–11.

  [5] Audit Commission. PbR data assurance framework 2007/08. Findings from the first year of the national clinical coding audit programme.

  Health National Report August 2008. London: Audit Commission; 2008. [6] Silverman E, Skinner J. Upcoding across hospital ownership. Journal of Health Economics 2004;23(2):369–89. [7] Hsia DC, Ahern CA, Ritchie BP, Moscoe LM, Krushat WM. Medi- care reimbursement accuracy under the prospective payment system, 1985 to 1988. Journal of the American Medical Association 1992;268:896–9. [8] Bryngelsson S, Bolin P, Karlsson M. Cost outlier, why? Proceedings of the PCSI 23rd Working Conference. 2007. [9] Rosenberg MA, Fryback DG, Katz DA. A statistical model to detect DRG upcoding. Health Services and Outcomes Research Methodology

  2000;1(3–4):233–52. [10] Rosenberg MA. A decision-theoretic method for assessing a change in the rate of non-acceptable inpatient claims. Health Services and

  Outcomes Research Methodology 2001;2:19–36. [11] Ignatova I, Edwards D. Probe samples and the minimum sum method for medicare fraud investigations. Health Services and Outcomes

  Research Methodology 2008, doi:10.1007/s10742-008-0038-7. [12] Becker D, Kessler D, McClellan M. Detecting medicare abuse. Journal of Health Economics 2005;24(1):189–210.

  [13] Canadian Institute for Health Information. Discharge abstract database data quality re-abstraction study, combined findings for 1999/2000 and 2000/2001. Ottawa, Canada: Canadian Institute for Health Information; 2003. [14] Canadian Institute for Health Information. CMG/Plx data quality re- abstraction study. Ottawa, Canada: Canadian Institute for Health

  Information; 2003. [15] Preyra C. Coding response to a case mix measurement system based on multiple diagnoses. Health Services Research 2004;39(4):

  1027–46. [16] Canadian Institute for Health Information. Coding variations in the Discharge Abstract Database (DAD) data, FY 1996–1997 to

  2000–2001. Ottawa, Canada: Canadian Institute for Health Informa- tion; 2003. [17] Canadian Institute for Health Information. Hospital report 2002, a joint initiative of the Ontario hospital association and the government of Ontario. Ottawa, Canada: Canadian Institute for Health Informa- tion; 2002. [18] Booth G, Fang J. Acute complications of diabetes. In: Hux JE, Booth GL, Slaughter PM, Laupacis A, editors. Diabetes in Ontario: an ICES prac- tice atlas. Institute for Clinical Evaluative Sciences; 2003. p. 2.21–51. [19] Ontario Joint Policy and Planning Committee. Summary Report of the Hospital Funding Committee on the Use of 2004/05 Cost and Activity

  Data, JPPC Reference Document #10-9. Toronto, Ontario: The Ontario Joint Policy and Planning Committee; February 2006. [20] Canadian Institute for Health Information. Canadian coding standard for ICD-10-CA and CCI for 2007. Ottawa, Canada: Canadian Institute for Health Information; 2007. [21] Sutherland JM, Botz CK. The effect of misclassification errors on case mix measurement. Health Policy 2006:195–202.