Statistical Issues in Measuring Agreement

Huom! Aika ja paikka muuttunut.

Suomen Biostatistiikan Seura järjestää iltapäivätilaisuuden aiheesta Statistical Issues in Measurign Agreement. Luennoijana on professori Bikas Sinha, Applied Statistics Division, Indian Statistical Institute, Kolkata, India.

Tilaisuus on maksuton, mutta osallistujia pyydetään ilmoittautumaan seuran sihteerille (sihteeri@biostatistiikanseura.org) 20.8. mennessä.

Ohjelma

12.15–13.45 luento 1
13.45–14.15 kahvitarjoilu
14.15–16.00 luento 2

Abstract

Measurements of agreement are of great importance for assessing the acceptability of new or generic process, methodology, and formulation in many fields of laboratory performance, instrument or assay validation, method comparisons, statistical process control, goodness of fit, and individual bioequivalence. There are numerous examples that illustrate these situations: the agreement of laboratory measurements collected in various laboratories, the agreement of a newly developed method with gold standard method, the agreement of manufacturing process measurements with specifications, the agreement of observed values with predicted values, and the agreement in bioavailability of a new or generic formulation with a commonly used formulation. By the way, measuring agreement has been used very often to designate the level of agreement between different data-generating sources referred to as raters. A rater could be a medical laboratory, a clinical chemist, a psychologist, a radiologist, a clinician, a nurse, a psychiatric classification system, or a general measurement instrument.

One of the most popular indices of agreement was originally presented by Cohen (1960), namely Cohen's Kappa statistic , as a reliability index for measuring agreement between two raters employing nominal scales. Later, in 1968, Cohen extended the original kappa statistic by presenting the weighted kappa which assigns unequal scores to such pairs. Moreover, in the work of Landis and Koch (1977), it was found that weighted kappa is appropriate for measuring agreement when the categories of response are ordinal.

When the measurements are taken in a continuous scale, a natural way to study agreement between two rates is to examine the extent of Pearsonian Correlation (rho) between the two sets of paired data generated in the process. Lawrence Lin and his co-authors, in a series of papers, examined this aspect of agreement and proposed a number of alternative procedures for assessing the extent of agreement.

Extensions have also been made to allow for more than two raters (Posner et al., 1990; Lehmann et al., 1995), more than two possible ratings (Donner and Eliasziw, 1992; Lau, 1993), ordinal data (Fleiss, 1971; Fleiss and Cohen, 1973; Banerjee et al., 1999), and Bayesian analysis (Basu et al., 2000). In addition, many other applications of kappa statistic in a variety of different contexts can be found recently in Pinfold et al. (2000), Barnhart and Williamson (2002), Blair et al. (2002), Lin et al. (2002), Washington et al. (2003), Glenn et al. (2005), and Ruamviboonsuk et al. (2005, 2006).

We propose to discuss both qualitative and quantitative aspects of data while presenting the results on statistical assessment of agreement, a topic which has fascinated applied statisticians for a long time.