Step 3. Quantifying Uncertainty

7. Step 3. Quantifying Uncertainty

7.1. Introduction

7.1.1. Having identified the uncertainty sources as

• Plan to obtain the further data required

explained in Step 2 (Chapter 6), the next step is to For sources of uncertainty not adequately quantify the uncertainty arising from these

covered by existing data, either seek sources. This can be done by

additional information from the literature or • evaluating the uncertainty arising from each

standing data (certificates, equipment individual source and then combining them as

specifications etc.), or plan experiments to described in Chapter 8. Examples A1 to A3

obtain the required additional data. illustrate the use of this procedure.

Additional experiments may take the form of specific studies of a single contribution to

or uncertainty, or the usual method performance • by determining directly the combined

studies conducted to ensure representative contribution to the uncertainty on the result

variation of important factors. from some or all of these sources using

7.2.2. It is important to recognise that not all of method performance data. Examples A4 to A6

the components will make a significant represent applications of this procedure.

contribution to the combined uncertainty; indeed, In practice, a combination of these is usually

in practice it is likely that only a small number necessary and convenient.

will. Unless there is a large number of them, components that are less than one third of the

7.1.2. Whichever of these approaches is used, largest need not be evaluated in detail. A most of the information needed to evaluate the

preliminary estimate of the contribution of each uncertainty is likely to be already available from

component or combination of components to the the results of validation studies, from QA/QC

uncertainty should be made and those that are not data and from other experimental work that has

significant eliminated.

been carried out to check the performance of the method. However, data may not be available to

7.2.3. The following sections provide guidance on evaluate the uncertainty from all of the sources

the procedures to be adopted, depending on the and it may be necessary to carry out further work

data available and on the additional information as described in sections 7.10. to 7.14.

required. Section 7.3. presents requirements for the use of prior experimental study data,

7.2. Uncertainty evaluation procedure

including validation data. Section 7.4. briefly discusses evaluation of uncertainty solely from

7.2.1. The procedure used for estimating the individual sources of uncertainty. This may be overall uncertainty depends on the data available

necessary for all, or for very few of the sources about the method performance. The stages

identified, depending on the data available, and is involved in developing the procedure are

consequently also considered in later sections.

• Reconcile the information requirements

Sections 7.5. to 7.9. describe the evaluation of

with the available data

uncertainty in a range of circumstances. Section

7.5. applies when using closely matched First, the list of uncertainty sources should be

reference materials. Section 7.6. covers the use of examined to see which sources of uncertainty

collaborative study data and 7.7. the use of in- are accounted for by the available data,

house validation data. 7.8. describes special whether by explicit study of the particular

considerations for empirical methods and 7.9. contribution or by implicit variation within

covers ad-hoc methods. Methods for quantifying the course of whole-method experiments.

individual components of uncertainty, including These sources should be checked against the

experimental studies, documentary and other list prepared in Step 2 and any remaining

data, modelling, and professional judgement are sources should be listed to provide an

covered in more detail in sections 7.10. to 7.14. auditable record of which contributions to the

Section 7.15. covers the treatment of known bias uncertainty have been included.

in uncertainty estimation.

Page 16

Quantifying Uncertainty Step 3. Quantifying Uncertainty

7.3. Relevance of prior studies

7.5. Closely matched certified

7.3.1. When uncertainty estimates are based at

reference materials

least partly on prior studies of method

7.5.1. Measurements on certified reference performance, it is necessary to demonstrate the

materials are normally carried out as part of validity of applying prior study results. Typically,

method validation or re-validation, effectively this will consist of:

constituting a calibration of the whole • Demonstration that a comparable precision to

measurement procedure against a traceable that obtained previously can be achieved.

reference. Because this procedure provides information on the combined effect of many of

• Demonstration that the use of the bias data the potential sources of uncertainty, it provides obtained previously is justified, typically

very good data for the assessment of uncertainty. through determination of bias on relevant

Further details are given in section 7.7.4. reference materials (see, for example, ISO

N OTE : ISO Guide 33 [H.8] gives a useful account of

Guide 33 [H.8]), by appropriate spiking

the use of reference materials in checking

studies, or by satisfactory performance on

method performance.

relevant proficiency schemes or other laboratory intercomparisons.

7.6. Uncertainty estimation using prior

• Continued performance within statistical

collaborative method development

control as shown by regular QC sample

and validation study data

results and the implementation of effective

7.6.1. A collaborative study carried out to analytical quality assurance procedures. validate a published method, for example

7.3.2. Where the conditions above are met, and according to the AOAC/IUPAC protocol [H.9] or the method is operated within its scope and field

ISO 5725 standard [H.10], is a valuable source of of application, it is normally acceptable to apply

data to support an uncertainty estimate. The data the data from prior studies (including validation

typically include estimates of reproducibility studies) directly to uncertainty estimates in the

standard deviation, s R , for several levels of laboratory in question.

response, a linear estimate of the dependence of s R on level of response, and may include an

7.4. Evaluating uncertainty by

estimate of bias based on CRM studies. How this

quantification of individual

data can be utilised depends on the factors taken

components into account when the study was carried out.

During the ‘reconciliation’ stage indicated above

7.4.1. In some cases, particularly when little or no (section 7.2.), it is necessary to identify any method performance data is available, the most

sources of uncertainty that are not covered by the suitable procedure may be to evaluate each

collaborative study data. The sources which may uncertainty component separately.

need particular consideration are:

7.4.2. The general procedure used in combining • Sampling. Collaborative studies rarely include individual components is to prepare a detailed

a sampling step. If the method used in-house quantitative model of the experimental procedure

involves sub-sampling, or the measurand (see (cf. sections 5. and 6., especially 6.4.), assess the

Specification) is estimating a bulk property standard uncertainties associated with the

from a small sample, then the effects of individual input parameters, and combine them

sampling should be investigated and their using the law of propagation of uncertainties as

effects included.

described in Section 8. • Pre-treatment. In most studies, samples are

7.4.3. In the interests of clarity, detailed guidance homogenised, and may additionally be on the assessment of individual contributions by

stabilised, before distribution. It may be experimental and other means is deferred to

necessary to investigate and add the effects of sections 7.10. to 7.14. Examples A1 to A3 in

the particular pre-treatment procedures Appendix A provide detailed illustrations of the

applied in-house.

procedure. Extensive guidance on the application • Method bias. Method bias is often examined of this procedure is also given in the ISO Guide

prior to or during interlaboratory study, where [H.2].

possible by comparison with reference

Page 17

Quantifying Uncertainty Step 3. Quantifying Uncertainty

methods or materials. Where the bias itself, • Quantification of any uncertainties associated the uncertainty in the reference values used,

with effects incompletely accounted for in the and the precision associated with the bias

above overall performance studies. check, are all small compared to s R , no

Precision study

additional allowance need be made for bias uncertainty. Otherwise, it will be necessary to

7.7.2. The precision should be estimated as far as make additional allowances.

possible over an extended time period, and • Variation in conditions.

chosen to allow natural variation of all factors Laboratories participating in a study may tend

affecting the result. This can be obtained from towards the means of allowed ranges of

• The standard deviation of results for a typical experimental conditions, resulting in an

sample analysed several times over a period of underestimate of the range of results possible

time, using different analysts and equipment within the method definition. Where such

where possible (the results of measurements effects have been investigated and shown to

on QC check samples can provide this

be insignificant across their full permitted

information).

range, however, no further allowance is required.

• The standard deviation obtained from replicate • analyses performed on each of several

Changes in sample matrix. The uncertainty

samples.

arising from matrix compositions or levels of interferents outside the range covered by the

N OTE : Replicates should be performed at materially

study will need to be considered.

different times to obtain estimates of intermediate precision; within-batch

7.6.2. Each significant source of uncertainty not

replication provides estimates of repeatability

covered by the collaborative study data should be

only.

evaluated in the form of a standard uncertainty • From formal multi-factor experimental and combined with the reproducibility standard designs, analysed by ANOVA to provide deviation s R in the usual way (section 8.) separate variance estimates for each factor.

7.6.3. For methods operating within their defined

7.7.3. Note that precision frequently varies scope, when the reconciliation stage shows that significantly with the level of response. For all the identified sources have been included in example, the observed standard deviation often the validation study or when the contributions increases significantly and systematically with from any remaining sources such as those analyte concentration. In such cases, the discussed in section 7.6.1. have been shown to be uncertainty estimate should be adjusted to allow negligible, then the reproducibility standard for the precision applicable to the particular deviation s R , adjusted for concentration if result. Appendix E.4 gives additional guidance on necessary, may be used as the combined standard handling level-dependent contributions to uncertainty.

uncertainty.

7.6.4. The use of this procedure is shown in

Bias study

example A6 (Appendix A)

7.7.4. Overall bias is best estimated by repeated

7.7. Uncertainty estimation using in-

analysis of a relevant CRM, using the complete

house development and validation

measurement procedure. Where this is done, and

studies the bias found to be insignificant, the uncertainty

associated with the bias is simply the combination

7.7.1. In-house development and validation of the standard uncertainty on the CRM value studies consist chiefly of the determination of the

with the standard deviation associated with the method performance parameters indicated in

bias.

section 3.1.3. Uncertainty estimation from these

N OTE :

Bias estimated in this way combines bias in

parameters utilises:

laboratory performance with any bias intrinsic

• to the method in use. Special considerations The best available estimate of overall

may apply where the method in use is

precision.

empirical; see section 7.8.1.

• The best available estimate(s) of overall bias • When the reference material is only and its uncertainty.

approximately representative of the test

Page 18

Quantifying Uncertainty Step 3. Quantifying Uncertainty

materials, additional factors should be

7.7.6. Overall bias can also be estimated by the considered, including (as appropriate)

addition of analyte to a previously studied differences in composition and homogeneity;

material. The same considerations apply as for reference materials are frequently more

the study of reference materials (above). In homogeneous that test samples. Estimates

addition, the differential behaviour of added based on professional judgement should be

material and material native to the sample should used, if necessary, to assign these

be considered and due allowance made. Such an uncertainties (see section 7.14.).

allowance can be made on the basis of: • Any effects following from different

• Studies of the distribution of the bias concentrations of analyte; for example, it is

observed for a range of matrices and levels of not uncommon to find that extraction losses

added analyte.

differ between high and low levels of analyte. • Comparison of result observed in a reference

7.7.5. Bias for a method under study can also be material with the recovery of added analyte in determined by comparison of the results with

the same reference material. those of a reference method. If the results show that the bias is not statistically significant, the

• Judgement on the basis of specific materials standard uncertainty is that for the reference

with known extreme behaviour. For example, method (if applicable; see section 7.8.1.),

oyster tissue, a common marine tissue combined with the standard uncertainty

reference, is well known for a tendency to co- associated with the measured difference between

precipitate some elements with calcium salts on digestion, and may provide an estimate of

methods. The latter contribution to uncertainty is ‘worst case’ recovery on which an uncertainty given by the standard deviation term used in the estimate can be based (e.g. By treating the significance test applied to decide whether the worst case as an extreme of a rectangular or difference is statistically significant, as explained

triangular distribution).

in the example below. • Judgement on the basis of prior experience.

EXAMPLE A method (method 1) for determining the

7.7.7. Bias may also be estimated by comparison

concentration of Selenium is compared with a

of the particular method with a value determined

reference method (method 2). The results (in

by the method of standard additions, in which

mg kg -1 ) from each method are as follows:

known quantities of the analyte are added to the test material, and the correct analyte

concentration inferred by extrapolation. The

Method 1

5.40 1.47 5 uncertainty associated with the bias is then normally dominated by the uncertainties

Method 2

4.76 2.75 5 associated with the extrapolation, combined

The standard deviations are pooled to give a

(where appropriate) with any significant pooled standard deviation s c contributions from the preparation and addition of

2 stock solution.

5 + 5 − 2 N OTE : To be directly relevant, the additions should be made to the original sample, rather than a

and a corresponding value of t:

prepared extract.

= 0 . 46 7.7.8. It is a general requirement of the ISO Guide

1 1  1 . 4 that corrections should be applied for all 2 . 205 +  

recognised and significant systematic effects. Where a correction is applied to allow for a

t crit is 2.3 for 8 degrees of freedom, so there is

significant overall bias, the uncertainty associated

no significant difference between the means of

with the bias is estimated as paragraph 7.7.5.

the results given by the two methods. However,

described in the case of insignificant bias

the difference (0.64) is compared with a standard deviation term of 1.4 above. This

7.7.9. Where the bias is significant, but is

value of 1.4 is the standard deviation associated

nonetheless neglected for practical purposes,

with the difference, and accordingly represents

additional action is necessary (see section 7.15.).

the relevant contribution to uncertainty associated with the measured bias.

Page 19

Quantifying Uncertainty Step 3. Quantifying Uncertainty Additional factors

leachable metals in ceramics and dietary fibre in foodstuffs (see also section 5.2. and example A5)

7.7.10. The effects of any remaining factors should be estimated separately, either by

7.8.2. Where such a method is in use within its experimental variation or by prediction from

defined field of application, the bias associated established theory. The uncertainty associated

with the method is defined as zero. In such with such factors should be estimated, recorded

circumstances, bias estimation need relate only to and combined with other contributions in the

the laboratory performance and should not normal way.

additionally account for bias intrinsic to the method. This has the following implications.

7.7.11. Where the effect of these remaining factors is demonstrated to be negligible compared

7.8.3. Reference material investigations, whether to the precision of the study (i.e. statistically

to demonstrate negligible bias or to measure bias, insignificant), it is recommended that an

should be conducted using reference materials uncertainty contribution equal to the standard

certified using the particular method, or for which deviation associated with the relevant

a value obtained with the particular method is significance test be associated with that factor.

available for comparison.

EXAMPLE

7.8.4. Where reference materials so characterised are unavailable, overall control of bias is

The effect of a permitted 1-hour extraction time variation is investigated by a t-test on five

associated with the control of method parameters

determinations each on the same sample, for the

affecting the result; typically such factors as

normal extraction time and a time reduced by 1

times, temperatures, masses, volumes etc. The

hour. The means and standard deviations (in

uncertainty associated with these input factors

mg l -1 ) were: Standard time: mean 1.8, standard

must accordingly be assessed and either shown to

deviation 0.21; alternate time: mean 1.7,

be negligible or quantified (see example A6).

standard deviation 0.17. A t-test uses the pooled variance of

7.8.5. Empirical methods are normally subjected to collaborative studies and hence the uncertainty

can be evaluated as described in section 7.6.

to obtain

7.9. Evaluation of uncertainty for ad-

hoc methods

7.9.1.  Ad-hoc methods are methods established to 

carry out exploratory studies in the short term, or

This is not significant compared to t crit = 2.3.

for a short run of test materials. Such methods are

But note that the difference (0.1) is compared

typically based on standard or well-established

with a calculated standard deviation term of

methods within the laboratory, but are adapted

0 . 037 × ( 1 / 5 + 1 / 5 ) =0.12. This value is the

substantially (for example to study a different analyte) and will not generally justify formal

contribution to uncertainty associated with the effect of permitted variation in extraction time.

validation studies for the particular material in question.

7.7.12. Where an effect is detected and is statistically significant, but remains sufficiently

7.9.2. Since limited effort will be available to small to neglect in practice, the provisions of

establish the relevant uncertainty contributions, it section 7.15. apply.

is necessary to rely largely on the known performance of related systems or blocks within

7.8. Evaluation of uncertainty for these systems. Uncertainty estimation should

accordingly be based on known performance on a

empirical methods

related system or systems. This performance

7.8.1. An ‘empirical method’ is a method agreed information should be supported by any study upon for the purposes of comparative

necessary to establish the relevance of the measurement within a particular field of

information. The following recommendations application where the measurand

assume that such a related system is available and characteristically depends upon the method in

has been examined sufficiently to obtain a use. The method accordingly defines the

reliable uncertainty estimate, or that the method measurand. Examples include methods for

consists of blocks from other methods and that

Page 20

Quantifying Uncertainty Step 3. Quantifying Uncertainty

the uncertainty in these blocks has been § By modelling from theoretical principles established previously.

§ Using judgement based on experience or

7.9.3. As a minimum, it is essential that an informed by modelling of assumptions estimate of overall bias and an indication of

These different methods are discussed briefly precision be available for the method in question.

below.

Bias will ideally be measured against a reference material, but will in practice more commonly be

7.11. Experimental estimation of

assessed from spike recovery. The considerations of section 7.7.4. then apply, except that spike

individual uncertainty

recoveries should be compared with those

contributions

observed on the related system to establish the

7.11.1. It is often possible and practical to obtain relevance of the prior studies to the ad-hoc

estimates of uncertainty contributions from method in question. The overall bias observed for

experimental studies specific to individual the ad-hoc method, on the materials under test,

parameters.

should be comparable to that observed for the related system, within the requirements of the

7.11.2. The standard uncertainty arising from study.

random effects is often measured from repeatability experiments and is quantified in

7.9.4. A minimum precision experiment consists terms of the standard deviation of the measured of a duplicate analysis. It is, however,

values. In practice, no more than about fifteen recommended that as many replicates as practical

replicates need normally be considered, unless a are performed. The precision should be compared

high precision is required.

with that for the related system; the standard deviation for the ad-hoc method should be

7.11.3. Other typical experiments include: comparable.

• Study of the effect of a variation of a single

N OTE : It recommended that the comparison be based

parameter on the result. This is particularly

on inspection. Statistical significance tests

appropriate in the case of continuous,

(e.g. an F-test) will generally be unreliable

controllable parameters, independent of other

with small numbers of replicates and will tend

effects, such as time or temperature. The rate

to lead to the conclusion that there is ‘no

of change of the result with the change in the

significant difference’ simply because of the

parameter can be obtained from the

low power of the test.

experimental data. This is then combined

7.9.5. Where the above conditions are met directly with the uncertainty in the parameter unequivocally, the uncertainty estimate for the

to obtain the relevant uncertainty contribution. related system may be applied directly to results

N OTE : The change in parameter should be sufficient

obtained by the ad-hoc method, making any

to change the result substantially compared to

adjustments appropriate for concentration

the precision available in the study (e.g. by

dependence and other known factors.

five times the standard deviation of replicate measurements)

7.10. Quantification of individual

• Robustness studies, systematically examining

components

the significance of moderate changes in

7.10.1. It is nearly always necessary to consider parameters. This is particularly appropriate for some sources of uncertainty individually. In some

rapid identification of significant effects, and cases, this is only necessary for a small number of

commonly used for method optimisation. The sources; in others, particularly when little or no

method can be applied in the case of discrete method performance data is available, every

effects, such as change of matrix, or small source may need separate study (see examples 1,2

equipment configuration changes, which have and 3 in Appendix A for illustrations). There are

unpredictable effects on the result. Where a several general methods for establishing

factor is found to be significant, it is normally individual uncertainty components:

necessary to investigate further. Where insignificant, the associated uncertainty is (at

§ Experimental variation of input variables least for initial estimation) that obtained from § From standing data such as measurement and

the robustness study.

calibration certificates

Page 21

Quantifying Uncertainty Step 3. Quantifying Uncertainty

• Systematic multifactor experimental designs would give the standard uncertainty. Of course, intended to estimate factor effects and

systematic deviation from traceable assigned interactions. Such studies are particularly

values and any other sources of uncertainty (such useful where a categorical variable is

as those noted in section 7.6.1.) must also be involved. A categorical variable is one in

taken into account.

which the value of the variable is unrelated to

7.12.3. Quality Assurance (QA) data. As noted the size of the effect; laboratory numbers in a

previously it is necessary to ensure that the study, analyst names, or sample types are quality criteria set out in standard operating examples of categorical variables. For procedures are achieved, and that measurements example, the effect of changes in matrix type on QA samples show that the criteria continue to (within a stated method scope) could be

be met. Where reference materials are used in QA estimated from recovery studies carried out in checks, section 7.5. shows how the data can be

a replicated multiple-matrix study. An analysis used to evaluate uncertainty. Where any other of variance would then provide within- and stable material is used, the QA data provides an between-matrix components of variance for estimate of intermediate precision (Section observed analytical recovery. The between- 7.7.2.). QA data also forms a continuing check on

matrix component of variance would provide a the value quoted for the uncertainty. Clearly, the standard uncertainty associated with matrix

combined uncertainty arising from random effects variation.

cannot be less than the standard deviation of the QA measurements.

7.12. Estimation based on other results or data

7.12.4. Suppliers' information. For many sources of uncertainty, calibration certificates or suppliers

7.12.1. It is often possible to estimate some of the catalogues provide information. For example, the standard uncertainties using whatever relevant

tolerance of volumetric glassware may be information is available about the uncertainty on

obtained from the manufacturer’s catalogue or a the quantity concerned. The following paragraphs

calibration certificate relating to a particular item suggest some sources of information.

in advance of its use.

7.12.2. Proficiency testing (PT) schemes. A laboratory’s results from participation in PT

7.13. Modelling from theoretical

schemes can be used as a check on the evaluated

principles

uncertainty, since the uncertainty should be

7.13.1. In many cases, well-established physical compatible with the spread of results obtained by theory provides good models for effects on the that laboratory over a number of proficiency test result. For example, temperature effects on rounds. Further, in the special case where volumes and densities are well understood. In

• the compositions of samples used in the such cases, uncertainties can be calculated or scheme cover the full range analysed

estimated from the form of the relationship using routinely

the uncertainty propagation methods described in • section 8.

the assigned values in each round are traceable to appropriate reference values, and

7.13.2. In other circumstances, it may be • necessary to use approximate theoretical models

the uncertainty on the assigned value is small combined with experimental data. For example, compared to the observed spread of results

where an analytical measurement depends on a timed derivatisation reaction, it may be necessary

then the dispersion of the differences between the to assess uncertainties associated with timing. reported values and the assigned values obtained This might be done by simple variation of elapsed in repeated rounds provides a basis for a good time. However, it may be better to establish an estimate of the uncertainty arising from those approximate rate model from brief experimental parts of the measurement procedure within the studies of the derivatisation kinetics near the scope of the scheme. For example, for a scheme concentrations of interest, and assess the

operating with similar materials and analyte uncertainty from the predicted rate of change at a levels, the standard deviation of differences

given time.

Page 22

Quantifying Uncertainty Step 3. Quantifying Uncertainty

7.14. Estimation based on judgement

the extent to which a proclaimed standard substance reasonably resembles the nature of

7.14.1. The evaluation of uncertainty is neither a the samples in a particular situation. routine task nor a purely mathematical one; it

depends on detailed knowledge of the nature of • Another source of uncertainty arises when the the measurand and of the measurement method

measurand is insufficiently defined by the and procedure used. The quality and utility of the

procedure. Consider the determination of uncertainty quoted for the result of a

"permanganate oxidizable substances" that measurement therefore ultimately depends on the

are undoubtedly different whether one understanding, critical analysis, and integrity of

analyses ground water or municipal waste those who contribute to the assignment of its

water. Not only factors such as oxidation value.

temperature, but also chemical effects such as matrix composition or interference, may

7.14.2. Most distributions of data can be have an influence on this specification. interpreted in the sense that it is less likely to

observe data in the margins of the distribution

A common practice in analytical chemistry than in the centre. The quantification of these

calls for spiking with a single substance, such distributions and their associated standard

as a close structural analogue or isotopomer, deviations is done through repeated

from which either the recovery of the measurements.

respective native substance or even that of a whole class of compounds is judged. Clearly,

7.14.3. However, other assessments of intervals the associated uncertainty is experimentally may be required in cases when repeated

assessable provided the analyst is prepared to measurements cannot be performed or do not

study the recovery at all concentration levels provide a meaningful measure of a particular

and ratios of measurands to the spike, and all uncertainty component.

"relevant" matrices. But frequently this

7.14.4. There are numerous instances in experimentation is avoided and substituted by analytical chemistry when the latter prevails, and

judgements on

judgement is required. For example: • the concentration dependence of • An assessment of recovery and its associated

recoveries of measurand, uncertainty cannot be made for every single

• the concentration dependence of sample. Instead, an assessment is made for

recoveries of spike,

classes of samples (e.g. grouped by type of matrix), and the results applied to all samples

• the dependence of recoveries on (sub)type of similar type. The degree of similarity is

of matrix,

itself an unknown, thus this inference (from • the identity of binding modes of native type of matrix to a specific sample) is

and spiked substances. associated with an extra element of

uncertainty that has no frequentistic

7.14.5. Judgement of this type is not based on interpretation.

immediate experimental results, but rather on a subjective (personal) probability, an expression

• The model of the measurement as defined by which here can be used synonymously with the specification of the analytical procedure

"degree of belief", "intuitive probability" and is used for converting the measured quantity

"credibility" [H.11]. It is also assumed that a to the value of the measurand (analytical

degree of belief is not based on a snap judgement, result). This model is - like all models in

but on a well considered mature judgement of science - subject to uncertainty. It is only

probability.

assumed that nature behaves according to the specific model, but this can never be known

7.14.6. Although it is recognised that subjective with ultimate certainty.

probabilities vary from one person to another, and even from time to time for a single person, they

• The use of reference materials is highly are not arbitrary as they are influenced by

encouraged, but there remains uncertainty common sense, expert knowledge, and by earlier regarding not only the true value, but also

experiments and observations. regarding the relevance of a particular

reference material for the analysis of a

7.14.7. This may appear to be a disadvantage, but specific sample. A judgement is required of

need not lead in practice to worse estimates than

Page 23

Quantifying Uncertainty Step 3. Quantifying Uncertainty

those from repeated measurements. This applies

7.15. Significance of bias

particularly if the true, real - life, variability in experimental conditions cannot be simulated and

7.15.1. It is a general requirement of the ISO the resulting variability in data thus does not give

Guide that corrections should be applied for all

a realistic picture. recognised and significant systematic effects.

7.15.2. In deciding whether a known bias can long-term variability needs to be assessed when

7.14.8. A typical problem of this nature arises if

reasonably be neglected, the following approach no collaborative study data are available. A

is recommended:

scientist who dismisses the option of substituting

i) Estimate the combined uncertainty without subjective probability for an actually measured

considering the relevant bias. one (when the latter is not available) is likely to

ii) Compare the bias with the combined ignore important contributions to combined

uncertainty.

uncertainty, thus being ultimately less objective than one who relies on subjective probabilities.

iii) Where the bias is not significant compared to

the combined uncertainty, the bias may be For the purpose of estimation of combined

neglected.

uncertainties two features of degree of belief estimations are essential:

iv) Where the bias is significant compared to the • combined uncertainty, additional action is

degree of belief is regarded as interval valued required. Appropriate actions might: which is to say that a lower and an upper

bound similar to a classical probability • Eliminate or correct for the bias, making distribution is provided,

due allowance for the uncertainty of the • correction.

the same computational rules apply in combining 'degree of belief' contributions of

• Report the observed bias and its uncertainty to a combined uncertainty as for

uncertainty in addition to the result. standard deviations derived by other

N OTE : Where a known bias is uncorrected by methods. convention, the method should be considered

empirical (see section 7.8).

Page 24

Quantifying Uncertainty Step 4. Calculating the Combined Uncertainty