The mean and maximum value tests in CLR 7

The mean value test involves determining a value, US₉₅, the upper 95th percentile bound, that the true average concentration within an averaging area will be less than, with a 95% confidence level. The US₉₅ is then compared with the relevant assessment criterion (Table 2).

The maximum value test involves calculating a value T, based on the logarithms, y, of the measured contaminant concentrations, x, and comparing T with a critical value of T in CLR 7. The maximum value test is carried out to determine if the maximum value in the data set represents an outlier, and therefore a potential hotspot of contamination (Table 3). Although CLR 7 implies that the mean value test is carried out first, it makes more sense to carry out the maximum value test first to ensure that all data are taken from the same population.

CLR 7 does not provide detailed advice on how and when to apply or not to apply these statistical tests. However, what advice there is (Appendix 1) is unequivocal and not in accordance with other equally authoritative guidance (e.g. Environment Agency 2000). Since the publication in March 2002 of CLR 7, a number of reports in which the tests have been misapplied have been peer reviewed by the author on behalf of several local authorities.

It would appear that the tests, or rather their application and interpretation, are creating some confusion and there is a possibility that land that should be re-mediated will not be. Thus leaving receptors potentially exposed to unacceptable contaminant concentrations, or that land that does not require re-mediation will be re-mediated, with a resultant waste of resources.

This technical note highlights some of the limitations of these tests and points out ways to overcome them.

Both US₉₅ and T depend on the number of samples taken within the averaging area. CLR 7 is however silent on the spatial distribution of samples or on how to deal with heterogeneous ground conditions within an averaging area although Paragraph 4.7 of CLR 7 states that ‘zoning may take into account such characteristics as variations in soil properties or historical existing or proposed land uses’. The definition of averaging (Appendix 1) implies that the US₉₅ should be determined for individual tracts of ground where a single receptor could be exposed to contamination. The alternative of determining US₉₅ for a zone of uniform ground conditions is not recognized as being acceptable in CLR 7. It also fails to point out that the tests do not apply to targeted or judgmental sampling patterns.

Guidance in P5-066/TR

P5-066/TR, the Soil Sampling Secondary Model Procedure (Environment Agency 2000), provides extensive guidance on the development of soil sampling strategies. This is the precursor to the comparison of the results of analyses of samples from such sampling strategies with assessment criteria. Although it might be expected that the guidance in CLR 7 would be tightly tied into the advice in P5-066/TR, this is not the case. P5-066/TR states that ‘it is difficult to overemphasize the importance of the conceptual model in describing initial views about the risks likely to be posed by the site, and in guiding any additional data collection effort’ (p.6). However one could go further in that the conceptual model has a role in detailed quantitative risk assessment and in determining which assessment criteria are relevant. The Statutory Guidance in DETR Circular 2/2000 (DETR 2000) supporting Part IIA of the Environmental Protection Act 1990 recognizes that in paragraph B48. P5-066/TR recognizes the potential for site zoning to reflect variations in ground conditions (p.11). P5-066/TR compares targeted (judgmental) and non-targeted (systematic) sampling. P5-066/TR points out that targeted sampling is based on prior information collected during the desk study and walkover phase, and may be used to confirm that an area is or is not contaminated. Non-targeted sampling is used ‘where representative information on the whole of a site (or zone) is required’. This however is likely to be rare, as prior information will help eliminate large tracts of many sites, and indeed many sites, from the need for intrusive investigation. P5-066/TR summarizes the examples of sampling patterns presented in CLR 4 by the DoE (1994). However, CLR 4 deals with the investigation of potential hotspots and not the design of sampling patterns to provide ‘representative information about the site as a whole’. P5-066/TR recognizes that receptor behaviour may result in exposure that is concentrated in one part of an averaging area rather than from within the entire averaging area (p.62 & p.65). However, the statistical tests in CLR 7 and the assumptions in the soil guideline values ignore this for ‘standard’ land uses (i.e. land uses for which soil guideline values have been determined). P5-066/TR relies on ever increasing sampling densities to provide additional certainty, but makes no reference to the degree of spatial correlation in contaminant concentrations. Without recourse to a variogram, the benefits of increased sampling densities cannot be quantified (Nathanail 1997). P5-066/TR concludes that the interpretation of sampling data relies ‘heavily on professional judgement’ and ‘does not rely solely on the statistical basis for sampling’ (p.64).

Overall P5-066/TR contains much valuable guidance on the design of sampling strategies, but steps back from making a clear link with the interpretation of the outcomes of the sampling. CLR 7 on the other hand fails to recognize the guidance in P5-066/TR and to provide robust guidance on when mean and maximum value tests should or should not be used.

The effect of clustering

The assumption behind the statistics in both the mean and maximum value tests in CLR 7 is that each sample represents an equal fraction of the averaging area. There is an implicit underlying assumption that samples are located on a square regular grid pattern with each sample having the same area of influence ( Fig. 1). Where samples are not evenly spaced, this is not true. For example, where 2 or more rounds of sampling have been carried out by competent and ethical site investigators, later sampling may be infilling the original non-targeted sample grid and targeting areas around high concentrations to determine the extent of the high concentration area ( Fig. 2). Using the mean value test in this case is inappropriate, as each of the later stage samples represents a much smaller area and furthermore represents an area suspected of having a higher concentration.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1

Indicative Non Targeted Sampling Pattern.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2

Additional Samples.

A site investigator could, on the other hand, choose to target later stages of sampling in areas of low concentration in order to try and make the site or averaging area pass the mean value test (Fig. 2). Once again, the clustered samples are not representing the same area as the initial samples.

If the mean value test is to be applied to unevenly spaced, or clustered, datasets then the data must be declustered. This could be achieved by weighting each sample location by the area of its Thiessen polygon. A Thiessen polygon of a sample defines the area, which is closer to that sample than to any other sample in the data set. Thiessen polygons are closely related to Delaunay Triangulations, the basis of many computer triangulations, contouring algorithms. Where samples are closely spaced, Thiessen polygons will be small and weights will be correspondingly low. Where samples are more widely spaced, Thiessen polygons will be larger and weights will be greater. A description of the use of Thiessen polygons for declustering is given in Nathanail (1994). Davis (2002) describes the theoretical basis of Thiessen polygons. Nathanail (1993, 1994) describes the use of bespoke Thiessen polygon calculat-ing software to assist in coal reserve estimation. Similar software tailored to the requirements of CLR 7 would be of assistance to risk assessors.

The effect of heterogeneity

The mean value test in CLR 7 assumes that the samples are drawn from the same population. However even over the scale of an individual garden, soil type can vary significantly ( Fig. 3). The samples are therefore taken from two different populations and the US₉₅ cannot be taken as an estimate of the population mean of the overall averaging area.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 3

The Influence of Ground Conditions.

If a statistical approach for an individual averaging area must be taken, then it is better to determine the US₉₅ for each soil type and then determine a weighed average of the US₉₅ for the entire averaging area with weights based on the relative areas of each soil outcrop. This would also be in keeping with the guidance in P5-066/TR (Environment Agency 2000).

The effect of a targeted sampling strategy

The mean and maximum value tests assume an evenly spaced grid of samples. No warning is given that the tests do not apply to sampling patterns that have been targeted to investigate specific features determined during the desk study or site walkover (e.g. fuel storage tanks, stained ground or Hazchem warning signs).

Measurements below the limit of detection

Chemical analysis of samples of potentially contaminated land is part of the process of reducing uncertainty (Thompson & Nathanail 2003). As such, there is a limit below which analysis cannot reliably measure contaminant concentrations. CLR 7 recommends that samples recorded as being below this limit of detection should have a value assigned to them that is equal to the detection limit. In other circumstances, for example where seeking to estimate ore reserves, the cautious approach would have been to assign a value of zero or a value some way between zero and the limit of detection. In land contamination cases, the limit of detection should, where possible, be chosen to be well below any assessment criterion. If the results of the tests in CLR 7 depend on how samples below the detection limit are treated, then it is likely that the detection limit is too high. However soil guideline values for residential land use with plant uptake for some metals are at or close to laboratory reporting limits (see for example DEFRA & Environment Agency 2002b).

This suggests that risk assessors should be aware of the likely magnitude of the assessment criteria before commissioning analyses of potentially contaminated soils.

Conclusions

This paper has outlined the limitations of the maximum and mean value tests. Regulators and contaminated land practitioners should be aware of the effects of clustered sample locations, which may generate false results from the mean value test. Clustering around a previous high concentration will artificially raise the sample mean and could result in unnecessary interpretation that the SGV has been exceeded. Conversely, clustering around a low concentration value will result in a lower sample mean and could result in incorrect interpretation that the SGV has not been exceeded. A program to calculate Thiessen polygons, using sample locations and averaging area boundary as inputs, would help ensure US₉₅ is suitably declustered.

Use of the statistical tests presented in CLR 7 where there is a variation in the ground conditions could result in a false decision being made by regulators. Reference to the conceptual model should be made in order to determine the likely range of materials within the averaging area to which the critical receptor may be exposed. If the geochemistry of the materials differs significantly, then each material should be assessed individually.

Finally it is unlikely that large plots of land that are intended for residential use will be so intensively investigated as to ensure an adequate number of samples within each future garden area. Indeed the location of the gardens may not be known at the time of the investigation. Developers could of course:

1. choose not to provide soft landscaped gardens in which exposure could occur;

2. carry out investigations of garden areas once they are established;

3. characterize the materials that are likely to form the ground cover in garden areas and determine on a material by material basis the need for re-mediation independently of the location of proposed gardens.

Nathanail et al. (2002) provided a synthesis of CLR 7 (DEFRA & Environment Agency 2002a) and P5-066/TR, the Soil Sampling Secondary Model Procedure (Environment Agency 2000) to assist risk assessors to ensure that samples are relevant and representative. The tests in CLR 7 are robust and powerful and can in certain circumstances be applied directly to the results of a site investigation to assist in the interpretation of the significance of contaminant concentrations in soil.

Appendix 1. Use of Soil Guideline Values in Part IIA (Extracted from CLR 7, DEFRA & Environment Agency 2002a)

4.10 When considering how many data points are needed for an area of land to provide a representative indicator of receptor exposure to the contaminant, the concept of averaging areas needs to be applied. ‘An averaging area (or area of interest) is that area (together with a consideration of depth) of soil to which a receptor is exposed or which otherwise contributes to the creation of hazardous conditions’.

‘The soil in the averaging area will contain variable concentrations of contaminants, which, when averaged across the area, will provide a representative indicatorof how much contaminant the receptor is exposed to. Defining such areas requires knowledge of the behaviour pattern of the receptor. For example, a single gardenon a housing estate may be an appropriate averaging area for a child living in the house. The entire area ofan abandoned industrial site that forms an informal play area for neighbourhood children could also be anappropriate averaging area to assess the risk from contaminants.’ More detailed guidance on sampling strategies is available [Environment Agency 2000]. It describes methods of defining sampling grid sizes to allow detection of given areas of interest (that is, contaminated areas) to a stated degree of statistical confidence.

4.11 Once the results are available for an appropriate averaging area, the key question is whether the mean measured contaminant concentration is greater or less than a published Soil Guideline Value (or a comparable figure from a site-specific study). Even where all the analytical results contributing to the mean lie below (or above) the Soil Guideline Value, errors in estimating the ‘true’ mean from a limited number of samples could result in incorrect conclusions being drawn. In all cases, the statistical tests outlined below and in Appendix A[of CLR 7] should be followed. Contaminant concentrations vary across a site, and sampling and analysis will introduce measurement errors. As a result, the mean concentration determined from a limited number of samples will have uncertainty associated with it, and will not necessarily equal the true mean concentration at the site. A comparison between a mean measured concentration and the Soil Guideline Value must take this uncertainty into account. Appendix A and Annex 1 set out a mean value test which compares the Soil Guideline Value with the upper 95th percentile of the mean measured concentration. Where the mean value test is applied, and the data passes the test, local authorities may consider that the site requires no further action. Conversely, where a set of data fails the mean valuetest, a local authority can make judgements about the benefits of undertaking more comprehensive sampling, where only a small number of data points exist, or move towards determination as contaminated land under Part IIA, assuming that a significant pollutant linkage has been identified. The mean value test should be applied to the data set for each of the averaging areas being considered.

4.13 In some data sets, individual concentrations may have been measured at particular locations that are much higher than the rest of the data. Decisions need to be made on whether these concentrations fall within the maximum of the range of values that can be expected from the sample population, or whether they are indicative of an area of higher contamination (in effect, a different population with a higher mean). Data points that do not fall within the expected distribution of measurements for the sample population are termed ‘outliers’. The maximum value test (see Appendix Aand Annex 2[of CLR 7]) can be used to define whether the maximum measured concentration in the soilshould be classified as an outlier, and hence whether additional investigation might be warranted in thevicinity to clarify further the extent and nature of the contamination.