In the UK the Public Health Laboratory Service distributes simulated water samples with low or moderate counts of organisms, usually coliforms. Sometimes these include small numbers of E. coli. Statistical analysis can be used to assess the performance of laboratories. The reasoning behind the statistical methodology must be made clear to participants.

In any one trial the counts recorded by participating laboratories can be highly variable, but this may merely reflect the distribution of organisms between samples. Even with carefully prepared batches of water samples, where the aim is for random distribution of the organisms, there is no guarantee that randomness is achieved and greater variation may be observed. It would be unfair to criticise laboratories returning a result at the extreme ends of the range of counts without further evidence.

After a series of trials it is possible to assess whether there are any laboratories which are returning extreme low (or high) counts more often than would be expected by chance. As an example a non-parametric test (Cochran's Q) is used to demonstrate, from a series of ten trials, whether there were any laboratories giving poor results. With coliform organism counts an arbitrary definition of low or high counts is used.

It is important that the statistical assessment uses an approach which aims to detect poor performance when it is present but not when it is absent. The practice of automatically condemning a predetermined proportion (e.g. 5%) seems unfair, since all laboratories may be performing adequately or a large proportion may be inadequate.

The assessment demonstrated here is designed to cope with mixed methods - dilution series or direct counts.

This content is only available as a PDF.
You do not currently have access to this content.