Treatment of 'less-than' measurements

Overview

Time series with 'less-than' measurements present all sorts of statistical problems, particularly if the limit of detection varies over time. The appropriate way of dealing with them depends on the proportion of measurements that are less-thans and the distribution of less-thans through the time series (i.e. all at the start, or spread evenly through the time series). Efficient statistical methods can often be developed for time series with full and reliable information. However, these methods can be complex and difficult to apply in large-scale assessments where the quality of information can degrade going back in time. Consequently, pragmatic adjustments to the standard methodology have been made to allow sensible, but sub-optimal, assessments of all time series that contain less-thans. There are adjustments to the calculation of the annual indices and to the length of the time series that is used to assess temporal trends. When most of the data are less-thans, a non-parametric test is used to compare levels with assessment criteria.

Calculating annual indices

The calculation of the annual indices is unaffected by less-thans. However, an annual index is treated as a less-than index if, that year, any measurement greater or equal to the median value is a less-than.

Formally, let c_ti, i = 1 ... n_t be the measurements in year t, t = 1 ... T. Further, assume the measurements are ordered within-years so that c_t1 ≤ c_t2 ≤ ... ≤ c_tnt. The annual index in year t is then regarded as a less-than if any of the c_ti, i = m ... n_t is a less-than, where m = (n_t + 1)/2 if n_t is odd or m = n_t/2 if n_t is even.

Length of time series

When some of the annual indices are less-thans the time series is truncated to ensure that the less-thans do not unduly influence the assessment of temporal trends. The three rules are that:

1. The time series is truncated from the left; i.e. early years are omitted, later years are retained.
2. The time series cannot start with a less-than.
3. The time series cannot contain more than one less-than.

Let y_t be the annual index in year t, t = 1 ... T, and let z_t = 1 if y_t is a less-than and z_t = 0 otherwise. Then the time series that is assessed for temporal trends is y_t, t = m ... T, where m is chosen to give the longest time series subject to the constraints that z_m = 0 and

`\sum_m^T z_t <= 1`.

Non-parametric assessment of environmental status

If the length of the truncated time series is 2 years of less, then there are insufficient years to fit a parametric model and make a formal assessment of environmental status. However, if the original time series has more than five years of data, a one-sided sign test is used instead to provide a non-parametric test of status. All the indices in the last twenty years (the same period used to assess recent trends) are used to test the null hypothesis: H₀: μ ≥ AC against the alternative: H₁: μ < AC, where μ is the mean level and AC is the assessment criterion.