Home > Research > Services > Report Abstracts > 317

Abstract from BTO Research Report 317:

Joys, A.C., Noble, D.G. & Baillie, S.R. (2003)

Evaluation of species coverage and precision using the BBS indexing method.

Executive summary

1. The Breeding Bird Survey (BBS) is a volunteer based survey, funded by the British Trust for Ornithology (BTO), Joint Nature Conservation Committee (JNCC) and the Royal Society for the Protection of Birds (RSPB) and has been running since 1994. It was primarily set up to increase both the geographical coverage and the range of habitats and species covered compared to the earlier CBC. Annual bird population trends for a range of common and widespread birds are provided for the United Kingdom, the four countries (England, Wales, Scotland and Northern Ireland) and the nine Government Office Regions/Regional Development Agency (RDA). Volunteers are required to survey a 1 x 1 km square using two 1 km parallel routes, with each transect split into five 200 m sections. Habitat information is recorded in April with two later visits; one in April/early May and the latter in late May/June to record the birds. All bird registrations (both sight and sound) are recorded into either one of three distance categories along the transect (<25m, 25-100 m and beyond 100 m) or noted as in flight for each of the 200 m sections.

2. BBS population indices are estimated using a log-linear model with Poisson error terms, with the sum of the counts across the four distance categories for the 1 km square being modelled as a function of year and site effects. Counts are corrected for over-dispersion and weighted to account for differences in sampling effort amongst regions. Only squares that have been surveyed in two or more years in the period of interest can be included in the analysis. With current methodology, national population trends are reported for all species which occur in greater than 50 squares on a yearly basis with this being decreased to a sample size of 30 for country level and 20 for the RDAs. It is important to ensure these indices are reliable and reflect the true population. This report aims to develop a protocol that can be used to determine whether datasets for particular species and geographical areas should be used to calculate population indices. The protocol takes account of both the relationship between sample size and precision and aspects of species biology that determine whether BBS methods are likely to produce reliable indices.

3. This report aims to assess the precision of the indices and formulate objective criteria for the reporting of population trends based on sample size (yearly average of squares in which the species is seen), below which it is inappropriate to report population change due to low precision. This will focus on exploration of the existing data and the statistical models used to produce the trends and secondly simulation-based power analysis to assess effects of sample size and variability of counts on the power of the statistical model. This report uses data collected between 1994-2000.

4. For most species, count data is over-dispersed in relation to the Poisson distribution. This is especially the case for several geese, gulls and waders. Missing site by year combinations in the index model account for less then half of the whole site by year data matrix for species for which the BBS model is currently run. Species with a low mean count tend to have low variance and dispersion. Most species which show a tendency to flock (waders, gulls, terns, geese, wild fowl and colonial seabirds) have high mean counts, maximum counts, dispersion and spatial variability. This suggests that it is appropriate to exclude large counts for these species, as routinely done for current BBS indices. There is little difference in the shape of the distribution when birds in flight are included or excluded, although maximum counts are likely to be different.

5. The precision of the population change index between 1994 to 2000 was assessed using the minimum detectable effect size (‘effect size’), this is the minimum change which could be detected as significant. Greatest variation in effects size with sample size occurs below 200; decreasing sample size increases minimum detectable effect size, i.e. it is harder to detect small significant changes in the population. As percent of zero’s and degree of missing values in the model increases, the ability to detect a small significant change decreases. Mean count and over-dispersion have less influence on effect size. For the period covered, only population changes greater than 50% are likely to be detected for species with a sample size less than 40. Population changes in the range 20-50% can potentially be detected with sample sizes between 40 and 100.

6. With the simulation-based approach, the majority of species with a sample size >20 and <100 have a power less than 70% to detect a 25% population decline over a period of 25 years. This is likely to be an overestimate due to the simple approach used in simulation. Power increased with sample size and decreasing percent of zero’s. Little association existed between the power from the simulation-based approach and the minimum detectable effect size. Due to the simplistic nature of the simulation approach, the results from the effect size analysis are more likely to reflect the true characteristics of the data in terms of over-dispersion, spatial and temporal variability.

7. The precision of the reported population trends at the country and regional level are likely to be poor using the current criteria of sample size above 30 and 20 respectively. Using a sample size above 40 is more appropriate and is in line with the national criteria for reporting. Precision is clearly still relatively low for species with a sample size below 100 at the national level. Restricting the reporting of trends to those species above 100 at the national level, as opposed to 50, would result in 14 more species being excluded.

8. A protocol is suggested for the reporting of BBS trends based on both numerical sample size and biological criteria. Sample size (yearly mean number of squares in which the species is seen) has the most influence. Three levels are proposed. The first includes species seen in greater than 40 squares on an annual basis (sample size) and where the BBS sampling design is appropriate. For these species there should be a high probability that we can detect a 50% decline and for which it is appropriate to report BBS indices/trends. A sample size of greater than 40 is required to be able to detect a 50% or less population change. For species seen in less than 40 squares, one is only likely to be able to detect a change of greater than 50%. However, there are a number of ‘difficult’ species where their mean sample is at least 40 squares, for which the BBS trends should be treated with caution (category two). For these species (see Table 1), population trends should be reported with a caveat signifying that the reported trend may not reflect the status of the breeding population in the UK, as their counts are likely to be strongly influenced by non-breeders, migrants, or because the BBS methodology may not reliably sample the population (e.g. nocturnal or colonial species). The third category includes species for which indices should not be produced as sample sizes are less than 40 or for species whose counts include a high proportion of non-breeding birds, wintering birds or those seen during migration (Table 1 lists the latter species for category 3).

Back to Research Reports 289-344