Note on statistical reliability and confidence intervals

Participants in a survey such as GPPS represent only a sample of the total population of interest – this means we cannot be certain that the results of a question are exactly the same as if everybody within that population had taken part (“true values”). However, we can estimate the true value by considering the size of the sample on which results are based, and the number of times a particular answer is given. The confidence with which we make this estimate is usually chosen to be 95% – that is, the chances are 95 in 100 that the true value will fall within a specified range (the “95% confidence interval”).

The table below gives examples of what the confidence intervals look like for a PCN and GP practice with an average number of responses, as well as the confidence intervals at the national level, based on weighted data. Confidence intervals will be wider when the results are based on a smaller number of responses.

For example, taking a PCN where 539 people responded and where 30% gave a particular answer, there is a 95% likelihood that the true value (which would have been obtained if the whole population had taken part in the survey) will fall within the range of +/-5.22 percentage points from that question’s result (i.e. between 24.78% and 35.22%).

When results are compared between separate groups within a sample, the difference may be “real” or it may occur by chance (because not everyone in the population has taken part in the survey).