What is weighting? Why do you weight the data?
Weighting changes the data to account for differences between all patients at a practice and the sub-set of patients who actually complete the questionnaire.
For example: what if one GP practice has many more 16 to 24-year-old patients than 75 to 84-year-olds, but only a small number of 16 to 24-year-olds actually fill out the survey? By applying more weight to the responses from 16 to 24-year-olds, the results for a practice will more accurately reflect the views of the practice population as a whole.
How does the weighting actually work?
One of the problems with surveys is non response, which can cause some groups to be over or under represented. In order to correct for this weighting is applied. People in underrepresented groups get a weight larger than 1, and those in over represented groups get a weight smaller than 1.
Using the example above: if a lower proportion of 16 to 24-year-olds returned questionnaires (5%) than the proportion of 16 to 24-year-olds within the practice (10%). This means that 16 to 24-year-olds would be under-represented in the survey results. A ‘weight’ for responses from the 16 to 24-year-old group can be calculated by dividing the practice proportion (10%) by the response proportion (5%). Responses from 16 to 24-year-olds are then multiplied by the weight (2.0) to increase the influence of these responses in the final results, to make up for the low representation. A weight of less than 1 can be calculated when there is a higher proportion of responses than the practice proportion. This will decrease the influence of these responses in the final results.
The example above only uses age as a factor when deciding the weight. The weighting scheme for the GP Patient Survey includes many factors such as age, gender, geo-demographic classification (ACORN) and Government Office Region. In addition to other area level factors such as: deprivation, crime levels, ethnicity, marital status, overcrowding in households, household tenure and employment status.
More information about the weighting scheme is included in the latest version of the technical annex.
Why should I look at the weighted data rather than the unweighted data?
GPPS data is available in both weighted and unweighted formats. All official statistical publications lead with the weighted data. Weighting ensures results are more like the population of adult patients registered with a GP. This is the default option when viewing results through the GPPS website. Weighted data is useful for practices where fewer patients of a certain group (for example, younger patients) have filled in the survey than we would expect.
The unweighted data is raw, unadjusted data. It’s useful if you care about seeing individual responses, but less representative of how all patients at a practice might feel, and therefore also less useful for accurately comparing different organisations.
Why do you recommend using weighted data to look at GPPS results to compare practices, CCGs or other subgroups?
GPPS data is available in both weighted and unweighted formats. All official statistical publications lead with the weighted data. Ideally, the patients who return a GP Patient Survey should be representative of the patient population. The patient population are those aged 16+ who are registered with a GP. However, some groups may be over or under-represented in the final sample, for example certain groups may be more or less likely to respond. Weighting is used to change the data to account for the differences between all patients aged 16+ who are registered with a GP. It also accounts for the differences between all the patients who actually completed the questionnaire. It is therefore recommended that weighted data is used when looking at GPPS results to compare practices, CCGs or other subgroups.
Where can I learn more about the reasons for the current weighting scheme?
In Summer 2011, Ipsos MORI began a weighting investigation. The aim was to find out if the weighting scheme could be refined and developed further to help improve the accuracy of the results. The investigation used the previous years’ data (i.e. fieldwork conducted between 5th April 2010 and 7th April 2011). The results showed that when neighbourhood statistics such as ethnicity and deprivation are accounted for, non-response bias is reduced, improving the accuracy of findings.
Please click on the link below for a summary of the rationale for the adjustment to the weighting scheme. More detailed information about the weighting scheme is included in the latest version of the technical annex.
Summary of weighting strategy for 2011-2012
PLEASE NOTE: results for the 2011-2012 survey onwards use the new weighting scheme. The changes to the weighting scheme, questionnaire design and survey frequency, mean direct comparisons cannot be made with previous years’ data. This is true even in cases where the same questions have been asked.
Why has the weighting scheme been adjusted?
The weighting scheme was adjusted in 2011-12 to make the data better represent the views of the population as a whole. They now take into account local factors (such as deprivation, crime levels, ethnicity, marital status, overcrowding in households, household tenure and employment status).
The changes to the weighting scheme in 2012, as well as the questionnaire and survey frequency, mean that it is not possible to make like-for-like comparisons. This means that when looking at historic data, even in cases where the same questions have been asked, direct comparisons cannot be made.
