What is weighting? Why do you weight the data?
Weighting adjusts the data to account for differences between all patients at a practice and the sub-set of patients who actually complete the questionnaire.
For example: what if one GP practice has many more 16 to 24-year-old patients than 75 to 84-year-olds, but only a small number of 16 to 24-year-olds actually fill out the survey? By applying more weight to the responses from 16 to 24-year-olds, the results for a practice will more accurately reflect the views of the practice population as a whole.
How does the weighting actually work?
One of the problems with surveys is non response, which can cause some groups to be over or under represented. In order to correct for this weighting is applied. Persons in underrepresented groups get a weight larger than 1, and those in over represented groups get a weight smaller than 1.
Using the example above: if a lower proportion of 16 to 24-year-olds returned questionnaires (5%) than the proportion of 16 to 24-year-olds within the practice (10%). This means that 16 to 24-year-olds would be under-represented in the survey results. A ‘weight’ for responses from the 16 to 24-year-old group can be calculated by dividing the practice proportion (10%) by the response proportion (5%). Responses from 16 to 24-year-olds are then multiplied by the weight (2.0) to increase the influence of these responses in the final results, thus counteracting the low representation. A weight of less than 1 can also be calculated when there is a higher proportion of responses than the practice proportion to decrease the influence of these responses in the final results.
The example above only considers age as a factor when determining the weight. The weighting scheme for the GP Patient Survey includes multiple factors such as age, gender, geo-demographic classification (ACORN), Government Office Region and a number of area level characteristics including: deprivation, crime levels, ethnicity, marital status, overcrowding in households, household tenure and employment status.
More detailed information about the weighting scheme is included in the latest version of the technical annex.
Why should I look at the weighted data rather than the unweighted data?
While GPPS data is available in both weighted and unweighted formats, all official statistical publications lead with the weighted data; weighting ensures results are more representative of the population of adult patients registered with a GP, and this is the default option when viewing results through the GPPS website. Weighted data is useful for practices where fewer patients of a certain group (for example, younger patients) have filled in the survey than we would expect.
The unweighted data is raw, unadjusted data. It’s useful if you care about seeing individual responses, but less representative of how all patients at a practice might feel, and therefore also less useful for accurately comparing different organisations.
Why do you recommend using weighted data to look at GPPS results to compare practices, CCGs or other subgroups?
While GPPS data is available in both weighted and unweighted formats, all official statistical publications lead with the weighted data. Ideally, the patients who return a GP Patient Survey should be representative of the patient population (patients aged 16+ who are registered with a GP). However, some groups may be over or under-represented in the final sample, for example certain groups may be more or less likely to respond. Weighting is used to adjust the data to account for the differences between all patients aged 16+ who are registered with a GP, and the sub-set of patients who actually completed the questionnaire. It is therefore recommended that weighted data is used when looking at GPPS results to compare practices, CCGs or other subgroups.
Where can I learn more about the rationale for the current weighting scheme?
In Summer 2011, Ipsos MORI undertook a weighting investigation to find out if the weighting scheme could be refined and developed further to help improve the accuracy of the results. The investigation was conducted on the previous years’ data (i.e. fieldwork conducted between 5th April 2010 and 7th April 2011). The results of the investigation concluded that when neighbourhood statistics such as ethnicity and deprivation are accounted for, non-response bias is reduced, thus improving the accuracy of findings.
Please click on the link below for a summary of the rationale for the adjustment to the weighting scheme. More detailed information about the weighting scheme is included in the latest version of the technical annex.
Summary of weighting strategy for 2011-2012
PLEASE NOTE: results for the 2011-2012 survey onwards use the new weighting scheme. The changes to the weighting scheme, as well as changes to questionnaire design and survey frequency, mean that it is not possible to make direct comparisons with previous years’ data, even in cases where the same questions have been asked.
Why has the weighting scheme been adjusted?
The weighting scheme was adjusted in 2011-12 to make the data better represent the views of the population as a whole. They now take into account local factors (such as deprivation, crime levels, ethnicity, marital status, overcrowding in households, household tenure and employment status).
The changes to the weighting scheme in 2012, as well as the questionnaire and survey frequency, mean that it is not possible to make like-for-like comparisons. This means that when looking at historic data, even in cases where the same questions have been asked in previous years, direct comparisons cannot be made.
