Frequently asked questions
Completing the survey
- General questions
- The online survey
- The paper survey
- Confidentiality and data protection
What is weighting? Why do you weight the data?
Weighting adjusts the data to account for differences between all patients at a surgery and the sub-set of patients who actually complete the questionnaire.
For example: what if one GP surgery has many more 18 to 24-year-old patients than 75 to 84-year-olds, but only a small number of 18 to 24-year-olds actually fill out the survey? By applying more weight to the responses from 18 to 24-year-olds, the results for a practice will more accurately reflect the views of the practice population as a whole.
How does the weighting actually work?
One of the problems with surveys is non response, which can cause some groups to be over or under represented. In order to correct for this weighting is applied. Persons in underrepresented groups get a weight larger than 1, and those in over represented groups get a weight smaller than 1.
Using the example above: if a lower proportion of 18 to 24-year-olds returned questionnaires (5%) than the proportion of 18 to 24-year-olds within the practice (10%). This means that 18 to 24-year-olds would be under-represented in the survey results. A ‘weight’ for responses from the 18 to 24-year-old group can be calculated by dividing the practice proportion (10%) by the response proportion (5%). Responses from 18 to 24-year-olds are then multiplied by the weight (2.0) to increase the influence of these responses in the final results, thus counteracting the low representation. A weight of less than 1 can also be calculated when there is a higher proportion of responses than the practice proportion to decrease the influence of these responses in the final results.
The example above only considers age as a factor when determining the weight. The weighting scheme for the GP Patient Survey includes multiple factors such as age, gender, geo-demographic classification (ACORN), Government Office Region and a number of area level characteristics including: deprivation, crime levels, ethnicity, marital status, overcrowding in households, household tenure and employment status.
More detailed information about the weighting scheme is included in the latest version of the technical annex.
Why should I look at the weighted data rather than the unweighted data?
While GPPS data is available in both weighted and unweighted formats, all official statistical publications lead with the weighted data; weighting ensures results are more representative of the population of adult patients registered with a GP, and this is the default option when viewing results through the GPPS website. Weighted data is useful for practices where fewer patients of a certain group (for example, younger patients) have filled in the survey than we would expect.
The unweighted data is raw, unadjusted data. It’s useful if you care about seeing individual responses, but less representative of how all patients at a practice might feel, and therefore also less useful for accurately comparing different organisations.
Why do you recommend using weighted data to look at GPPS results over time?
Changes were made to the mailout strategy used for the survey as part of fieldwork in July-September 2015. The changes involved redesigned cover letters being included within each full survey mailing, and a postcard reminder being sent to all sampled patients one week after the first survey pack mailing.
Analysis carried out by Ipsos MORI shows that there are some small but statistically significant changes to the unweighted profile of patients responding to the survey as a result of these changes to the mailout strategy. This change in the profile of responses means that unweighted results from January 2016 are not directly comparable with results from previous publications, even where questions remain the same.
As mentioned above, while GPPS data is available in both weighted and unweighted formats, all official statistical publications lead with the weighted data, and weighting ensures results are more representative of the population of adult patients registered with a GP. Analysis carried out on the weighted results shows that there are no changes in the weighted profile since the refinements were made to the mailout strategy. This means that weighted results from January 2016 are still comparable with results from previous publications.
Given these reasons, it is recommended that weighted data is used when looking at GPPS results over time.
More details on these changes and the impact on January 2016 results can be found here.
Where can I learn more about the rationale for the current weighting scheme?
In Summer 2011, Ipsos MORI undertook a weighting investigation to find out if the weighting scheme could be refined and developed further to help improve the accuracy of the results. The investigation was conducted on the previous years’ data (i.e. fieldwork conducted between 5th April 2010 and 7th April 2011). The results of the investigation concluded that when neighbourhood statistics such as ethnicity and deprivation are accounted for, non-response bias is reduced, thus improving the accuracy of findings.
Please click on the link below for a summary of the rationale for the adjustment to the weighting scheme. More detailed information about the weighting scheme is included in the latest version of the technical annex.
PLEASE NOTE: results for the 2011-2012 survey onwards use the new weighting scheme. The changes to the weighting scheme, as well as changes to questionnaire design and survey frequency, mean that it is not possible to make direct comparisons with previous years’ data, even in cases where the same questions have been asked.
Why has the weighting scheme been adjusted?
The weighting scheme was adjusted in 2011-12 to make the data better represent the views of the population as a whole. They now take into account local factors (such as deprivation, crime levels, ethnicity, marital status, overcrowding in households, household tenure and employment status).
Why won’t it be possible to make direct comparisons between previous years’ data and data from 2011-2012 onwards?
Changes to the weighting scheme, as well as the questionnaire and survey frequency, mean that it is not possible to make like-for-like comparisons. This means that even in cases where the same questions have been asked in the new survey, and in previous years, direct comparisons between results from 2011-2012 onwards and previous years' results cannot be made.