Technical annex Download
10 Appendix
10.1 Survey materals
The following survey materials are available on the website:
-
(note: questions 55 and 56 are currently under review by NHS England)
-
First reminder e-letter
In early January I sent you a letter asking you to take part in a survey to help improve local NHS services. Taking part will help you, your family and your community. Please click on the link for your invitation letter. Any questions? Please call Freephone 0800 819 9135 or go to www.gp-patient.co.uk/taking-part for more information. Thank you, Dr Neil Churchill, NHS England
-
SMS reminders
We recently sent you a letter about your experience of your GP practice. Please click on the link to take part in the NHS GP Patient Survey. You don't need to enter your log-in details. Questions? Please call Freephone 0800 819 9135. Thank you.
Second SMS
The NHS would welcome your feedback - there's still time to have your say. Please take part in the NHS GP Patient Survey. Any questions? Please call Freephone 0800 819 9135. Thank you.
Third SMS
Your last chance to tell us how your GP practice is doing. Click on the link to take part before the survey closes. Even if you haven't seen your GP recently, your feedback is still really useful. Any questions? Please call Freephone 0800 819 9135. Thank you.
Final SMS
Please tell us how your GP practice is doing. Take part before the survey closes. Questions? Call Free 0800 819 9135. Thank you.
10.2 Experiments
As part of the 2025 GPPS fieldwork, Ipsos and NHS England undertook a series of experiments to try to improve cost-effectiveness while maintaining response rates, as well as examining any changes in sample profile, survey estimates and costs, compared with the main survey design.
Experiments were carried out using a factorial design. Factorial design involves testing interventions in combination, rather than one at a time. Interventions can then be analysed by combining all groups involving that particular intervention. In addition, this has the advantage of allowing the impact of a combination of interventions to be analysed (e.g. potentially demonstrating that a positive impact on response rates from the introduction of a second e-letter instead of the final paper mailing and a separate positive impact from sending an email still applies when the two are combined).
This led to a series of experiments on a sub-sample of GPPS respondents, testing the impact of the following interventions:
-
Replacing the final paper mailing with an e-letter: designed to reduce survey costs by decreasing the volume of printing and postage and encouraging more people to take part online.
-
Removing the questionnaire from the final paper mailing: designed to reduce survey costs by decreasing the volume of printing and encouraging more people to take part online.
-
Extra contact by email: providing an additional way of reminding people to take part, who may have otherwise waited until the next reminder (postal or SMS) or who may not have taken part at all.
-
Contact by Rich Communication Service (RCS) message: RCS messages are an upgrade on traditional SMS and MMS, offering a more engaging messaging experience, including the ability to use branding, richer content and sender verification. RCS also offers features which may increase engagement and reduce costs, such as improved security and no character limits
It was hoped that additional experiments could be run, including working with NHS Notify and using the NHS App. However, the legal direction for research had not been approved for Notify or the App by the time of fieldwork.
10.3 Allocation of sample to treatment groups
With the total sample of 2.72 million cases, roughly 54,500 cases were systematically selected to take part in the various experiments (using the method of random start and fixed interval). The sample was stratified by practice, with a ‘1 in n’ random selection. The 54,500 cases were randomly allocated to each of 11 treatment groups, with the rest of the sample receiving the ‘control’ approach.
10.4 Results
Analysis was carried out looking at the interventions across the different treatment groups (for example, looking at all the groups that had the e-letter) to look at the differences for each intervention compared with the groups that had not experienced that intervention (for example, looking at all the groups that did not have the e-letter). This concluded that:
-
In the under 70s, replacing the final mailing with an e-letter or removing the paper questionnaire had a small negative impact on the response rate (1-2 percentage points). However, for those aged 70+, the negative impact was much more sizable (around 10 percentage points).
-
Adding an additional email contact had a small positive impact on the response rate for groups that did not receive a paper questionnaire (1-3 percentage points). However, there was no noticeable impact on groups that received a paper questionnaire.
-
RCS had minimal impact on the response rate (associated with an increase of about half a percentage point). However, this may have been impacted by the practical limitations of RCS for Apple handsets during fieldwork, which would need to be reviewed.
Further analysis is planned to assess the implications on cost, data quality, and non-response bias of implementing any of these changes. The non-response bias analysis will look at any differences in how respondents answer survey questions, as well as differences in demographic profiles of respondents and variation in response rates between demographic groups.
Analysis has already been carried out to look at the differences in results between the experiment groups and the main survey across a range of key survey estimates and demographic profiles. This found that there would be no impact on the results from including the experiment groups within the overall responses. Responses to the experiment interventions have therefore been included within published results for 2025; they equate to 1.9% of the total number of returned questionnaires.
Our standards and accreditations
Ipsos’s standards and accreditations provide our clients with the peace of mind that they can always depend on us to deliver reliable, sustainable findings. Moreover our focus on quality and continuous improvement means we have embedded a ‘right first time’ approach throughout our organisation.
ISO 20252
This is the international market research specific standard that supersedes BS 7911/MRQSA and incorporates IQCS (Interviewer Quality Control Scheme). It covers the five stages of a Market Research project. Ipsos UK was the first company in the world to gain this accreditation.
Market Research Society (MRS) Company Partnership
By being an MRS Company Partner, Ipsos UK endorse and support the core MRS brand values of professionalism, research excellence and business effectiveness, and commit to comply with the MRS Code of Conduct throughout the organisation & we were the first company to sign our organisation up to the requirements and self-regulation of the MRS Code; more than 350 companies have followed our lead.
ISO 9001
International general company standard with a focus on continual improvement through quality management systems. In 1994 we became one of the early adopters of the ISO 9001 business standard.
ISO 27001
International standard for information security designed to ensure the selection of adequate and proportionate security controls. Ipsos UK was the first research company in the UK to be awarded this in August 2008.
The UK General Data Protection Regulation (GDPR) and the UK Data Protection Act (DPA) 2018
Ipsos UK is required to comply with the UK GDPR and the UK DPA. It covers the processing of personal data and the protection of privacy.
HMG Cyber Essentials
This is a government-backed scheme and a key deliverable of the UK’s National Cyber Security Programme. Ipsos was assessment-validated for Cyber Essentials certification in 2016. Cyber Essentials defines a set of controls which, when properly implemented, provide organisations with basic protection from the most prevalent forms of threat coming from the internet.
Fair Data
Ipsos UK is signed up as a “Fair Data” company, agreeing to adhere to 10 core principles. The principles support and complement other standards such as ISOs, and the requirements of Data Protection legislation.
For more information
3 Thomas More Square London E1W 1YW
t: +44 (0)20 3059 5000
About Ipsos Public Affairs
Ipsos Public Affairs works closely with national governments, local public services and the not-for-profit sector. Its c.200 research staff focus on public service and policy issues. Each has expertise in a particular part of the public sector, ensuring we have a detailed understanding of specific sectors and policy challenges. Combined with our methods and communications expertise, this helps ensure that our research makes a difference for decision makers and communities.