Technical annex Download

Contents

9 Data quality satement

9.1 Purpose

This data quality statement aims to provide users with an evidence-based assessment of the quality of the data used in the GP Patient Survey. It reports against the European Statistical System (ESS) quality dimensions and principles appropriate to this output.

In doing so, this meets NHSE England’s obligation to comply with the UK Statistics Authority (UKSA) Code of Practice for Offical Statistics particularly Principle Q3 which states that ‘Producers of statistics and data should explain clearly how they assure themselves that statistics and data are accurate, reliable, coherent and timely’.

9.2 Relevance

The degree to which a statistical product meets user needs in terms of content and coverage

The survey is designed in collaboration with a steering group consisting of members from national healthcare organisations, patient groups, charities, ICBs and GP practices, plus people with lived experience. The survey is regularly tested with a range of users, in particular when new questions are included.

The outputs are used by a wide range of users such as:

  • NHS England and Department of Health and Social Care, to monitor the experience of patients and in policy development.

  • ICBs, PCNs and GP practices, to understand aspects of patient experience in their area and how these have changed over time.

  • Charities, to understand how experiences vary across different patient groups.

  • Academic researchers, to inform research into aspects of primary care experience and how these vary across patient groups.

9.3 Accuracy and reliability

The difference between an estimated value and the true value

The figures in this publication come from a survey which gathers information from a sample rather than from the whole population. The sample is designed to be as accurate as possible given practical limitations such as time and cost constraints – see Chapter 3: Sampling for a description of how the sample is drawn, and Chapter 6: Data analysis on how this is accounted for in the presentation of the results. Results from surveys are always estimates, not precise figures. This can have an impact on how differences in the estimates should be interpreted.

As the number of people available in the sample gets smaller, the variability of the estimates that we can make from that sample size gets larger. Estimates for small groups are less reliable and tend to be more volatile than for larger aggregated groups.

To quantify the uncertainty around point estimates, 95% confidence intervals are presented in the outputs (see section 6.7 for more information).

Another aspect of accuracy comes from the quality of the data collected in the survey and subsequent quality assurance, which is described in general terms elsewhere in this document.

In particular we are reviewing the accuracy and reliability of two specific questions:

Which of the following best describes you?

  • Male

  • Female

  • Non-binary

  • Prefer to self-describe

  • Prefer not to say

Is your gender identity the same as the sex you were registered at birth?

  • Yes

  • No

  • Prefer not to say

These questions were introduced in 2021 after full testing and significant engagement. They were intended to be an inclusive way of capturing gender identity and a proxy for gender reassignment, while continuing to collect information on the protected characteristic of sex, since a combination of the two questions was designed to also be used to derive sex (see the survey FAQs on the website for more detail). However, there are indications that some responders are answering these questions in potentially unexpected ways, and we are carrying out an in-depth review before drawing conclusions and making future recommendations. In the meantime we have improved how we label these questions in the reporting outputs in order to improve accuracy and to help users interpret the data.

9.4 Timeliness and punctuality

Timeliness describes the time between the date of publication and the date to which the data refer. Punctuality describes the time between the actual publication and the planned publication of a statistic.

The survey is collected and published annually. The publication contains responses that were received over a three-month period, with questions asking respondents to reflect on their experiences over a longer time frame. The complexities and cost of running an in-depth, representative, robust survey such as this mean that it is not currently possible to run the survey more frequently than once a year.

The publication is released several months after the survey closes to allow for data processing, analysis and quality assurance. In 2024, some outputs were staggered and released later; however this is not the case this year and it has been possible to release all outputs at the same time which will benefit users.

The release is in line with pre-announced publication dates.

9.5 Accessibility and clarity

Accessibility describes the ease with which users can access data. Clarity describes the quality and sufficiency of metadata, illustrations and accompanying advice

This publication is available in a bespoke website and is a combination of HTML web pages and Word and PowerPoint documents for description, charts, and graphs, with data provided in usable formats, such as Excel spreadsheets, csv files and web-based report building dashboard tools. The publication includes many of the Government Statistical Service recommendations on improving accessibility of spreadsheets for users.

The survey questionnaires and fieldwork communication materials are shared alongside the published results. This technical documentation describes the methodologies used to generate the survey sample and the subsequent results.

9.6 Coherence and comparability

Coherence is the degree to which data derived from different sources or methods, but that refers to the same topic, is similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level

The area of patient experience of GP practice is also explored in the ONS Health Insight Survey (HIS). This is conducted monthly and uses different methods and questions, therefore GPPS and HIS cannot be directly compared. Nevertheless there are some similarities in the questions that are asked, and statisticians and stakeholders from both surveys are in regular contact.

GPPS has been running for many years, however changes in questions asked, how the survey is administered and underlying methodology mean it is not possible to compare data across the full lifetime of the survey. Comparing across different domains is possible, taking into account the inherent uncertainty in the results and associated confidence intervals (see Accuracy and Reliability section).

9.7 Trade-offs between output quality components

The extent to which different aspects of quality are balanced against each other

Service delivery, national policy and survey administration methods can and do change over time and therefore it is necessary to revise both the questions and how the survey is delivered. However, changes of this nature are likely to break trends and therefore they must be balanced against user requirements for time series data. In practice this usually means large scale changes across different aspects of the survey are introduced in one go, and approximately every 5 years, in order to prevent more regular disruption to the time-series.

A survey of this scale and complexity requires a relatively long time period to administer, analyse and change, and this does come somewhat at the expense of timeliness. However, this is balanced against the depth and robustness of the resulting data.

Completion of the survey and all questions within it are voluntary and self-completed. Support is provided in a variety of ways to help respondents to complete as accurately as possible (see 4.2: Support for participants).