Last chance saloon: preserving the integrity of the sex question in the 2021 census

Last month, following a parliamentary evidence session on the 2021 census, the Scottish Parliament’s Culture, Tourism, Europe and External Affairs Committee (CTEEA) Committee wrote to the National Records of Scotland (NRS), asking for further clarification on several points. This blog discusses the NRS response to the Committee, as it relates to the sex question and highlights a number of concerns. The full NRS letter may be accessed here.

1.  NRS current position on the sex question

Draft sex question and accompanying guidance, NRS, September 2019

The key statement in the NRS response is that its statistician group recommends that the sex question in the 2021 census should be based on self-identification and that this justifies their proposed accompanying guidance stating that people can respond to the sex question based on their self-identified gender (see above). NRS state that this is based on a number of factors, including user needs, previous testing and stakeholder feedback, as discussed below.

‘The recommendation from the National Records of Scotland’s Government Statistical Services statistician group continues to be that self-identification provides the best balance in meeting the diverse range of user needs across the full Census dataset. This recommendation is based on results of previous testing, approaches taken across the UK, and the feedback from discussions with stakeholders and peers. We have recently commissioned further independent testing by ScotCen which will provide valuable information on how respondents require and use guidance. This work will report in late December 2019.’ (Our emphasis)

2.  ‘User needs’

The NRS response indicates that guiding respondents to answer the sex question based on their self-identified gender provides the best balance to meet a ‘diverse range of user needs.’ To the best of our knowledge, the argument for a sex question that equates to respondents’ self-identified gender is based only on the needs of some respondents. For example, activists have argued that asking respondents to answer the sex question based on their biological or legal sex will ‘roll back trans rights‘. Given the risks to data reliability and fudging of two categories in the same question, we think it is unlikely that this approach will meet user needs in any way that is preferable to a question based on clear criteria, particularly given the introduction in 2021 of a separate question that will ask respondents about their gender identity.

3.  Previous testing

The NRS response states that the current recommendation takes into account previous testing. To illustrate this, NRS present findings based on earlier cognitive testing, which appears to suggest that people (both members of the general population and members of the trans community) had different understandings of the sex question:

‘Cognitive testing provides an insight into the mental processes respondents use when answering survey questions, helping to identify problems with question wording and questionnaire design. This approach was taken to provide an understanding of the acceptability, understanding and ability to select an answer.

The first question tested was ‘What is your sex?’ with a set of binary sex response options (i.e. ‘Male’ and ‘Female’) and no guidance was provided. Results showed:

Whilst general population respondents had mixed understanding of the term ‘sex’ all were able to provide an answer to the binary sex question during the cognitive interviews. Their understanding of the term sex did not affect the answers they gave in any way.

Trans respondents also had mixed understandings on what was meant by the term ‘sex’. However, for this group understanding of the term did impact on their response given and whether they answered the question at all.’

However, this evidence provides only a small snapshot of the previous testing, and is based on a very small sample: the cognitive testing consisted of two strands, with just 35 participants drawn from the general population and 23 participants from the LGBT community. As independent researcher Susan Sinclair has documented, the NRS Sex and Gender Identity Topic report also shows that many participants were unaware that the sex question was supposed to be based on self-identified sex.

“From earlier testing it was apparent that trans and non-binary respondents were unaware that the sex question allowed them to answer with their self-identified sex instead of their biological sex.”

“When shown the question on sex in isolation, respondents interpreted the word ‘sex’ as biological sex.”

“The majority of the respondents interpreted the word ‘sex’ in … the question as biological sex.”

“Some respondents assumed sex referred to ‘biological sex’ or ‘assigned sex at birth’.”

“The understanding of ‘sex’ as biological occurred in both the general population respondents and the trans respondents. Sex as biological was the dominant understanding amongst trans respondents.”

“It should be noted that this group understood the term sex to refer to biological sex or assigned sex at birth. This was not the intended definition of the sex question using current NRS measurement objectives.” (our emphasis)

4. Stakeholder views

The Scottish Government statisticians group recommenation appears to be at odds with independent expert academic opinion. In September 2019 a group of independent senior social scientists and census data users wrote to the CTEEA Committee expressing their concerns about the impact of a sex question which is based on self-identified gender:

‘With a self-identification approach, our ability to monitor sex-based discrimination and disadvantage would be reduced, and to a degree which we cannot assess since the trans identity question is voluntary.’

In relation to the introduction of guidance which advised respondents to answer the sex question based on their self-identified gender in the 2011 census, the same independent group further noted that this was brought in without proper scrutiny or consultation:

‘The Equality Impact Assessment for the 2011 Census contains no discussion of this change, however, and there was no consultation on this advice that we are aware of. It cannot be seen as a precedent for what is being proposed now.’

Overall, the recent NRS response to the CTEEA Committee provides insufficient evidence of equal engagement with all perspectives. As we have previously noted, during the formative question development phase, NRS only consulted with LGBT groups on the sex question. A more recent Freedom of Information response detailing correspondence between the Equality Network (which advocates for a sex question based on respondents self-identified gender) and the NRS also suggests a close working relationship between the two organisations.

5.  Equality Act 2010

NRS state that they recognise ‘the critical importance of the Equality Act 2010 and agree with the committee that it is the cornerstone of UK-wide equalities legislation’, and that the ‘NRS is fully committed to the Act and wish to ensure Scotland’s Census 2021 is exemplar in this area’. More specifically NRS state that the 2021 census will provide:

‘comprehensive, high quality anonymised statistical population estimates, enabling accurate funding allocations to local and national services. Specifically on sex, collecting information on men and women will be valuable to service providers to ensure that appropriate services can be provided which leads to better outcomes for both men and women.’

We do not think that this statement is consistent with the current recommendation, which muddles data on sex and gender identity, and so does not reflect the protected characteristic of sex. While NRS note that census questions need not mirror the categories in the Equality Act as a matter of law, we would suggest that if census data is intended to inform equalities monitoring, then the best way to achieve this for the protected characteristic of sex is to align the census question with the Act.

We are aware that the National Records of Scotland have been advised by the Equality Network that the protected characteristic of sex extends to what campaigners increasingly refer to as ‘lived sex’ because it is possible that a person may be discriminated on the basis of their perceived sex. Any person, whether transgender or not, has protection under the Equality Act from being discriminated against on the basis of the sex they are perceived as, whether or not that perception is correct. But this does not mean that the general definition of the protected characteristic extends to cover any person who may, or believes they may, be perceived as the opposite sex. The protection against discrimination by perception covers all the protected characteristics (except marriage and civil partnership, and pregnancy and maternity), as an additional safeguard against discriminatory behaviour based purely on the motivation of the person doing the discriminating. It is relevant only when a particular act of discrimination takes place based on a misbelief and does not mean that a person who is, or believes they could be,  incorrectly perceived to have a protected characteristic (for example, to be disabled or have a particular religious belief) acquires that particular protected characteristic more generally.

6.  Data quality and reliability

Given the emphasis on data reliability in the UK Statistics Authority Code of Practice, we are surprised that a statisticians’ group has recommended an approach that actively reduces the reliability of census outputs by introducing additional uncertainty.

We also think that the current approach is not consistent with the underpinning data principles recently published by the Scottish Government Working Group on Sex and Gender data (of which NRS is a member). These state that ‘risks to data quality should be minimised’ and that ‘statistics should be consistent and comparable, while remaining relevant to society’. Given that the conflation of two distinct concepts in a single question reduces data reliability, while a lack of robust data on changes in the trans population poses risks in terms of consistency, it is arguable that the NRS recommendation meets neither standard.

The decision also goes against the recommendation made in Caroline Criado Perez’s book Invisible Women that we must meticulously collect sex-disaggregated data, and is not consistent with the statement by Cabinet Minister Shirley-Anne Somerville which recognised the need for disaggregated data: 

“We accept that sex and gender are distinct concepts. The Scottish Government agrees that there is a need to have disaggregated data to allow for the impacts of policies on men and women to be demonstrated.”

7.  Our view on the 2021 census

The current recommendation presents risks in relation to data reliability, and the ability to accurately monitor and track sex-based discrimination. More fundamentally, the approach represents a profound conceptual shift in how sex is defined. This would abandon the idea that the census can gather objective data on sex in line with its definition in the Equality Act 2010, where it is a protected category distinct from gender reassignment. Any redefinition of sex within the context of the census is also likely to be replicated in other data collection exercises. This introduces the risk that robust data on sex will be lost, either through its conflation with, or replacement by, data based on self-defined gender identity.

It does not appear that NRS has reached this position based on the careful analysis of data use and discussion with those with expertise in the collection and analysis of population data. Instead, all the available evidence suggests that a sex question which asks for respondents’ self-identified gender has been included in response to a request from advocacy groups who hold a particular, contested view on the irrelevance of physical sex relative to a person’s subjective sense of self. We are concerned that this decision will be to the lasting detriment of official statistics.

We believe the 2021 census should collect data on both sex and on trans status and identity, separately, to generate the richest, most high quality data on both characteristics, and that this means that the sex question should mirror the legal definition of sex under the Equality Act 2010.