1.4 Biometric Monitoring Technologies

Biometric monitoring technologies represent a somewhat ‘extreme’ technology for the purposes of this report – compared to say, teletherapy – given that the insights such technologies are purported to reveal about a person’s health, body, cognition, affective state and so on, create challenging ethical, social, legal and political issues.⁹⁸ Biometric monitoring technologies use sensors in devices, including smartphones, wearable and connected devices, cameras and even pills, to remotely generate data concerning a person’s biology, physiology or behaviour.

There are various ways to describe biometric monitoring technologies. Computer scientists may refer to ‘context sensing’, ‘personal sensing’, or ‘mobile sensing’. In mental health settings, several prominent psychiatrists and psychologists have begun to refer to ‘digital phenotyping’, particularly in relation to the assessment of behaviour, mood and cognition through biometric data generated by devices, such as smartphones and FitBits.⁹⁹ An advertisement for the prominent direct-to-consumer app company, Mindstrong, for example, describes this practice this way:

Proponents view biometric monitoring with therapeutic aims as a reasonable method for real-time tracking that in the right persons may enhance therapeutic alliance between mental health practitioners and individuals seeking help. This enthusiasm is typically accompanied by acknowledgement that some patients will not want to use nor gain from such measures. Similar forms of behavioural tracking holds appeal to actors in sectors outside of formal mental health services, including education, the military, the insurance industry, and the criminal justice system.¹⁰⁵

Biometric technology has also started to appear in video monitoring and surveillance in acute psychiatric settings, in ways that do not involve on-body sensors

Photo by Arno Senoner on Unsplash.

Other forms of biometric sensing go ‘beneath the skin’, where ingestible sensors have been integrated with psychopharmaceutical pills

Although biometric technology is relatively exploratory in the mental health context, its use is expanding. For example, the Oxehealth system used in the British trials is reported by the company who produces them to be ‘relied on by one in three English mental health trusts as well as acute hospitals, care homes, skilled nursing facilities, prisons and police forces in the UK and Europe’.¹¹⁵

1.4.1 Power and Justice in the Biometric and Digital Turn

The rise of biometric monitoring in mental health care is being debated on several fronts. This includes contested claims about what ‘digital markers’ of behaviour can reveal.¹¹⁶ Even the term ‘digital phenotyping’ is contested and terminology remains unsettled. David Mohr and colleagues raise concerns that the term fails to convey the reality that the practice constitutes surveillance over intimate aspects of a person’s life.¹¹⁷ Mohr and colleagues, who ultimately endorse the potential value of the technology, state:

The authors call for language that is more transparent about the intent and practice behind this technology, arguing the term ‘personal sensing’ is more appropriate.

Other commentators have drawn attention to deeper issues of justice and power. The use of biometric technologies to purportedly infer a person’s mental state or characteristics, and its use in pervasive forms of monitoring and surveillance, have raised particular concern.¹¹⁹ Leah Harris warns of biometric technologies developed by psychiatric or psychological professionals being used in forms of social control over marginalised individuals, not just in mental health settings, but also in prisons and other sites of carceral control, including in the ‘community’.¹²⁰ Harris refers to Michel Foucault’s theorisation of the Panopticon, discussing the way ‘power is based on both the ability to observe others and the knowledge obtained through that observation’.¹²¹ The Panopticon was originally an architectural system and idea developed in the eighteenth century by Jeremy Bentham. Its purpose is to continuously observe prisoners in confinement. For Foucault, panopticism is a surveillance mechanism used to exert disciplinary power throughout society by professionals, bureaucracies, government agencies, market actors, and so on, by allowing for an ‘absolute and constant visibility surrounding the bodies of individuals’.¹²²

Toward the end of his life, Michel Foucault conceptualised a shift in Western societies away from the dominance of disciplinary environments such as largescale psychiatric institutions, to systems of constant external surveillance. He wrote, ‘[o]ne also sees the spread of disciplinary procedures, not in the form of enclosed institutions, but as centres of observation disseminated throughout society’.¹²³ He charts these societal shifts toward forms of control that are less costly and complex to manage.¹²⁴ Harris relates panopticism to biometric monitoring in the mental health context, warning that ‘[t]here is always an inherent power imbalance between the “omnipresent” and “invisible” watchers and their “permanently visible” subjects’ and that such imbalances have been expressed in psychiatry historically through its role in governing a marginalised and oppressed group¹²⁵. Harris’s framing has commonalities with broader critiques of the information economy in the current era, including Shoshanna Zuboff’s prominent characterisation of ‘surveillance capitalism’.¹²⁶

How might surveillance capitalism operate in the mental health context? Various critical accounts have been offered. Lisa Cosgrove and colleagues’ state:

Examples of such data-extraction are included throughout this paper. Jonah Bossewitch warns of the ‘arrival of surveillance psychiatry’ and queries its role in the growing information economy, whereby ‘huge pools of data are being used to train algorithms to identify signs of mental illness’:¹³³

One challenge for advocates will be to correctly identify the business models of companies generating or processing such data. Without transparency on this matter, which companies will not necessarily divulge, observers may be left to speculate. One obvious business model would be targeting platform users with commercial products, as the next example suggests.

We will discuss private sector interests and the role of data concerning mental health in the information economy throughout the report.

Returning to biometric monitoring, others have raised concerns that people who use algorithmic interpretations of data concerning emotions are misled about the extent to which such systems can ‘capture’ the reality of emotional experiences.¹³⁹ Victoria Hollis and colleagues point to a survey of people (n=188) who showed strong interest in automatic stress and emotion tracking, where ‘many respondents expected these systems to provide objective measurements for their emotional experiences’ despite this simply not being possible.¹⁴⁰ This framing effect (which is often exaggerated by tech vendors) can even change the way people construe their own emotions. In another study, Hollis examined how algorithmic sensor feedback influences emotional self-judgments in a mixed-methods study with 64 participants.¹⁴¹ ‘Despite users reporting strategies to test system outputs, users still deferred to feedback and their perceived emotions were significantly influenced by feedback frames’ with some users even ‘overr[iding] personal judgments, believing the system had access to privileged information about their emotions.¹⁴²

Similarly, Lisa Parker and colleagues, in their survey of the messaging of mental health apps, argued that prominent apps tend to over-medicalise states of distress and may over-emphasise ‘individual responsibility for mental well-being’.¹⁴³ As a broad comment, the user/survivor/ex-patient movement and others have advanced reasons to demedicalise approaches to supporting people in distress; which would seemingly extend to caution about framing personal mental crises as medical problems amenable to digital technological solutions.¹⁴⁴ The framing effects of biometric monitoring often go unremarked, but the studies noted above suggest the effects can alienate people from their own self-perceptions. For their part, Hollis and colleagues argue that the framing effects of should be acknowledged and used in ways to promote agency and help individuals more actively construe their personal experiences.¹⁴⁵

Concerns raised by Harris, Bossewitch and others move beyond questions of how to make particular technologies like biometric monitoring more equitable or ethical (for example, by ensuring the datasets adequately cover diverse communities that accommodate distinct ways of being and self-presenting). Instead, their questions relate to law and political economy, questioning whether technologies are creating a market for surveillance in the mental health context that perpetuates and even extends the worst power imbalances, inequities and harms of current mental health practices.¹⁴⁶ Kaitlin Costello and Diana Floegel, for example, argue that the ‘link between the carceral state and mental healthcare in the United States is alarming’ and that biometric monitoring technologies ‘are poised to only further strengthen that link, despite calls to the contrary’.¹⁴⁷ More fundamentally, this new ensemble of AI and mental health looks set to change what it is to be considered well or unwell.¹⁴⁸

Moving the Frame from ‘What does the technology do?’ to ‘Who is benefiting and who is not?’

One analytical strategy to help counter these negative possibilities is to place the emphasis away from the technology itself and toward questions of who is benefiting from the push for these technologies, and – perhaps more importantly – who is losing. This framing challenges the common presentation of computational monitoring and evaluation as naturally being in people’s interests on the basis that ‘the more we know the more we can help’. Such an optimistic view can easily dovetail with widely-held understandings about the legitimacy and unquestioned benefit of monitoring persons experiencing distress, lived experience and disability. As Sharon Snyder and David Mitchell have argued, ‘[o]ne of the primary oppressions experienced by disabled people is that they are marked as perpetually available for all kinds of intrusions, public and private.¹⁴⁹

The broad group of critical commentators raising concerns with biometric monitoring draw attention to the potential intrinsic harms of processes of computational observation and measurement. Just as the ‘medical gaze’ has been used as a concept to critique the biomedical and individualistic framing of distress and other human experiences, some commentators have considered the potential harms of the ‘data gaze’. The ACLU, for example, describe a potential ‘nightmare scenario’ whereby a ‘data gaze’ extends to omni-present AI-powered monitoring and surveillance:

These concerns were not raised about the mental health context in particular, though they resonate with the concerns discussed in this section.

Others have raised concerns about the subtle harms caused by the way technological surveillance leads to an abstraction of the human body, which is then reassembled through a series of data flows.¹⁵¹ Jathan Sadowski has argued that the abstraction of ‘datafication’ is itself a form of violence.¹⁵² Extending these critiques to the disability context, Jackie Leach Scully and Georgia Van Toorn have argued that broader ‘datafication’ of the human body will delineate increasingly rigid boundaries between normality and disability.¹⁵³ This impulse to quantify and distinguish embodied difference, they argue, ‘diverts attention from the realities of disabled lives, at a time when disability scholars and activists are arguing for more rather than less attention to the lived experience of disability’.¹⁵⁴ LLana James, discussing algorithmic racism and the impacts of the digital turn on other marginalised groups, has discussed how datafication can undermine the need to ‘act on the reliable narrator’ (that is, listening to the person or populations affected and how they articulate their needs).¹⁵⁵ Instead, dominant narratives about technology insist on new and alternative ways to undertake expert observation and monitoring using data-driven technology.¹⁵⁶ In the disability context, including the mental health context, the use of automation risks diverting attention from the experienced reality of disabled lives.¹⁵⁷

If these concerns are taken seriously, the use of technologies like AI to make assumptions and judgements about who we are, and who we will become is much more than a potential invasion of privacy; it is an existential threat to human autonomy and the ability to explore, develop and express our identities. It is potentially a normalising of surveillance in a way that is reminiscent of 19^th century asylums as a state-authorised site of control over disabled lives, but using 21^st century techniques of ubiquitous observation and computational ‘processing’. Grappling with these possibilities will be a necessary part of discussion about the potential harms and public benefits afforded by technology in the mental health context in general, particularly biometric monitoring.

1.4.2 Governing the Future of Biometric Monitoring in Mental Health Settings

Biometrics more generally are the subject of a growing field of research, practice, advocacy, activism, and law reform.¹⁵⁸ In healthcare, the COVID-19 pandemic has accelerated the international adoption of forms of biomonitoring and surveillance, and other public health monitoring and security technologies, whether adopted by states, private entities or individuals.¹⁵⁹

In the mental health context, scholarship that explores the legal, ethical, social, and political concerns with biometric technologies is emerging.¹⁶⁰ More work is clearly required. Later themes discussed in this report will engage with some of the questions directly relevant to biometrics. Such questions include asking if those deemed through biometric monitoring to be ‘cognitively impaired’, ‘mentally disordered’, ‘suicidal’, or likely to become any of those things, will be informed that such attributions have been made. Will they be able to opt-out of the monitoring process in the first place? Will they be able to contest such labels before data are transferred to others? Given the purported ease with which mobile phone data-points can be used for automated profiling to determine cognitive impairment,¹⁶¹ are there sufficient safeguards to govern whether or how this should occur? More pointedly, should moratoria apply to some forms of biometric monitoring and surveillance in the mental health and disability context on the basis that they are fundamentally harmful or inconsistent with human rights? How would such a decision be made? What role is currently being played by psychiatric and psychological sciences in advancing such technologies? What role should they play?

This is a critical moment to reflect how the current choices being made in various institutions concerning ‘digital mental health’ – from research, services, policies and programming – might affect future approaches to distress, anguish, mental crises and so on. To conclude Part 1 we turn to the glaring omission from these choices of the very people for whom the technologies are purportedly designed