Amina Abbas-Nazari

Human computer interactions increasingly incorporate the voice as a key mode of communication, with expanding proliferation of conversational artificially intelligent (AI) systems, such as Amazon Echo. This thesis investigates practices of profiling humans from the sounding of their voices (Singh, 2019) and using profiling as frameworks for the sound design of synthesised voices (West, Kraut & Chew, 2019 and Nass & Brave, 2005) in conversational AI systems.

Ethical implications of these practices are explored through their reliance on and reinforcement of normative expectations (Amaro, n.d.) and by means of categorising and labelling of human traits by AI analysis, which ultimately brings harm to already marginalised people (Birhane, 2021). A metaphor of a ‘vacuum’ is used to describe current understandings of voices in conversational AI systems to foreground how currently it neglects the materiality of voices, building on Eidsheim (2015, 2019, 2014). This thesis argues that a voice is always multiple, as it is material and has materiality, and should be imagined as part of a choir, forming a constituant part of a whole, which defines the research methodology.

I define the concepts of polyphonic embodiment(s), and speculative voicing as practice led methods which function to critique current profiling practices, while exploring alternative frameworks for understanding and presenting voices in conversational AI systems.

To summarise Polyphonic Embodiment(s), one voice in conversational AI systems can:

1. be embodied and co-created with other matter
2. be embodied and co-created with many other bodies and voices, such as in a choir
3. embody many voices, shaped by the body or instrument
4. embody many disembodied voices, shaped by the conversational AI system

This thesis is situated at the intersection of sound, design and technology and utilises speculative methods building on Oliveira (2016) which combines Design Fiction (Dunne & Raby, 2013) and Sonic Fiction (Eshun, 1998). The written component contextualises the practice which tests the propositions through predominantly oral and aural based practice works and experiments. The written theoretical enquiry feeds into the practice, while the practice functions as thought-experiments enacted and illustrations of alternative frameworks through which to understand and present voices in conversational AI systems. In my practice projects I explore speculative voicing as materially contingent, whereby human and non-human voices become matter shaped by the body, technology and environment in conversational AI systems. Within the practice my own voice functions as a testing ground to experiment with ideas and present projects. The body of practice work centres around two major projects. In addition I’ve conducted a workshop series (three workshops, plus a reworking of the workshop as an interactive telephone experience) and a number of smaller experiments.

Research Questions:

How can the materiality of voices be used to critique practices of profiling in conversational AI systems, through practice?

How can the materiality of voices be used to present alternatives to practices of profiling, for the sound and sounding of voices in conversational AI systems, through practice?

How can this understanding of voices be used to highlight the voice for speculative design and sonic practice?