Final year project using Microsoft HoloLens and Self-Attachment Therapy
Guest post by Nana Asiedu-Ampem Master of Engineering (MENG) student, Imperial College London
Introduction
Psychotherapy is a form of treatment aimed at improving the mental health of people suffering from mental illnesses such as depression and anxiety disorders. These kinds of treatment usually involve a patient suffering from a mental illness discussing their feelings and emotions in a way that allows them to be better equipped to control them. This project focuses on a type of psychotherapy known as Self-Attachment Therapy (SAT) (1) with the aid of Augmented Reality (AR).
Mental illness is often linked to a person's lack of secure attachment during their childhood (e.g. uncaring and unreliable parents) which has affected their ability to form meaningful connections that can help them overcome their illness. Self-Attachment Therapy aims to help patients suffering from mental illness form a secure attachment with themselves so they become more self-reliant when it comes to controlling their emotions. The patient initially takes the role of the "adult-self" who is perceived as a primary caregiver to an imaginary child that embodies the person's "inner-child". The adult-self represents the rational part of the patient as well as their strengths. The inner-child is a representation of the patient's emotions, insecurities and vulnerabilities. It is up to the patient, who is imitating the primary caregiver for the imaginary inner-child, to comfort and show affection towards the inner-child to make them feel loved and valued. This is mainly done through the patient verbally complimenting their inner-child. Through the process of the patient comforting their inner-child, they form an important bond with them that they can use to overcome their troubles.
The project involves making an application using the Microsoft HoloLens that performs the Self-Attachment Therapy with a patient by allowing them to visually view their inner-child through a 3D avatar model. The 3D avatar will be perceived as a hologram placed in a position set by the application in the space around them. The hologram will display several key features of a child such as their face, hair and body. The patient would be able to view the emotions of the avatar as well as interact with it through speech. The avatar will initially show a sad emotion and frown, but through the interactions with the patient they can become happy and smile. This is part of the process of the patient forming a bond with their inner-child. The goal of the project is to provide a proof of concept of an application that can significantly improve the patient's engagement with SAT by using technology to make them feel more immersed in a safe environment where they can improve their mental health.
How it was implemented
The application was developed on the Unity game engine using the C# programming language to implement scripts. The app made extensive use of the Microsoft Mixed Reality Tool Kit. A 3D model of a generic child was created in Blender. It was then imported into Unity so it can be used as the holographic representation of the patient’s inner-child. The model was created by a colleague who was working on a similar project that creates child models based on the patient’s features.
The model has a number of animations which allows it to express various emotional states such as: happy, sad and scared. The model also has an animation that allows it to perform a dance.
SAT comprises of 4 stages that the patient has to perform to form a loving connection with their inner-child (Check the reference for more details on the stages (1)). To help the patient feel more immersed with the therapy, the inner-child was given spatial awareness on the patient’s location and their surroundings. Meaning the application places the inner child in an open space in the patient’s room with its shoes positioned on the floor. This was achieved by having the HoloLens scan the patient’s environment using the devices many sensors and the spatial mapping library provided by the Mixed Reality Tool Kit. This allowed the application to create an internal representation of the room’s floor which is used to place the inner-child on the floor. Also, 3D meshes that matched various objects in the patient’s room were created (such as tables and chairs). Unity uses ray casting to test collisions with these objects within the application. This was used to avoid placing the inner-child in positions that were obstructed with objects (e.g. a clear open space in the room will be found to place the inner-child, as opposed to placing it in the middle of a table). The inner-child is also constantly turning to face the patient. These features were implemented to make the inner-child feel like a more realistic addition to the user’s environment. This will make it significantly easier for the patient to perform the therapy stages.
How it will be used
Self-Attachment Therapy was developed by the project supervisor, Edalat Abbas. He created the project so he can research how well SAT works when combined with immersive technology. This project will be used in his research to help improve the process of the therapy.
Therapists will use the application with the HoloLens to help explain the process of SAT to their patients. Patients will be able to read vital information regarding the therapy through holographic boards that follow their line of sight. This will make it much easier for therapists to administer the therapy as the patient will already have access to all important information through the application itself. Initially, SAT required patients to close their eyes and imagine a younger version of themselves that embodies their imaginary inner-child was in front of them. Some patients will struggle with consistently imagining a realistic inner-child which will hinder their ability to form a meaningful connection with them. This will be a big challenge for therapists who will find it difficult to demonstrate how their patient's inner-child should look and how they should react during each stage of the therapy. However, the HoloLens alleviates these concerns from the therapist and patient by visually providing the patient with a representation of their inner-child. This makes it much easier for the patient to perform the therapy without having to constantly imagine a child they have to interact with. This will make the therapist’s work much easier as they no longer have to dictate the exact action the patient or inner-child has to perform at each stage.
Future projects
There is a lot of room for future extensions that will enhance the impact of the therapy. The application could make use of the Internet to allow a patient to incorporate content they enjoy with the therapy. For example, part of the therapy has the patient sing a soothing song to relax their inner-child. The Internet can be used to play an instrumental version of a song the user enjoys to help them through the process. The therapy could also benefit through the use of a more comprehensive spatial mapping process. Currently, the room is scanned once before the therapy begins. This can be updated to allow the application to scan the patient’s environment in real-time providing the application with a more detailed representation of the patient’s surroundings. This can then be used to augment the user’s room further by adding holograms that interact with real world objects. For example, the patient could download a photo they like from the Internet and turn it into a photo frame they can place on their wall. These extensions are intended to make the experience of performing the therapy with the HoloLens more enjoying, thus improving the patient’s motivation to carry out the therapy.
References
1: Self-attachment: A self-administrable intervention for chronic anxiety and depression https://www.doc.ic.ac.uk/research/technicalreports/2017/DTRS17-3.pdf