A virtual reality app (VR) uses advanced display and immersive audio technologies to create an interactive, three-dimensional image or environment. An augmented reality app (AR), meanwhile, uses digital technology to overlay video and audio onto the physical world to provide information and embellish our experiences.
At the University of ĢĒŠÄlogo weāre crossing disciplines to collaborate on VR/AR innovations that will revolutionize how we , , heal, and as we work to make the world ever better.
What are new, innovative ways to increase community access to much-needed mental and behavioral health services?
Thatās the big question Michael Hasselbergās department chair asked him to ponder. Hasselberg, an assistant professor of psychiatry and clinical nursing, is acutely aware of the high demand, regionally and nationally, for behavioral health services and specialists to treat conditions like depression, anxiety, substance abuse, and eating disorders.
In this line of work, the gold standard treatment is cognitive-behavioral therapy. CBT is a form of psychotherapy that usually involves a therapist and patient working together to recognize and change the latterās thinking patterns while also developing coping skills, according to the . āItās our first-line treatment for several of the diagnostic clusters we treat, specifically the depressive and anxiety disorders, and thereās a lot of evidence to show its effectiveness,ā Hasselberg explains.
Given that a majority of Americans today own a smartphoneā77 percent, according to the āa web or mobile app seems like an obvious answer. Indeed, such apps abound.
āThe idea of putting cognitive-behavioral therapy on a computerized app isnāt new,ā Hasselberg says. Typically, such computerized cognitive-behavioral therapy (cCBT) features a voiceover with animation or perhaps a video of a therapist talkingābut no meaningful interaction.
āPatients download the app, but after using it once or twice, they disengage from it.ā
Virtual reality app offer cognitive-behavioral therapy
Could a more immersive, customized experience, one harnessing todayās advances in virtual reality and augmented reality (VR/AR), help keep patients engaged with an app?
Hasselberg pitched the idea a year ago during a meeting of the , a multidisciplinary campus collaboration that leverages computing, analytics, and technology to solve challenges in medicine. Afterward, two faculty members approached him: Jiebo Luo, professor of computer science in the Hajim School of Engineering and Applied Sciences, and Matthew Brown, professor of music theory at the Eastman School of Music.
Luo, whose research interests include machine learning, data mining, and personal analytics, has previously worked on a computer program that could . Brown, meanwhile, directs , a musical multimedia ensemble at Eastman, and has always ābeen very interested in the social impact and therapeutic functions of music.ā
Both were keen to bring their expertise to bear on such a project. Galvanized, Hasselberg tapped his colleague Wendi Cross, a clinical psychologist and associate professor in the Department of Psychiatry, to help develop evidence-based scripts and storyboards for a working prototype of the virtual reality app.
Cross immediately thought ahead to how patients would use such an app. āIn a typical psychotherapy situation, youāre interacting in the office for 45 minutes,ā she explains. āWell, nobody is going to stay online in an app for 45 minutesāpeople like to come and go in ways that suit them. With this virtual reality app, the psychotherapy session is literally in the hands of the patient.ā
According to Christopher Winders, a laboratory technician in the Department of Electrical and Computer Engineering, “These videos are being embedded into the mobile app and have our doctor [Lauren DeCaporale-Ryan, a clinical psychologist at URMC] laying out tenants of the CBT process. We filmed Lauren in front of a green screen and then I composited that footage into a 360-photograph of the office setting.”
A multidisciplinary collaboration yields a working prototype
Hereās how the virtual reality app works:
- Patients use their smartphones to access the app. They can see and hear the therapist, but can also look around the therapistās room, which was filmed using a virtual reality camera.
- During a session, the therapist might assign the patient a task, such as completing a questionnaire about anxiety levels.
- The patient completes the in-app questionnaire. The answers dictate which video of the therapist plays next.
Later, the therapist might suggest a mindfulness exercise in which the patient is asked to imagine being in a relaxing environment, such as a beach. - The patient is then shown environmental photographs and asked to select a preference. Earlier in the module, the patient was able to identify personal preferences for calming music.
- The appās algorithms then serve up the patientās preferred 3D environment (a beach, complete with the sound of waves) and accompanying music. Meanwhile, the virtual therapist talks the patient through the mindfulness exercise.
The next time the patient uses the app for an on-demand therapy session, the previous answers along with visual and aural preferences will be taken into account.
Evidence-based psychotherapeutic practices? Check.
Aesthetically pleasing and immersive video and music? Check.
Built-in algorithms that deliver personalized content based on your needs and preferences? Check.
And no special headsets required. āIf people have to use headsets like Oculus or even the cardboard headsets for mobile phones, that really limits the patient population we could reach with this technology,ā says Hasselberg. āAnd our goal is to touch as many patient lives as possible.ā
With a working prototype in hand, the teamās next step is conducting usability testing to fine-tune the technology. If this intervention proves useful with patient populations, the big-picture goal is having the in-app data synch with a patientās electronic health records.
āA common purpose is so importantā
With funding from Arts, Sciences & Engineering and the Medical Center, āwe formed a multidisciplinary team to create something that is otherwise not possible,ā says Luo.
āI canāt tell you how much Iāve enjoyed working with colleagues toward a common purpose, one that will have a social impact,ā says Brown. Cross agrees: āA common purpose is so important. Frankly, this project has been an adventure. Itās not what I do every day, but itās taking what I do every day and thinking about it in a brand new way with people I donāt usually work with.ā
And Hasselberg contends that collaborating with researchers and professionals outside of medicineāmusicians, computer scientists, data scientistsāhas made him into a better clinician and health care innovator. āIn the end, I think bringing these multiple perspectives to a problem will allow us to help more people.ā
