Ada - Speak Your Mind
A wearable companion that provides emotional guidance through conversation in everyday situations
JAMES PAI, Renee Chang, Tingwei Chang, Jessica Chow, Natalee Ouzts, Emily Phan
Photoshop, Premiere PRO, After Effects, Illustrator, DSLR, AUdacity
Ada is a personalized conversational user interface (CUI) that sits discreetly behind the ear in order to provide emotional and motivational support through verbal conversation. It is designed to help people overcome stressful everyday situations, facilitate self-reflection to increase mindfulness, and provide reassurance through feedback.
Rooted in research on anxiety and mental health, Ada borrow techniques from psychotherapy to enhance the wearer’s well-being by helping them tackle difficult situations one step at a time. Unlike traditional outlets for stress and anxiety, Ada is always accessible - even in public spaces where privacy is a concern.
Ada was a 10-week Microsoft Design Expo project that culminated in a ten-minute presentation to faculty, industry liaisons, and peers. The project involved ideation, research, storyboarding, conversation design, video prototyping, physical prototyping, and slide presentation.
All team members were involved in all stages of the Ada's design, from ideation to refinement. I was independently responsible for producing the video prototype (including all filming, editing, motion graphics, compositing, audio recording & editing). Another teammate and I also prototyped different versions of the physical device.
Achieving Symbiosis and the Conversational User Interface - Microsoft Design Expo 2016
"Design a product, service or solution that demonstrates the value and differentiation of the CUI. Your creation should demonstrate the best qualities of a symbiotic human-computer experience which features an interface designed to interpret human language and intent..."
Microsoft's Design Expo is an annual event in which top design schools around the world take on a design challenge. For an interaction design course focusing on this challenge, I formed a multidisciplinary team with 5 other students studying interaction design, visual communication, human-centered design and engineering, and human-computer interaction. Our project was one of six that came out of the course.
Our team began by conducting secondary research to explore the different existing and upcoming applications of CUI. We gathered several examples that we found to be particularly compelling and presented them as a slide deck. These gave us some inspiration for how we wanted to approach the problem and eventually steered us towards emotional wellness and self-improvement.
After our initial research, we began brainstorming using post-it notes on a whiteboard. Each of us came up with as many broad ideas a possible in a short amount of time and then stuck them on the board. The next step was to group our ideas under different categories by moving the notes around.
In order to narrow down our ideas, we each placed markers to vote on the ones we were most interested in pursuing and eliminated the rest. We were left with a few general topics: journaling and peripheral attention.
We stepped back again and individually generated numerous potential applications of these two themes and placed post-it notes on the board under the relevant theme. Again, these ideas were very diverse, so we grouped them into several themes followed by a second round of voting. We moved forward with the direction of self-improvement, reflection, and mental health.
From these brainstorming sessions, we ended up with the concept of a memory log that would allow people to store and retrieve memories through conversation. The memory log facilitates self-reflection to improve mental health. It was extremely challenging to reach a single concept that everyone was happy with, so we compromised by broadening the concept once again to push forward.
We began a third round of broad ideation, this time focusing on different aspects, characteristics, and features that a CUI might have in order to promote self-improvement, reflection, and mental well-being. We each wrote several post-it notes, categorized them, and voted. This session resulted in 7 concrete project ideas, which we discussed thoroughly before finally agreeing on the theme of self-improvement. Our concept was a contextually-aware CUI coach that facilitates self-improvement by putting mental state in context for self-reflection.
In order to develop our concept, we brainstormed potential problem spaces to which our concept could be applied. Some of these included stress and social anxiety, so we decided to conduct interviews to gain a better understanding. We interviewed a licensed clinical psychologist specializing in mental health therapy, as well as other students who struggle with anxiety.
People need a more accessible means of coping with stress and anxiety, because it is often difficult to get help when it is most needed.
- anxiety is a 'fight or flight' biological response
- college students are some of the most susceptible to anxiety and stress because of sheer workload / increased activities
- the therapist often tries to help individuals develop a gentler, less critical internal language
- most people find therapy useful because they need someone who is actively listening and engaged
- coping methods for anxiety differ for each individual
We were examined several existing CUI applications such as Siri, Cortana, and Google Now as well as fictional applications from movies such as Her, 2001: A Space Odyssey, and Big Hero 6. However, we heard many comparisons of our concept to Samantha (Her) in our early critiques, so we created a comparison matrix in a spreadsheet to define where our CUI solution fell and how it was differentiated from other examples. This helped us map out our design objectives for the interaction.
scenario Design objectives
- learns about the user through conversation and context
- facilitates self-reflection & journaling to help motivation
- accompany and help the user overcome challenges
- detect high-stress situations and reduce anxiety accordingly
- provide support via subtle cues when verbal communication is not an option
- provide reflection and reassurance after critical events
Scenarios + dialogue
The next step was to prototype the interaction by designing the dialogue aspect of our CUI, which we named Ada to help establish a sense of personal connection. We brainstormed several possible narratives and chose the scenarios surrounding a student's first job interview. Then, we came up with 7 scenes for the narrative to show how Ada is able to help someone overcome a significant obstacle and improve themselves through conversation. The narrative started out with an out-of-box experience and ended with a post-interview conversation. A week was spent writing and revising dialogue for each scene and we produced an experience storyboard with recorded dialogue. The scenarios communicated our design objectives and presented our storyboard to faculty and peers for feedback.
Based on the feedback we received, we made many changes to our concept. Most importantly, we eliminated a few scenes from the narrative that were not necessary to convey our idea, including the actual job interview scene. This particular scene was problematic because we wanted to show how Ada could provide support through nonverbal means, however it was difficult to show how this might be effective while also preventing the scene from feeling too 'magical'. For the sake of streamlining our concept and prototype, we also removed our initial idea of a physical device that could transform from an earpiece into a more subtle form factor (e.g. wristband). After creating a revised storyboard and gathering more feedback, we moved onto the live-action video prototype.
Since I had experience with video production, I took on the responsibility of filming the scenes and producing our video prototype. I completed the filming and editing for our rough cut over a weekend. We planned out the different shots we would need and then scouted for locations around campus to film. I used a Canon 60D to film and a wireless lapel mic to record the actress's dialogue. A shotgun mic was used to record ambient background noise to be overlaid.
After filming was completed, I compiled a rough cut with Premiere Pro by editing the best shots together. I used After Effects to animate the messaging interface in a scene in which Chloe is texting Ada. Audio was synced up and filtered for noise reduction. Since we didn't have a proper boom mic setup, I had to manually edit out noise artifacts. I also added sound effects in the form of a heart beating in order to show convey a sense of stress. Finally, I made adjustments for color correction and image stabilization.
In order to record the Ada's voiceover, we recruited a classmate and I used a Blue Yeti microphone in a small room to record her voice. I then chopped up the recordings in Audacity and ran filters for noise reduction and normalization. A slight chorus effect was applied to make the Ada sound different from voices in the scenes.
The rough cut was presented to industry liaisons, faculty, and peers. We received a lot of positive feedback regarding the quality of video production, but some major issues were the physical form of the device, waiting room location, and the heartbeat sounds from the waiting room. The physical form looked too bulky and unrefined compared to the rest of our work, the waiting room looked too sterile for an architecture firm, and the heartbeat sounds were too dramatic for our desired effect.
We then created a new physical device and reshot the necessary scenes. For the waiting room, we found a new location that suited the look of an architecture firm which worked much better than the previous location. In the final cut, I also changed the heartbeat sound to a more rhythmic soundtrack, added music to a couple other scenes, composited a heartbeat interface, and added a motion-tracked LED to the device with After Effects.
When exploring the physical form of our CUI, we initially wanted the device to be worn around the ear but also transform into a wristband. Since we scrapped the idea, we focused on an earpiece that could make contact with skin, provide audio through bone conduction, and maintain a low physical profile.
Our first version of the device was constructed from a wooden button covered with aluminum foil. This ended up looking too bulky and rough, so another teammate and I began designing a new version that would be smaller, higher fidelity, and more aesthetically pleasing.
We used Rhino to create several variations that were then laser cut out of birch. These versions had a small dot for better motion tracking in After Effects, but the device still looked too wooden and very much like a prototype.
For the third and final versions of the prototype, we covered the birch with white electrical tape before laser cutting in order to give the device a smooth white finish. We also wanted to take product photos of the device so we laser cut a hole just large enough to hold a small segment of clear plastic pigment. This would allow an Arduino powered LED to shine through the opposite site and give the appearance of a built-in LED. We then cut a tiny hole on a white surface and lined it up with the prototype. The results were surprisingly successful.
We compiled a slide deck for our final presentation to accompany our video prototype. The presentation was 10 minutes and delivered to Microsoft liaisons, faculty, and peers. Overall, I thought our presentation was very successful at conveying our design objectives for Ada, and our video prototype was able to show our design scenarios in a professionalnarrative-driven way.
Feedback was very positive and the audience praised the quality of our video and presentation, as well as our concept which had come a long way since our initial experience storyboards.
While we had the most difficulty agreeing on a project direction early on (at times it felt like we were going backwards), we pushed on with ideation and I think the end result is a concept we all found to be compelling. I also learned quite a lot about the importance of team and project management in such a project involving multiple designers with different mindsets.
I'm extremely proud of what my team was able to accomplish in a such fast-paced timeframe, even if our project wasn't chosen to attend the expo. I gained a substantial amount of experience designing an experience through scenarios and conversational dialogue and have a newfound appreciation for current applications of CUI. I was also able to make use of my experience in video production and motion design, while also teaching myself how to deal with audio properly.