Moment
A mixed reality system that supports conversation and memory recall, which intends to help aging adults connect with others and preserve memories in the moment.
Role: UI/UX, Branding and Video production
Brief: Design a product, service or solution that demonstrates the value and differentiation of Mixed Reality
12 weeks group project
Sponsored by Microsoft Research
T H E P R O B L E M
T H E S O L U T I O N
As the population ages, cognitive decline is, and will remain, a pressing issue. Most existing tech solutions for older adults primarily address physiological and safety needs.
The way we preserve and enhance memory has evolved from analog to digital. Moment supports conversations and memory recall by providing relevant multimedia in real-time, transitioning from 2D to 3D formats. It functions like a voice assistant but replaces voice responses with visual results.
. . .
T H E A P P R O A C H
System that learns to facilitate a conversation
In Moment, the focus isn’t on the user conversing with the system. Instead, it relies on two users engaging in a natural conversation. As the system learns about the users, it facilitates their conversation by pulling memory triggers (media) into the feed.
T H E D E S I G N
Moment Interactions
To designate any flat surface as a workspace, the user can perform a "bloom" gesture or long-press the 'mic' button
To scroll through the feed, the user can either gaze at the arrow or swipe a finger left to right on the remote
To place media into the shared space, the user can gaze at the item, tap and hold, then drag it into the space using a gesture or the remote
To scale or rotate an item, the user can hold it and move their hands vertically or horizontally
To delete an item, user can gaze at it and swipe it in the direction opposite to the other person
At the end of the conversation, the user can save the items along with the conversation as a memory packet by pressing the save button on the controller
O U R F O C U S
Key Design Considerations
. . .
Isolating Private and Shared Spaces
To foster user autonomy and control, Moment offers separate private and shared spaces. These spaces allow users to take the driver’s seat, deciding what others can see while enabling collaboration in the shared space.
Leveraging Existing Practices
To introduce Moment early, we considered how users could onboard to the system using today’s technology, ensuring it starts with what feels familiar. By using it now, users can create a foundation that helps them recall their past more effectively in the future.
A browser extension and app for media collection
Design for All in Mixed Reality
To ensure accessibility for older adults, we designed a physical remote, guided by the belief that "there is security in the tangible." The grooves on the remote help users track direction on the pad by touch, while the microphone button serves as a start button, and the box button functions as a save button for conversations. To test how a physical remote would feel, we connected a tactile controller to the Hololens using Unity.
Additionally, we explored ways to maintain visual accessibility in Mixed Reality through tests on the Hololens. Our considerations included color, typography, form, and shading to ensure the best representation of visual assets.
. . .
H O W W E G O T H E R E
How Might Mixed Reality Be Leveraged to Help People Age Well?
We believe aging is a meaningful and impactful area to explore as an application for Mixed Reality. As technology continues to evolve, it is essential to design a world that is inclusive and accessible to this growing segment of the population.
With our key question in mind, we gathered our ideas about what humans need to age well into five areas of intervention. Then, we started mapping out various platforms, stakeholders, and opportunities for Mixed Reality in this space.
T H E R E S E A R C H
Cultural Probes
We began our research using qualitative methods. Postcards were used to ask Baby Boomers about their relationship with technology. Additionally, we interviewed aging experts, caregivers, and adults ranging from Boomers to seniors to gain insights into how we should design for aging populations.
OPPORTUNITY SPACES
As a result of our exploratory research, we identified five gaps that present opportunities for Mixed Reality to assist people as they age:
KEY CHALLENGES
We recognized three challenges surrounding our problem space in relation to mixed reality.
Can Mixed Reality be more than Head-mounted display?
Current head-mounted display (HMD) technology is heavy, and in some ways, even makes a person temporarily disabled.
How can we reframe aging to be positive?
There are also many negative biases surrounding aging. They are difficult to solve as we try to make them one might look forward to.
How can we design technology that fails gracefully?
Boomers view technology as a tool. When something goes wrong, frustration ensues and trust in the technology diminishes.
. . .
Design Implications
We synthesized our findings and identified design implication to guide our concept development.
I D E A T I O N & I T E R A T I O N
Speed Dating with Boomers
From our initial brainstorm, we further developed the strongest concepts into a series of storyboards that explored different combinations of our design implications. We speed-dated the storyboards with boomers for feedback.
After speed dating, we prototyped and bodystormed to explore how two of the most prominent concepts might play out. Our parallel concepts were:
Learn + Preserve: how users acquire new skills over time, while maintaining their cognitive ability.
Recall + Relate: how users connect with others over memories and stories about the past.
Evaluating the Concept with Boomers
From our initial bodystorming on Recall+Relate, we decided to refine the bodystorming and test it out to our potential users, baby boomers. We conducted a workshop with Boomers, taking them through activities to talk about their memories surrounding the places they were familiar with. More detail process can be found here.
These activities allowed us to see how the varying levels of immersive media affected our participants’ abilities to remember more details about their chosen places.
T H E P R O T O T Y P E
Low to High Fidelity Prototyping
With the basic framework we tested in the workshop in mind, we started to develop user flow and interaction for MOMENT. We first developed a wireframe that shows how users interact with the screen interface from on-boarding to end of their conversation.
VIDEO PROTOTYPING
Once we had basic user flow for our screen interface, we prototyped the key interactions to see how users would interact with the media on the table, once they pulled out the media from the screen interface. We used Cinema 4D to design interaction of pinching, rotating and scaling media.
NATIVE AR EXPERIENCE
Because we wanted to create a close-to-native experience, we used Unity and HoloToolkit to simulate our key interactions. Using Vuforia image targets, we could test how an object appears and disappears depending on the image target direct our gaze towards. Further, we implemented a voice command to switch modes and scale and rotate an object using just hand movements.
