4 Designers (Greg Chen, Wei-Chieh Wang, Youngryun Cho,
Zhuoneng Wang)
The prompt is to design a museum experience for Carnegie Museum of Natural History (CMNH) using emerging technology. We thought it'd be more meaningful to consider their vision while designing for their future.
The vision of CMNH is to become the world’s most relevant history museum by enabling visitors to discover greater stories, including human being a caretaker of the Earth and interconnected relationships between nature and people.
According to the secondary research, to stay relevant, it’s important to engage GenZ (the next generation) in a deeper and creative way and to transform the museum to be more than an information provider, but an experiential and flexible space. Therefore, the problem space we define:
Beyond consists of two parts, “gamified museum experience” to actively engage with Gen Z and “participatory social interactions” to increase CMNH social influence on digital community. We build the concept on mixed reality goggles to ensure the most engaging and immersive experience. First, we introduce a scavenger hunt to create a game-like learning environment, inviting visitors to collect a digital copy of animals or plants they like in the museum. During the process, visitors are prompted to complete tasks with greater stories embedded. The goal is for them to learn while having fun. Furthermore, visitors are able to wear their collections and showcase them as a part of personal style in virtual setting. It also serves as an incentive for museum-goers to visit and revisit the museum. Lastly, visitors are encouraged to share what they learn and their collections as AR filters on social media. To make the sharing interactive and engaging, we design a mobile-based AR filter creation tools to let visitors be creative about their collections. The idea is tailored for the lifestyle of Gen Z. Such social interaction also increases CMHN’s social influence in the digital community.
Visitors are greeted by a sharing wall where past visitors can leave their thoughts with AR filters created with plants or animals collected during their trip. Next to it, it’s a sign directing visitors to start the experience. The wall serves as a space where visitors exchange thoughts, in the meantime, as a prompt for first-time goers to try out Beyond
Visitors get admission wristbands and Hololens2 at the ticketing counter. The admission tag is redesigned as a wristband as it is not only a proof of admission but an anchor point for the glass to display a control panel, leveraging the affordance of a watch
The onboarding gestural tutorial is prompted to users. It also allows users to have a voice command in hands-free scenarios
After wearing the glass, visitors can see other visitors’ collections, encouraging visitors to get their own collection and learn more about these plants and animals
With voice command, the glass can show visitors where to collect desired plants/ animals or any exhibit visitors are interested in
Once a visitor steps inside an activation zone of a diorama, ambiance of habitat in a given diorama would immediately play, immersing visitors in the environment. Visitors then can interact with a diorama with simple gestures, moving a palm as a pointer for hovering, pinching and dragging to grab a plant or an animal from a diorama
By grabbing a plant forward, the visitor could take a closer look at it in 360 degrees, re-vitalizing the dead-dead specimen in an out-of-fashion diorama. They could also learn more facts about the given plant
To collect the plant, visitors would be prompted to a mini-game in the context of the goal of the museum, environmental changes, and connections with humans. After learning through the game, users can collect it to the wristband
Transfer the collection of the day to APP before returning the glass, allowing users to review their trip history and learnings
Visitors can review the latest posts on the sharing wall, their past visits, and learning from the museum, filters, and collection in the APP
Create AR filters using the plants and animals collected in the museum and share their thoughts and learnings on social media. The posts would be synced to sharing wall with the designated hashtag, #CMNH. By doing so, more and more people are engaged, and greater stories are heard.
To help the museum stay relevant, we believe it’s important to understand the audiences they attract and the ones they do not because the future of the museum depends on securing the current visitors and attracting potential visitors. Due to confidentiality, we are unable to have visitor demographics of CMNH, we then rely on secondary research to understand which age group visit natural and history museum the most and the least. The data shows:
We decide to set Gen Z (born in 1996 - 2010) as our target audience as they are the next generation of visitors and donors, however, are least connected with museums. From the research, Gen Z expects museums to be:
In our first visit to CMNH, we noticed a few findings:
Children’s Museum of Pittsburgh
It offers interactive and participatory exhibits, engaging visitors for a longer time. It also uses technology, such as projection and motion detection, to create an immersive environment, allowing visitors to learn while having fun
The Cleveland Museum of Art
Similar to Children’s Museum of Pittsburgh, the museum applies technology to craft a multi-sensory and participatory learning experience. What inspires us the most is the use of Hololens2, it guides users to walk on a curated path to learn and interact with virtual objects from old times
We speak with Nicole, the curator of CMNH. She shares valuable insights about the role of museums, preferences of visitors, and the vision of the Carnegie Museum of Natural History. It includes:
Also after the interview with the curator, we spend time studying the museum’s current approach to increase public awareness. Some articles on the website mention several recent renovations are aimed to fulfill the task. Among them, Botany Hall caught our attention, as quoted from the piece,
We then decide to focus our research on Botany Hall as it aligns with the latest vision of the museum, giving us a window to see from their perspectives and find design opportunities.
The Botany Hall is known for its great collection of dioramas, and this renovation not only renews dioramas but connects them to the themes. For example, pointing out plants are a source of fiber in human’s daily life helps people realize the interconnectedness of humans and plants, or, by indicating the change of flowering and leading times are in fact affected by human-caused environmental change.
That said, we also find some opportunities for design intervention:
We summarize our findings from Secondary Research, Target Audience, Field Observations, Other Museum Observations, Curator Interview, and Focus Research:
Based on the insights, we define design principles:
We explore and define touchpoints in each phase of user journey. At post museum, we incorporate Social Media as one of the touch points since our target audience is Gen Z, a group that is highly attached to SNS
These are the keywords that guide us during storyboarding,
[Perceptive]
Provide wayfinding when necessary and remind users of essential steps
[Immersive][Mutli-sensory]
With MR glass, users could fully immerse in the habitats of each diorama. For example, users could hear birds chirping in Mount Rainier and wind blowing in Presque Isle
[Contextualized][Mutli-sensory]
Gamify the bigger stories the museum aims to tell by inviting users to play a mini-game before collecting plants/ animals
[Community-based][Beyond]
Extend the museum experience by encouraging users to review past visits and share thoughts with AR filters of their collection. By sharing, the impact on the digital community would grow and thereby raise public awareness about bigger stories
Hands on AR/ MR Devices
Since our team is passionate in MR/ VR technology, we decide to design the MR expeirence. We had a chance trying many devices on the market, including Oculus Quest, Hololens 1, Hololens 2, HP Reverb, and Varjo
Admission Ticket Redesign for Accurate Tracking
Although our design is build on Hololens 2, we did not follow the exact paradigm. For example, per MRTK suggestion, palm-up is to trigger menu. We decide to use “raise the wrist with palm-down” for menu. The reason is that raising the wrist with palm-down borrows the affordance of a watch, so it creates less friction for first-time users. It’s especially helpful in a use case like museum where people don’t go very often and need to get familiar with commands right away.
To address the possible increase of false activation caused by palm-down, we thereby add an anchor point on a admission tag for more accurate detection and redesign it as a wristband to reinforce the watch affordance.
We spend much effort simplifying gestural commands because the more gestures we define, the steeper the learning curve is going to be. Considering museum visit is usually short and occasional, visitors should be able to pick up all commands in a minute. We borrow the affordance of a watch for “Raise the wrist” and “Pinch and collect”.
For hovering, we decide to use palm pointing instead of finger pointing. Based on our secondary research, a finger cannot be tracked by camera very well due to the possibility of being blocked by a hand.
Although some studies suggest the “come on” gesture (left) is the most easily tracked and the most natural hand gesture to summon an object, we think this gesture is not the best choice as it has a negative connotation. We thereby conduct a small scale primary research, “Pinch and drag” is the most preferred and intuitive in the test.
We prototyped a “pinch and drag” scenario in Unity to demo grabbing a plant from a diorama. We used MRTK Kit as a base and customized it into our design. We then tested in Varjo, a see-through MR headset. It gave us a chance to test the gestural commands and font size. They all worked out well in the prototype.
In the final presentation, we also demo “pinch and drag” use case on a laptop with the help of Leap Motion to track hand gestures.
None of us has experience creating AR filters, so we explore all major AR filter-making software, including Adobe Dimension, Spark AR, and Lens Studio. We go with Spark AR studio in our final production. The tracking systems in the software are inspiring when defining gestures for the project. It also helps us when designing any interactions that involved motion tracking in the future.
We use ProtoPie for high-fidelity mobile APP prototyping.
We take this chance to explore how to prototype with 3D models in the APP, especially for the model to have a 3D rotate.
We have green as primary color and orange as the accent color. For 3D, we go with a low-play, faceted, and fun style, tailored for the taste of GenZ
In terms of font size, we follow the document by Microsoft to ensure the legibility in MR setting
One question was brought up during the critique of the final presentation - do people really need a blurred background in MR world just for better contrast between objects and real-world background? Would it create more problems? Such as safety in the public space, or a detached visual experience. The balance of it would be an interesting topic to explore.
Current BEYOND focuses on individual experience, and we believe extending it for a group of visitors can be meaningful as many people start their museum experience on a field trip at school or on a family trip.
We have secondary research on the current technology of tracking gestures in order to define gestural commands for BEYOND. We realize it still has a lot of limitations, for example, if a gesture is performed outside of camera’s view, it will not be tracked correctly. Another example is that because people move their hands a lot both intentionally (for communication and object manipulation) and unintentionally, adding another step to avoid false activation is necessary. Lastly, people’s hands are too useful to be free all the time, so it’s always good to consider hands-free scenarios. I think that’s why META now has a wrist-based neural interface for hands-free and accurate inputs.
I also learn many interactions that make sense for screen-based are not for MR world. For instance, MR devices are not meant for heavy reading for now. Therefore, avoiding text-based interactions is always preferred.
One of the comments from the guest judges is to be careful of the word, “climate change”. Some people with a specific political stand may not be interested in our design at all if we associate it with climate change. Environmental change might be a better term in this case. As an international student, it’s a culture shock and mind-blowing as I was not aware a political preference could be a factor if people would connect to a design or not. It makes me be more aware of the importance and the power of text.
