A collaborative virtual environment that facilitates communication and understanding in spatial planning
INTERACTION DESIGN, Ideation, PROTOTYPING, User Research
JAMES PAI, Steven dong, Xiao Yan
HTC (Yihsiu Chen, Brian Espinosa)
Unity, htc vive, Premiere PRO, After Effects, photoshop, Illustrator, DSLR
Unison is a collaborative virtual-reality application that facilitates client contribution in spatial planning in order to establish a shared vision among stakeholders. It accomplishes this by providing intuitive visual tools (e.g. multiple perspectives, spatial markup, object manipulation) and a natural understanding of 3D space. Clients are able to experience the real scale of an environment before it is built and make changes in real-time. Integration of Unison into the collaborative workflow allows designers to share and evaluate their concepts with clients to reach a solid agreement.
This project was sponsored by HTC and explores the future of collaboration in spatial design as a capstone in the Master of Human-Computer Interaction + Design program at the University of Washington. Over the course of 6 months, our team conducted research into the problem space, generated numerous design concepts, built and evaluated a working prototype, and delivered a presentation for the final concept. All team members were involved in all stages of the project, from initial research to final refinement.
We started our process by conducting research on the current state and trends of remote collaboration in order to gain insights of its future development. With the background knowledge gained from secondary research, we went out to conduct semi-structured interviews with experts in the industry and academia in order to learn more about the architectural process in the real world. Based on our research findings, we identified the focus of our project as remote collaboration in the early design stages. This helped us design our participatory activities to gain a first-hand understanding of how people collaborate remotely.
After we had a thorough understanding of our problem space, we proceeded to ideate potential solutions and arrived at our final concept through concept evaluation and grouping / combining ideas. We prototyped the concept with Unity to create the virtual environment. To evaluate the usability of our prototype, we conducted user tests which helped us refine the prototype. Finally, we produced a concept video and poster as the final deliverables for the project.
Spatial design is a discipline that encompasses architecture, landscape architecture, interior design, and many other fields of spatial and environmental planning.
The built environments that we experience are all designed and constructed through collaboration between numerous stakeholders from various backgrounds. Due to different levels of expertise between stakeholders, it often takes considerable time and effort to make sure that everyone is on the same page with a design before moving forward with the project.
Traditionally, early design concepts are presented through 2D mediums such as floor plans, images, and sketches. However, these often fail to provide enough visual reference for non-experts (e.g. project owners, investors) to fully understand spatial concepts. On the other hand, 3D mediums such as physical models and high-fidelity CAD models are not commonly used in early planning due to high production costs. Consequently, communication of design intentions can be particularly problematic in the early design stage.
How can we help non-expert collaborators (e.g. project owners) understand spatial aspects of the design while being able to easily express their design intentions?
As we began investigating our problem space, we came up with a list of research questions that would guide our research activities to help us better understand how architects and other designers currently work in the industry. We wanted to learn more about their needs and preferences as well as potential pain points in collaboration. These questions also helped us compare the two groups of interest: designers and their clients.
- Under what scenario or design stage does architects want the client to be involved in the design process?
- What are the current scenarios and use cases where designers have to develop or communicate three-dimensional concepts remotely?
- What tools are currently used and what are their most desired attributes, and what makes a good interaction versus a bad interaction?
- What are the pain points of communication in remote contexts?
- How do professionals and clients behave differently during concept discussion and critique?
- How do professionals and clients differ in terms of their needs and goals?
Sharing common ground (i.e. mutual knowledge, beliefs, and assumptions) is essential for communication between collaborators (Clark, H.H. & Brennan, S. E., 1991). However, architects and project owners working in a remote context face two major challenges.
First, it can be difficult for collaborators without spatial design backgrounds to fully understand design intentions. Differences of expertise in project owners make it easy to misunderstand 2D artifacts such as floor plans and images, or how the final design might look after being built. This kind of miscommunication can be extremely costly in the long run.
The second challenge is ensuring that people are referring to the same objects or locations during discussions. With gesture and gaze, referential communication diminishes the effort needed for mutual understanding during face-to-face communication. However, a remote context, people might not be able to rely on these non-verbal cues to establish common ground. Remote collaborators need to spend more effort to get on the same page, resulting in inefficiency and possible misunderstanding.
SUBJECT-MATTER EXPERT INTERVIEWS
To gain a deeper understanding of the topic, we conducted interviews with 8 industry professionals with backgrounds ranging from architecture, 3D rendering, interior design, urban planning, and project management. Our findings can be categorized into three primary themes:
- Social: Communication with clients is more conversational and emotionally driven
- Clarity: Remotely communicated feedback from clients can be difficult without verbal description and discussion
- Efficiency: Variety of communication and file sharing channels makes it difficult to manage projects effectively
One of our key insights came from Mike Fernandes, a Senior Project Manager at the City of Seattle, who shared that in his experience, "the sense of the scale is what people can't really understand." According to Mike, it is not uncommon for project owners to ask the architect to make changes after construction starts when they can finally experience the full scale of a space. This occurs because traditional mediums only provide indirect information of the design and layout, which is inadequate for the untrained.
We conducted a participatory activity to experience first-hand how people collaborate on spatial design. Each session of the activity involved an architect and a project owner, who were asked to collaborate in order to design a one bedroom apartment as well as a workplace under both remote and in-person contexts.
In total, we conducted 4 sessions of the study with 2 participants each. The conditions (remote vs. in-person collaboration, design a home or work space) were counter-balanced. We decided to use Lego blocks as the medium for creating the spaces since it has a very low learning curve and communicates 3D concepts effectively.
Our activity revealed three major challenges regarding spatial collaboration. First, lack of visual reference prevents collaborators from establishing common ground. Second, limitations in communication mediums make it difficult for project owners to understand design intentions. Third, use of jargon by the architects creates a technical language barrier that project owners are unable to understand. These problems are exacerbated by the remote context that adds a layer of complexity to collaboration and creates a lot of downtime making changes and waiting for review.
Through our research activities, we found that stakeholders face two major challenges during collaboration:
Understanding spatial layout
By only looking at the floor plan project owners might feel that the size of the room differs from what they expected, or they are simply not satisfied with the overall layout. Mike Fernandes, a Senior Project Manager at the City of Seattle, shared that in his experience, "the sense of the scale is what people can't really understand." It is not uncommon for project owners to ask the architect to make changes after construction starts, and this occurs because traditional mediums only provide indirect information of the design and layout, which is inadequate for the untrained.
Communicating design intentions
The problem with communicating design intentions is another critical issue that needs to be addressed. Our participatory activity helped us understand that project owners often struggle with expressing their design intentions in a way that architects can clearly understand. For example, architects might misinterpret the project owner’s design intention due to vagueness or project owners might get confused when architects use jargon. In either situation, this back and forth communication adds layers of complexity to the collaboration and consumes a lot of time making changes and waiting for review.
Understanding these two challenges helped us scope our topic down to a problem statement that would guide our design:
How can we help non-expert collaborators (e.g. project owners) understand spatial aspects of the design while being able to easily express their design intentions?
In order to guide our ideation activities, we created a set of design prompts as a group. Our process was to start with broad individual idea generation before scoping down to a few strong concepts. We used our design prompts to generate 60 rough concepts (20 from each team member) by sketching a thumbnail with a description. Then, we came together as a group to discuss our concepts and group them by common themes to narrow them down to a handful of strong directions.
Select Design Prompts
- How might we empower clients to feel like they are part of the design team?
- How can we make remote interactions feel more dynamic and personal?
- How can we make complicated design tools intuitive enough for non-experts to pick up and use?
- How can we help stakeholders transition between 2D and 3D perspectives for better understanding of spatial flow?
- How can we help clients view the design concept like an interactive story?
Generating ideas individually based on our design prompts was an efficient way to start the process since we might have different interpretations of the problem and expectations of how the problem might be solved. This also allowed us ideate without being influenced by others’ opinions.
In a group ideation session, we used a "wishing" activity to help us think outside the box. We first generated a number of wishes unrestrained by realism, then sorted them under different themes and brainstormed ways to make even our most unrealistic wishes possible in some manner.
Finally, we laid out all of the ideas generated from our activities on a table and identified nine themes among them. Through group discussion to combine and eliminate ideas, we narrowed 60+ concepts down to five design directions that we thought were the most feasible and compelling in order to move forward. We created a sketch and storyboard for each concept to illustrate how it would work and to highlight the ideal interaction.
Based on extensive group discussion and instructor / peer feedback, we decided to pursue a single idea of a shared virtual model-as-interface that takes certain ideas from the previous concepts we ended up with.
FINAL CONCEPT & STORYBOARD
Our initial version of our final concept was a model-based interface shared between remote collaborators that facilitates dialog and spatial understanding through the establishment of common ground. Stakeholders share the same virtual environment and experience asymmetrical viewpoints to gain a full sense of scale while allowing real-time manipulation of the model.
To illustrate how our solution works, we created a storyboard for an ideal interaction. Remote collaborators jump into a virtual space where the client is able to explore a virtual space and communicate with the designer from different viewpoints. The designer is able to observe the client and make real-time changes as requested. This results in immediate feedback and approval so that the project can proceed.
The system diagram maps out the stakeholders involved in the design process and their relationship with other components in the system. Steps 1 to 7 indicate the general design process and show how our solution is integrated into the system.
The model/space itself stands as the common interface for both designers and project owners. Designers manipulate the model in the small scale while project owners experience the updated changes in the full scale. Designers can understand project owners’ attention and needs by sharing their gaze, reading their gesture and listening to their voice. (This would adjusted later as we continued to refine our concept)
As we refined our solution and began planning our prototype, we decided to shift the focus of our solution to emphasize client contribution in both asynchronous and synchronous collaboration. This shift is a result of not only technical constraints but also careful consideration of our solution’s core values and discussion with instructors and sponsors. In the interest of our prototype and our time constraints, we wanted to rapidly create a working VR application for a select use case that can be thoroughly evaluated through our research questions.
We chose to build our prototype in Unity because it provides every function we need and has plenty of community support. It is also a tool that our team already has experience with, so we are able to do develop a rough prototype very quickly.
As for technology, we decided to explore virtual reality to realize our concept using HTC's Vive headset, which also happened to be the only virtual reality solution that supports room-scale locomotion and positionally-tracked controllers - two features that we required to satisfy our design goals. By using Vive, we could put users in the actual space to move around, experience the full-scale environments, and perform tasks the same way they would in real-life. According to our subject-matter experts, this kind of interaction and visualization is vital for non-experts to really understand the size and scale of a space before it is built, which is inadequate using traditional mediums or 2D displays.
We created the interaction user flow of our concept and selected the prototype focus based on a single interaction. Specifically, we decided to focus our solution on asynchronous collaboration context between clients and designers since we believed that solving for asynchronous collaboration would transfer more easily to synchronous. The main functions of our prototype are viewpoint switching, object manipulation, and 3D sketching. We hypothesized that the three functions would help users understand the 3D space and express their ideas.
The initial prototype with the three main functions was built within a week, with our team splitting up work between scripting, setting up 3D models, and hooking up assets. We created a floor plan that we then used to built a virtual environment on. The fidelity was kept low with no use of color in order to emphasize the focus on spatial layout and scale and reflect the same aspects of early planning mediums such as floor plans.
Once we had our initial prototype built, we created an evaluation protocol in order to evaluate the prototype with real users. The evaluation process was conducted by comparing two tasks, one of which was based on traditional floor plans and the other on a virtual reality environment. Each session evaluated a single participant (we recruited 4 participants) and took about one hour. Participants performed the same four tasks with both our solution and pen/paper, with the order of conditions counterbalanced. At the end, participants were interviewed about their experience using both mediums.
Understand how clients will use our solution to convey their ideas
Identify possible challenges and the learning curve of our solution for further refinement
Evaluate the core values that our solution brings to clients as well as its advantages and disadvantages
Participants had more satisfying and reactive experiences with the manipulation function in virtual space because it provides instant feedback of the spatial change.
Participants were more willing to experiment with different changes and specify the design when they are in VR compared to using pen and paper. Participants found that they easily got lost when sketching on paper due to the permanent nature of the medium.
- The flexibility of jumping between small-scale full-scale views increased participants’ spatial awareness. In particular, participants used the small-scale view for an overview of the spatial layout much like a traditional floor plan
- Sense of measurement was more subjective to our participant group. In our prototype, they measured size and distance (when manipulating objects) relative to other elements in the space instead of referring to the measurement grid. However, most participants expressed a desire for precise positioning and explicit measurement visualizations.
The core value of our final design is enabling real change in real time through a virtual environment. This helps users receive visual feedback immediately after they make any changes to the design. With clear visual feedback, users are better informed and able to reach a more confident design solution and thusly a more solid agreement between collaborators. With Unison, architects can invite clients to collaborate on early spatial plans and have them leave real-time markup in the virtual space while discussing the design. The virtual space as the common interface between collaborators eliminates barriers posed by traditional mediums and provides a clearer sense of what the future outcome of a design might feel like.
Our three design principles for Unison's key features focus on physical scale, spatial feedback, and user experimentation.
Provide a clear sense of physical scale
Users should be immersed in the space, but at the same time be able to see the bigger picture that a floor plan provides. In our prototype, users are able to shrink the space down to a small-scale or blow it up to full scale. Our evaluation revealed that users tend to use the small-scale view to manipulate the design and the full-scale view to experience their changes.
Enable collaborators to communicate feedback in spatial context
Communication should not be limited to verbal or written comments. Users are able to express their ideas through object manipulation and 3D sketching. This allows them to make direct changes on the spatial models.
Encourage users to experiment with different designs
The solution is aimed to provide a satisfying experience by enabling users to make changes and see the outcomes in real-time. Clear visual feedback encourages users to experiment with and evaluate different design concepts, in order to reach a more confident decision.
The three key features from our prototype are motivated by our design principles and are validated through user testing to support our design goal of facilitating communication in spatial planning.
In order to show how our product works, we produced a video using real footage as well as mixed reality. We chose to produce mixed-reality footage since it is the most effective way of showing exactly how someone is interacting within a virtual space, although it posed its own plethora of challenges. Our team rented an empty studio space and built a green screen for our mixed-reality set up. I shot all footage using a DSLR in one week edited the video (including compositing and audio editing) over a weekend.
By far the most challenging part of this process was calibrating the camera to match up with the virtual camera with Unity. It took us several tries and different approaches before we had an adequate result, though there is still much room for improvement. Thankfully, compositing the footage with the Unity screen recording in After Effects was relatively straightforward.
Our project culminated in a final presentation and poster session at the University of Washington. We wrote a script, compiled a slide deck to accompany our video, and presented our final concept to the UW community as well as industry professionals and peers. We later gave the same presentation to our sponsor at HTC Creative Labs. Overall, I thought our presentations were quite successful after having plenty of practice, and our video was able to demonstrate the value of Unison.
By focusing on the early design stage, our concept stands as a compelling virtual reality solution for remote communication and collaboration issues in spatial design. The prototype evaluation has suggested that it successfully facilitates spatial understanding, feedback, and experimentation.
We believe the core value of our solution will not only apply to the early design stage for future development - it can also be a potential solution for clients to understand construction specifications. For example, any design request from the client side can be simulated in order to inform clients about the final outcomes during the construction stage.
Built in virtual reality, our product provides a more realistic and intriguing user experience of spatial design activities. It is an example how new technologies such as virtual reality can change the way people collaborate, and the possibilities are limitless.
This capstone project was a significant undertaking that required our team to put in our very best efforts and the result is something we are extremely proud of. We were also lucky to have a great industry sponsor that could guide our project and provide valuable feedback from an industry perspective.