Future Hybrid MR Design Studio Environments

Gathering Research

2020 Oct. 5~6

Brief: The School of Design at Carnegie Mellon is hiring you to develop a mixed reality ​hybrid (people bothin person and remote)​ design studio environment to enhance the studio learning experience. Focus on ​one aspect ​of the studio experience. You will be designing and developing a low-fi prototype of one interaction in a hybrid studio experience for the client to review.

Aspects:

• Visible work: ​It is easy to see what other people are working on by what is on their desk, sketched on the whiteboard, or pinned up on the wall.
• Inspiration and exchange of ideas:​ One could see what others were working on and discussideas in person.
• Distraction:​ Unfortunately, working in the same environment where multiple conversations areongoing can be quite distracting.
• Impromptu chats: ​Professors or TAs might stop by between classes to check in with students.
• Office hours​: Sometimes students might stop by for office hours with a TA or professor.
• Social Cues:​ One can easily see when someone seems busy or available for a conversation.
• Cross-studio pollination:​ One might be inspired by conversations with friends that stopped byfor a visit from a different studio.
• Social Bonding:​ People can easily connect, inspire, and encourage each other while working in the same space. Students might take breaks together to Entropy and discuss their work overfood and drink.
• Social Support​: A word of encouragement might help someone else to have a breakthrough.

My Experience

Cons- I’ve been working remotely for this current semester and most of the difficulties that I’ve been facing usually have to do with social aspects.

• Although we use mediums to post our process, sometimes it’s hard to see if I’m heading in the right direction or not if people don’t update it regularly. Sometimes I feel like I’m falling behind whenever I see people in the studio.
• There is a limit to doing everything digitally. Getting feedback and critique are still helpful but it would be even more useful if I could see the professors and peers in real life and discuss about the projects.
• It’s hard to communicate with people especially with those I’m not really close with. Although we do have multiple group chats, for some reason, I feel intimidated to use them. Gather town isn’t used that much either; it’s mostly just Grace and me who are using them.
• Although I do like being alone, there are times when I miss bonding with people outside of studio and zoom calls.
• Being surrounded by hard working people encouraged me to work efficiently. However, working remotely, naturally took some of those aspects away from me.

Pros

• It’s a lot more relaxing being at home and I get to spend more time with my family. It’s just more comfortable compared to the studio and I’m feeling less stressed.
• I can use my space more freely and play music out-loud.
• I can wake up a minute before the class and still make it on time.
• I don’t have to spend time traveling or cooking ( my mom cooks for me).
• I don’t have to worry about getting distracted by other people, unless my neighbor is being extremely loud.

Interviewing Peers

Link: Interview

Before I began interviewing, I faced a problem that related to being remote. Since I’m not a talkative person, I haven’t really talked to many people since last semester. I felt bad for trying to interview people because it felt they would think that I would only approach them if I had a specific purpose. I ended up interviewing people that I usually talked, who are Daniel, Catherine, and Grace.

A lot of the problems that they discussed were either were related to learning and focusing aspects. Catherine and Daniel, who are both on campus, discussed the frustrations of limitations of the hybrid course over zoom. Catherine brought up the point of lack of individual critique that they got due to the structure of zoom. Often times, she didn’t get much helpful feedback despite being physically there.

In addition, Daniel talked about how it was hard for him to focus if he was alone so working in the studio worked better for him, unless it was noisy in the studio. Although Catherine and Grace said that focusing depended on their mindset, they mentioned the importance of seeing other people’s works and talking with peers. They talked about how socializing motivated them and ensured them that they are going in the right direction, which is something that they aren’t experiencing as much these days.

The lack of social actives was brought up multiple times. Catherine brought up the point that there are efficient ways for us to communicate but there isn’t an incentive. Grace also talked about how it was natural for people on campus to talk to people near them rather than reaching out to us when they don’t feel the need to.

However, Grace mentioned some of the positive aspects of having hybrid courses and zoom, such as being able to write a comment without disrupting anyone. Although, I don’t write that many comments in class, this was something I highly agreed with since I have a hard time speaking up in class. We also got to learn new alternative methods of communication, such as figma and notion.

Research

Google glasses

The google glasses first came into my mind when I first heard about this project. When I initially heard about this AR device back in 2015, I was really excited about the concept and how amazing it would be to mix the digital and physical world. However, unfortunately google glasses got a lot of backlashes and eventually was forced to stop prototyping. Google did launch upgraded versions later on but they still weren’t successful. One of the reasons that cuased it to be a failure was because of privacy issues. Since the cameras would be on every second its being worn, people were worried that they can be filmed without any consensus. Another reason was because it was simply not good enough to compete with smartphones. It does whatever smartphones do, however, the batteries drain faster and there was a lack of affordance and clarity of how to use them. In addition, it was ugly and no one wanted to wear it in the public. Overall, there was no clear explanation of why this technology existed as it didn’t serve a specific purpose. This project reminded me that whatever designers design needs to have a useful purpose and should be used easily without much frustration.

Personas

After learning and reading about personas, I realized that they should not be based on stereotypes but rather from realistic and diverse personalities. We also have to consider all types of people, such as those who are temporaliy blind due to an eye surgery. Therefore when creating personas, we should think about social, economical, permanent, temporal aspects of these people. After interviewing peers, I gathered specific frustrations and motivations that they carried and tried to incorporate the realistic factors into these personas.

However, although, I tried to stay away from stereotypical characteristics, I found that challenging. Since I was building these personas from only the people I talked to and from my own experience as well, I felt the limit of creating diverse backgrounds and personalities.

Storyboarding

Since I am still new to this project and I have no clue how I’m going to move forward, I began developing simple storyboards to get my initial thoughts going. At this stage, I didn’t really consider rather these were realistic or not.

For this one, I tried to tackle the problems that I faced learning through zoom. When talking about physical model, there is obviously a major difference between looking at the model physically and looking at pictures of it. In addition, when I was getting feedback through zoom, it was both difficult for me and the professors to talk about the issues because sometimes we couldn’t tell which part of the picture they were talking about. Although there is a drawing function on zoom, there are limits to how much that can do.

I drew this storyboard trying to tackle the issues mentioned, however after making it, I realized that people are only used to writing on flat surface. So having a 3D pen would bring up other confusing issues as well.

For this storyboard, I wanted to bring up the social aspect. Here you will be able to swipe through your peers who are online and see whether they are available or not. If they have the GPS track mode on, then it will show where they are as well. If they want to be alone, they can turn on “Do not disturb” mode on to make sure that no one will contact you. If the person is available, you can click on the profile and make a video call with that person. But I’m not completely sure if this will be available in 10 years or not.

I wanted to create something that is related to distraction in studio. During my first year, I always had my headphone with me because it was hard for me to concentrate when the studio was really loud. However, sometimes it was still challenging to focus since I was able to see what was happening around me. (People did weird stuff like yoga and skateboarding). With this device you can isolate yourself from you surrounding but still be able to see the desk and tools needed for working on the project.

Iteration 1

Oct. 6~7

Workspace + Habit

For next class, we were asked to document our desk space and what types of tools we use to work, communicate, and socialize.

For working and private space, I am isolated from my family as I have my desk in my room. I realized that in order for me to focus on my work, it’s better for me to be alone. I also just enjoy having time own time as well. Something I do is that I work right next to my bed, but not on my bed. I end up spending most of my time alone in my room, even though I’m with my family.

Also since I’m remote, the desk in my room is not fit for studio work as it’s smaller than the desks in studio. The desk gets pretty crowed and in order to do crafting work, I have to move downstairs. There’s barely any space for me to use my sketchbook.

For most part, having multiple monitors is fine but there are some moments when it can get annoying. The mircosoft laptop that I use for zoom is really slow and it crashes often since it’s old. So I do all my school work using Mac mini, which works fine but it doesn’t have an access to the monitor’s camera or mic. Therefore whenever I have to screen share, I have to open the same tabs in the laptop or transfer and download things using Google drive. This isn’t a big problem but sometimes it can be time consuming.

However, having a different screens for zoom is helpful when the professors are sharing their screen to demonstrate something. Then it’s easy for me to follow what they’re doing since I can follow and look at the same time.

When working, I have a lot of tabs and programs open because I don’t know when I’m going to go back to them so it’d be time consuming to open and close them repeatedly. When working like this, it does get frustrated at times, because sometimes I don’t know which tab to open or I exit out on a different tab by accident. Luckily, I haven’t had any incident where I couldn’t save my work because the program or computer crashed for design work yet. However, it might happen soon if I don’t take care of my computer.

The desktop gets messier as we get more in depth with the project

Most of my projects require Adobe Photoshop, Illustrator, and other Adobe software. I got an iPad not that long ago, so now I’ve been using procreate to draw my sketches as well.

For my desktop, I also keep folders that has all the websites that I use frequently, which include canvas and box.

For communicating and socializing, I mostly use my phone since that is the most convenient way for me, except when we’re using zoom. I use messenger and snapchat often and I’ve been using discord a lot more recently. However, I don’t directly message people a lot and most of the texts I get are from the group chat. I’ve been somewhat isolated from other design peers due to working remotely.

I also always use my phone to listen to music. It’s hard for me to concentrate or get motivated without listening to music.

Screenshots of my phone

Overall, I prefer working alone but I need some sort of social interaction in order to see where I am in the process or whether I’m going the right direction or not. I also need multiple tabs, screens, and programs open at the same time so that I can easily access them, which can be confusing and messy. In addition, ideally bigger desk is required if I want to use my space comfortably.

Brainstorming

For this iteration, I decided to combine some of the problems mentioned in my previous storyboards but mainly focus on the social aspect. I wanted this iteration to include information on who’s available, how we can share our physical model or work process, and how remote students can feel included. After getting my general thoughts down, I began creating visuals in Illustrator. Although I’m aware that as of now, it’s too early to make anything, but I just wanted to create these just to get myself motivated to work.

As shown above, contact list will be divided into peers and professors and the user will be able to divide into further categories by their location. The search bar can be used to see people who are in a certain location or to directly someone’s name. The user can also see who’s online or not. If the person is online, the user can message, call, video call, or view their works. If that person isn’t online, then the user won’t be able to make a call. There is a “Do Not Disturb” feature which disables the user from messaging or calling that person. But the user will still be able to view their work.

While calling, the user can minimize the screen so that they can see without getting the screen in their way but can still hear the voice. Also the size of each screen is adjustable so the user can either make them bigger or smaller according to their preference.

View Work

Storyboard for “View Work”

Viewing work will be similar to medium. When the user clicks that button, he/she will able to scroll through that person’s projects and view the updated work process. One thing different about this is that if the person uploaded a 3D scanned file of their physical model, the user can open that file and view the model in real life.

Share Work + Share View

Storyboard for “Share Work” and “Share View”

While calling, the user can use “Share Work” feature which allows the user to share their work with the person he or she is calling. One thing different with this feature and “View Work” is that, with a given permission, the person being called can edit or write on their work. Another thing is that the user can share whatever that they’re working on, meaning that the work may not have been uploaded for the “View Work” section. The user can also scan their physical model in that spot, if there is a 3D scanner nearby, and share it immediately.

The “Share View” allows the users to share whatever he or she is viewing through the MR glasses. The person being called can see the first person point of view of the user through a smaller screen. This can be useful when the user or the other person is getting feedback on their physical model since they can directly which part of the model he or she is referring to.

Prototyping

I layered the visuals on top of photo that I took so that people can have a better understanding of how this could work.

For the bottom ones, I wanted to see if multiple tabs can be opened at once, since that’s something that I do. Although this could get confusing, the users can view multiple people’s works at the same time.

Using Wiarframe, I created a simple video to test out how this could look in real life. I couldn’t figure out how to make the second picture appear after staring at the first picture. So I just made it really small and scale up whenever the interaction takes place. Also the first picture is supposed to get smaller and move to the side when the second picture comes out, but for some reason, it wasn’t doing that.

Self-reflection

As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?

I believe that one of the designer’s biggest responsibility is to enhance the user’s experience. There is so much possibilities that come with technology is endless, meaning that we have an overwhelmingly many ways to design our environments. If used appropriately, the digital interactions should feel natural using affordances. For example, zooming in or saving certain objects can’t happen in real life but can be achieved through technology. Technically, this should feel weird but it enhances our experience and comes naturally to us. However, whatever we design should not distract or annoy the user. Therefore we must carefully decide on what is necessary and what is not so that confusion is reduced and efficiency is maximized. Everything that we design should have a purpose.

Another thing that designers should consider is the diversity of audience. If we ignore certain types of people, then it contributes to widening the gap. For example, as of now, MR itself it expensive and only the wealthy can afford to buy it. Therefore designing and working to create better MR experiences could unintentionally lead to widening the gap between the poor and the rich. Therefore we should acknowledge all types of people and try our best to incorporate all perspectives into our works.

Takeaways from Class

• Work beyond the flat screens; consider new ways to explore MR
• Think about the gestures that can be used
• Private vs Public
• Consider all the details for an ideal outcome
• Explore outside of comfort zone since this is for the future

More Brainstorming

Oct. 10~12

I decided to develop my ideas further by making more storyboards.

While thinking about this project, I realized that unlike phones, whatever appears on the screen will block the users’ views. So I decided to come up with ways to open up the menu so that the resting screen won’t bother the users.

Here, the user can double tap on the side of the glasses to open up the menu. But after creating this, I wanted to see if there could be a way to break the boundary.

After looking at Jenny’s sketch video, I thought it would be a good idea to use other parts of the bodies as well. I thought that it could be convenient if users could look at the apps and time as if they could view it like a watch.

Next, I began thinking about how users could take pictures.

Camera

They will be able to take pictures and 3D scans with certain hand gestures. Later on, they can see their saved own photos and scans, as well as other downloaded ones, by going to the gallery.

Would only show those who have their GPS on

Here, I began exploring ways to break out of the frame with a map. Users can see the locations of their peers with this device. They can rotate and zoom into the globe to see more specific locations. However, I realized that this process could take too many steps for users to get to a certain location. In addition, it might feel creepy since people can track where they are every second by zooming into their location. It might be fine if they are in studio, but if they are working remotely in their personal homes, then that would be a different story.

View Work

For this version, I decided to separate the view work feature from the contact so that it would be easier for the users to see their peers’ works. For viewing their works, as of now, I am thinking that it would work like a medium except you can view 3D scans as well. One thing uncertain about this is that this is still stuck in screen and I don’t think I’m taking advantage of having no boundary enough.

After reviewing the storyboards, I decided to narrow down the “apps” into these five.

Prototyping

Users can view the apps and time like a watch

Unlike the storyboard shown above, I decided to show all of them at once instead of using hand gestures to either scroll through or expand them since I narrowed down the number of apps.

Too big to be uploaded as a gif

I tried making a video of how taking a picture with MR glasses might look like. I didn’t make a video on taking a 3D scan yet but hopefully, I’ll get to learn more about working with 3D programs.

Overall, I’m concerned that there is too much going on and I might not have enough time to fully develop on each aspect. For further development, I am considering cutting down the ideas and focusing on one aspect.

Feedback

Oct. 14

Jason

• Think more about the benefits of using MR; give reason for people on campus to use it as well
• The “watch” interaction can feel natural
• Leverage the environments

Resource from Jason: Video

Daphne

• Try not to tackle all the issues; do one thing well
• Don’t limit to buttons or screens, include voice recognition
• For the watch interaction, the apps can be directly on the arm instead of floating around for clarity
• Think about my own previous collaborative projects experiences and how it would’ve helped if MR was available

The feedback I got emphasize the reason for having MR instead of just regular phones, encouraging the students on campus to use it as well. They both seemed to like the idea of “share view” so I decided to focus more on that direction.

Further Developing

Oct. 14~15

Deciding on Topic

It was challenging for me to focus on one topic since a lot of them were related. After considering, I decided to remove “view work” and focus more on social interaction, such as getting feedback or inspirations from their peers. Although being able to see everyone’s work would be helpful, at this point I needed to cut something down. Also having social interaction was something that I wanted to explore further as a remote student this semester. So instead of displaying work publicly, users can save their work in their own gallery and share them with peers when they are digitally meeting them for feedback.

Updated “apps”

Giving Reason

2D to 3D

After talking to Jason, I realized that my project so far did not give many benefits for people on campus. For example, they technically don’t need to take pictures or scans of their 3D models if they are getting feedback from other people on campus. So I took inspiration from the video that Jason sent, and thought about the possibility of turning a 2D drawing or print into a digital 3D model, which could help students to visualize before making a physical model.

Social Iteraction

Since I wanted to focus more on exchanging feedback with peers, I decided to develop further into that idea. To be honest, I felt really stuck not knowing what to do next. I was struggling how I can make the students benefit from MR. Although the “share view” option did something that current phone or laptops had difficulty of doing, it still relied heavily on screen. But then I saw this video that David’s sent, which helped me a lot. In spatial, people can virtually meet as avatars, use the white board to discuss ideas, and can pull up images and 3D models.

For my project, I thought I could include these as well. I decided to get rid of phone calling and add “meet” option instead. Here students can meet as avatars, as long as they have empty wall space available, ideally from two to four, and exchange ideas or give feedback. They can also pull up 3D scans or images from their gallery and share with their peers by putting them up on the board. The users can also write on the board or the images. Lastly, they can also save the board so that they can either pull it up during the next meeting or go through them individually in their own gallery.

Peers can suggest to meet each other through text messages. One can invite others by creating a meeting and sending invites to people.

They can either manually click peers or use voice recognition feature to send invitations. Invited peers can enter the meeting by clicking on the invites text. Also, instead of displaying their locations, I decided to display what time zone they’re in instead.

Meet quick prototype

Sketch Video Plan

Starts with a person wearing the MR glasses.

Part 1: Explaining the camera (all three parts) and saving in gallery.

Part 2: Receives text message from peer asking to meet. The person enters the virtual meeting room and invites the peer, showing both buttons and voice recognition features)

Part 3: User pulls up images and models from their gallery, writes on the board, and discusses their work process with their peer.

Feedback

Oct. 16

• Voice input ideally shouldn’t require much moving, instead of double tapping certain keyword should activate it
• Apps appearing on arm might be unnatural

Sketch Video Iteration 1

Oct. 16 ~19

Initial Stage

Before I started filming, I wrote down what I wanted to include in the video step by step and drew a very quick storyboard. I also made some minor changes in the graphics that will be presented in the video.

I thought if I didn’t fill them completely, they might seem less awkward on arms. Also for the 3D model, I decided to make it and animate in blender instead of using wiarframe. But since it was my first time using this software, I couldn’t make anything complex due to the limited time.

I also changed the voice input activation from double tapping the glasses to saying “Hey, Carnegie” to minimize the effort for users.

Overall, one thing I learned from this process is that using green screen is extremely helpful.

Video

I didn’t get to finish editing the video or put any music on it.

Feedback

Oct. 20

Daphne and Davis

• Make all button interactions happen on the arm
• Specific apps should appear if the user enters the main apps

Final Sketch Video

Oct. 20 ~ 22

Listening to the feedback, I decided to make most of the interactions happen on the arm.

Meet and Camera

The specific meet apps have < sign because I thought it’d useful if users can use camera or text peers without having to exit the meeting room. But I didn’t want to stuff everything on the arm so now users can scroll through the apps.

Final Video

Self Reflection

How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?

Both projects heavily focused on combining physical and digital worlds and how digital interactions can improve people’s experience. But the first project was based more around the physical environment while the second one was more focused on digital environment.

In my experience, the first project was helpful in terms of designing space and studying how to effectively deliver the exhibit’s, or any place’s, specific purpose to the audience. For this, we had to consider whether or not the digital interactions would distract the audience, instead of enhancing their experience, and if they would understand the exhibit as we intended for them to. Meanwhile, the setting of the second project was supposed to be in ten years from now so we had much more freedom. For this it was easier to ask and answer the question “would people actually use this?” since I was also part of the audience. However, it was harder to settle down on a concept because of the freedom and futuristic concept. At the end, I think I learned to break out of the comfort zone and just push forward with what I have.

Something similar about both projects was that I can’t make what I designed in real life. I can’t build or code, so for prototyping, I needed to learn new software skills such as learning how to learn use SketchUp or After Effects in order to properly present my ideas.

After taking Environments mini, with my current knowledge, I now believe that the job of an Environment designer is to effectively and intentionally improve the users’ experiences. Designers should consider all aspects and deeply explore before presenting the ideas. Our designs are the guidelines for programmers and engineers. Therefore, whatever we produce must have a purpose that improve the quality of digital and physical interactions, which hopefully can lead to an ideal future.