Personal logo
WORK
ABOUT
RESUME
Personal logo
DEFINE PROBLEM
IDEATION METHOD
DESIGN PROCESS
FINAL PROTOTYPE
If you want to know more about my work, I'm Here!
LinkedIn icon
oshipeng@umich.edu
WORK
ABOUT
RESUME

Tooling Unification

Tooling unification cover image
#website #ux-design #ux-research

Project Overview

Role

Product Design Intern

Team

1 Product Manager
1 Product Designer
1 Product Design Intern
4 Engineers
1 Engineer Intern
2 Data Analysts

Timeline

May - August 2022

Deliverable

User Journey Maps
High-fidelity Prototypes
Our team is an internal tooling team that falls under the larger assessment team who owned all the assessments of the company. There are 2 types of assessments, Assessment Type A and Assessment Type B, that the assessment team currently produces and maintains. Therefore, the responsibility of our team as an internal tooling team is to provide tools to support assessment production and maintenance.

We currently support 2 different tools that were built specifically for the assessment type A and B because they have different functional requirements in tools. However, having multiple tools to produce different assessments is becoming a problem because it limits scalability of future assessment types. Supporting and maintaining 2 different tools is also unproductive in terms of resource allocation and context switching. Therefore, the goal of this tooling unification project is to create a single tool that unifies the production and maintenance of different assessment types to allow scalability of the assessments and increase productivity.

My Impact

Research
Created user journey maps that captured the diverging assessment production processes, user pain points and future touch points to align team's understanding and vision, and informed design decision and future discovery.

Design
The entire scope of the project included redesign for the assessment setup page, question editor page and maintenance page, but I was only responsible for the question editor page redesign, the most important part of the entire project.

Main Collaborators

Although I've been given full autonomy on both research and design, I collaborated extensively with Product Designer and Product Manager to understand the historical context of our tools and for design reviews. I also communicated regularly with the engineers, especially the Tech Lead to understand any technical constraints that I needed to consider to make better design decisions.

Problem

How might we provide a single tool that not only supports production of the 2 existing assessment types but also allows future expansion into providing the 3rd assessment type?
Tool unification diagram

COntext

Assessments are part of the core products the company provides to improve learning experiences and provide feedback to learners, and they are all owned by the larger team called the assessment team. Our team is part of the assessment team who provides assessment production toolings. The assessment team currently produces 2 types of assessments, Assessment Type A and Assessment Type B. Because Assessment Type B came after Assessment Type A that the tool originally built for Assessment Type A was incompatible for Assessment Type B, we had built a second tool specifically for Assessment Type B. As a result, we are currently supporting 2 different assessment tools built specifically for 2 different assessment types - A and B.

Impact of this unification

01.

Allow scalability of the different assessment types. There is an emerging need for the 3rd type of assessment, Assessment Type C, to improve learning experiences. Because the existing 2 tools were built based on needs of the 2 different assessment types, they may not be compatible for this 3rd assessment type. Therefore, we need a new unified tool that removes constraints of the existing tools to allow scalability and compatibility of the future assessment types.

02.

Increase productivity by reducing the number of tools our team needs to support. Supporting and maintaining 2 different tools is becoming a productivity issue for our team because our engineers need to constantly switch context given how different the tools are. This often resulted in neglecting maintenance of 1 tool in favor of the other. Therefore, replacing the existing 2 tools with a unified tool can largely improve the situation our engineers are facing.

research & Synthesis

User Interviews

To design a unified tool for assessment production, we needed to understand how different assessments are produced. We were also interested in understanding the pain points our users have experienced and discovering future opportunities for our tooling. Therefore, user interviews were used as the research method to help us understand the production processes and discover user pain points and potential user touch points with our tool.
User diagram
The users of our tools include assessment content producers (ACPs), authors, and reviewers, but assessment content producers are our primary users and main focus of this research because they are responsible for the entire assessment production cycle from start to finish, including the maintenance after completion.
User domain diagram
‍The assessment content producers recently experienced a reorganization into 4 different curriculum domains that started to influence how they produce assessments in their own domains. Except for domain 2, all the other domains are only responsible for producing Assessment Type A. At the time of the user interviews, there was no users working in domain 4, so I wasn't able to collect information from users regarding this domain. Given the small user pool we had, all of the primary users from domain 1 to 3 were invited in the interviews but they were grouped based on the curriculum domains they were in. As a result, a total of 3 user interviews were conducted with 6 users to understand the assessment production cycle from their domains, their pain points and potential touch points with our tools.

Production Journey Maps

To synthesize the findings from the user interviews, affinity mapping was used. When synthesizing, I found that domain 3 was constituted by 2 sub-domains in which the production processes were different. Based on the assessment types and the domains that fell under, I created a total of 5 journey maps to synthesize the production process, user pain points and future touch points. These journey maps laid a good foundation for the tooling user flow chart built during the brainstorming phase.

There were 4 journey maps for Assessment A corresponding to the 4 curriculum domains including the sub-domains, and 1 journey map for Assessment B. Below is 1 example of the journey map I produced.
Journey map example

ideation

Before I started brainstorming for possible design solutions, I needed to understand the key properties of individual assessment types and how were they different from one another, the information architecture in which how information should be presented and in what hierarchy, and what were some process variations I needed to consider.

Assessment comparison

Assessment comparison chart
To start, it was critical that I understood the key properties of individual assessment types and how were they different from one another because they led to the different functional requirements our tool needed to support and the different production processes our users needed to went through to produce different assessments. Therefore, I created an assessment comparison chart to visualize some of the the key assessment differences.

Information architecture

Information architecture diagram for functional requirements
I received a list of functional requirements for this unified tool from my product manager. Some functionalities provided in the list were either optional or not applicable to all assessment types due to the distinct characteristics some assessment types possessed. To determine when and how should these functionalities be surfaced, I started by grouping the different functional requirements into 6 major categories. This helped me understand the information architecture as to the order of how information should be presented and what functionalities should be surfaced together. Because some of these optional functionalities could be surfaced simply by adding an additional dropdown selection or element, categorization helped me decide where they should appear if certain condition was met.

tooling Journey Map

Toolling journey map
1. Process variations
2. What we support
3. Future opportunities
Based on the production journey maps, I built the tooling journey map to lay out only the steps relevant to the development of this unified tool. The reason why I needed a separate journey map for the tooling instead of using the original production journey maps was because the purpose of the production journey maps was to capture the complete assessment production process, including processes and steps unrelated to the use of our production tools. This tooling journey map also summarized the process variations I needed to consider for this unified tool design, the steps our tools currently support or will support, and future opportunities for our tools to explore.

design

For the MVP of this project, we only focused on providing key functionalities pertaining to creating objectives, writing questions and reviewing questions.

Because different assessment types possessed some distinct characteristics, they required different UI architecture to present information. Therefore, I produced 2 separate designs with slight variations in the information architecture to accommodate the needs of different assessment types, as titled Assessment A and Assessment B & C.

Based on the tooling journey map and my understanding of the different phases in assessment production, I also decided to have separate pages for the objective overview and the question editor so that users will not be overwhelmed by the information and interactions irrelevant to their tasks at hand.

Assessment A

Objective Overview

Assessment A high-fidelity prototypes

Question Editor

Assessment A high-fidelity prototypes

Assessment b & C

Objective View

Objective Overview

Assessment B and C high-fidelity prototypes

Question Editor

Assessment B and C high-fidelity prototypes

Form View

Objective Overview
Assessment B and C high-fidelity prototypes
Question Editor
Assessment B and C high-fidelity prototypes

Usability test

Usability testing results
I conducted a total of 4 usability testings to iterate and refine the design. During the sessions, participants were assigned 7 tasks that reflected their day-to-day responsibilities. Each task was also given a rating on level of impact, which was rated based on how often our users needed to perform the task. This level of impact helped me prioritized the problem to solve if participants had a difficult time completing tasks or did not complete the tasks at all.

Throughout the usability session, all participants followed the think-out-loud protocol to help me understand their behaviors and thoughts. At the end of each session, as a moderator, I asked follow-up questions for participants to share additional thoughts they had.

Design system

All the design elements and components used in the prototypes followed the company's design guidelines and design system.

REflection

i like...

I like how I was given the full autonomy on this project to challenge myself. Instead of following blindly on what's been told to do, autonomy gave me the freedom to make my own decision on how I wanted to approach research and design. One advantage of this was that it forced me to think more deeply about the problem because a good decision could only be made based on a good understanding of the context. More importantly, this autonomy prepared me for dealing with unexpected circumstances so that when the my mentor, the product designer in our team who I often reached out for advice left, I was able to take the initiative to reach out to other resources for help, including Tech Lead, product manager and other product designers.

i wish...

I wish I could have more opportunities to collaborate with other designers during the design process. Because I was the only designer working on this design problem, every time I needed some help or a second eye on my design, I needed to explain the problem space. To a certain extent, it discouraged me from reaching out although I knew I should step out of my comfort zone and take the initiative. More importantly, the reason why I want to collaborate with other designers on a single project is because I also believe in collective wisdom that the best ideas are always the ideas that build upon one another's.
I wish I could have more time to refine my design and explore on the opportunities I discovered along the way. Technically, I was only given 2 months to work on this project in which I spent a majority of the time understanding the products, the problems and brainstorming for solutions, leaving me only two weeks to conduct usability testings, perform analysis and refine my design. I don't regret spending a lot of time on the problems, because I might be solving the wrong problem if I didn't fully understand the problem space. Nonetheless, there were many opportunities I discovered during the process and many interesting findings I got out of the usability testings that I wish I could have more time to explore.

i wonder...

I wonder what if I've also done user interviews and usability testings with other users, not just the primary users. Although at the project level, our primary users are not the authors, authors are the primary users for the question editor page that I was responsible for designing. If I had the opportunities to understand the tools from the authors' perspective, I wouldn't have to design based on the assumptions I had. Understanding authors could also benefit our primary users because some of their pain points were related to their collaboration with the authors. Therefore, improving our authors' experience with the tools could possibly also provide a better user experience for our primary users.
I wonder what if I've pitched my design to the engineers.

i learn...

I've learned to work on the projects with high uncertainty and ambiguity. This project that I worked on was full of uncertainty and ambiguity because we were in a middle of a big change in which we didn't know when and how the change would happen. There were also inconsistencies everywhere such as the use of different language to describe the same thing. Therefore, I've learned to leverage whatever information I had to make decisions appropriate for this given circumstances and prepare for future changes.
© Vicky Ou 2022