Marketplace Configuration and Customisation Journey
High-level UX research in with Mixed-methods
Task analysis, User interview, Descriptive statistics, Survey, Customer Journey Map
Timeline
Nov. 2021 - Feb. 2022 (4 months)
My Role
Leading the research activities, formulating the research plan, and executing it.
Closely collaborating with the Product Designers, Managers and the owner.
Purposefully selecting stakeholders and users through sampling to create stakeholder and user map.
Tools
Google Form, Miro, Google Meet, Dovetail, PowerBI, Jira, Confluence
Summary of the Project
CloudBlue, a division of Ingrammicro and a large distributor of Microsoft provides a marketplace platform tailored for Cloud Service Providers and Resellers. The platform empowers them to sell and procure software and solutions efficiently.
Medium and small-sized reseller companies predominantly depend on creating various sales channels and tailoring their marketplaces to attract unidentified end customers. However, customer feedback in 2021 unveiled both low product satisfaction and a limited number of SMB Reseller companies successfully setting up their first marketplace during or after onboarding. As a result, CloudBlue product team decided to improve the current user experience of the marketplace configuration and customisation tool.
During the initial discovery phase, the main objective of this project revolved around pinpointing user journey, their key task scenarios, and the areas of concern. The results supported the designers and product team in formulating a new product roadmap and determining project priorities.
I spearheaded the discovery research efforts, employed a mix of methodologies to uncover user task scenarios, requirements, and challenges. I crafted a Customer Journey Map, outlining essential Usability Problems along the journey that require resolution. The results supported the designers and product team in formulating a new product roadmap and determining project priorities.
Afterward, the complete user journey was partitioned into segments, each managed by various product designers, POs and PMs, and researchers, to progress in parallel. This collaborative effort involved dig into user needs and frustration scenarios, generate new ideas, crafting concepts, conducting tests, and refining designs. I personally undertook certain components of this effort, including the enhancement of the banner editor element's design (see the case study).
01. Situation
CloudBlue, aIngrammicro's division, offers marketplace services for Cloud Service Providers and Resellers to sell and procure software and solutions. Recently, the small-business and medium-sized (SMB) Cloud Reseller companies requests diverse sales channels and customized marketplaces to expand the market, attract more end customers and increase their brand visibility. Therefore, CloudBlue has designed and developed a marketplace configuration and customisation tool and launched it in 2020 for the users.
02. Project Proposed
2.1 The Problem
However, customer feedback in 2021 unveiled both low product satisfaction and a limited number of SMB Reseller companies successfully setting up their first marketplace during or after onboarding. This issue poses significant challenges for both business and organization. Firstly, the underperforming product may result in a high churn rate. Secondly, it places a heavy workload on the Customer Success Team due to the influx of technical support requests. To tackle these issues, the Reseller Marketplace Product team and Product Designers launched an epic project to improve the user experience of customisation and configuration tool.
2.2 The business goal
The Product Team aims to launch the new product in Q4 2022, with the goal of increasing the number of SMB users who self-initiate the building and management of their marketplaces, to elevate satisfaction rates and enhance customer loyalty.
2.3 My Role
This project represents my first major endeavor since joining Ingrammicro. From December 2021 to January 2022, I conducted a high-level research on the entire configuration and customisation tool by identifying users, segmenting key user scenarios and journeys, and providing prioritizing projects. In February 2022, I dedicated myself to compiling the report and preparing for the presentation.
Following the high-level discovery and prioritization of projects, I initiated research projects focused on redesigning various parts of the entire tool.
03. Actions
3.1 Stakeholder meeting
I convened a stakeholder meeting, involving key individuals such as the Head of the Marketplace Product Team, Product Manager & Owner of the Marketplace, the Customer Success Manager, and Product Designers who involved in the project. In the meeting, I addressed the following questions to understand the scope of the project, roadmap of the new product, as well as if there are any existing research studies and what metrics are important to measure the success:
"What previous research has been conducted on the product?"
"Who are our target users?"“Can you pull out a list of the representatives of the users?”
"What are your expectations for the insights gained from this study?"
"How will we measure the success of the entire project?"
"Do we have a timeline for launching the new product?"
"What outcomes are expected from this project during the high-level discovery phase?"
3.2 Define the Objectives
From the stakeholder meeting, I defined the goal and objectives of the high-level discovery study:
Comprehend Key Task Scenarios and Construct User Journeys, identifying primary usability issues hindering users' self-setup and customization experiences.
Identify a list of usability issues based on frequency of encountering and impact (severity level) on user experiences.
Collect user insights and offer actionable recommendations to the Product Design Team to prioritise the upcoming projects.
Consolidate the findings into a cohesive report, including actionable recommendations.
To ensure confidence, this case study exclusively presents the User Journey and list of the usability issues, including actionable recommendations for issues demanding immediate attention and resolution.
3.3 Existing studies and hypotheses
Based on the existing knowledge, we have the following hypotheses:
The user journeys may be influenced by user demographics and the size of their companies.
User satisfaction levels may be influenced by their job function experiences.
Most users tend to complete only one task per login, or may not complete any tasks at all.
To validate the hypotheses we need to recruit participants from different region and companies, with different roles and functions.
3.4 Formulate Research Questions
In order to approach to the expectation of the stakeholder step-by-step, I need to break down the tasks into smaller ones by formulated a set of specific Research Questions, and conduct different studies to gather answers, the research questions are as following:
Who constitutes the user demographic? How many user categories are present?
What specific tasks do users endeavor to accomplish?
How do they typically go about completing these tasks? (e.g., the sequence of steps followed)
What points of frustration arise throughout their journey?
How should we determine the severity level of these issues?
3.4.1 Segmentate the users
Research Question 1.
Who constitutes the user demographic? How many user categories are present?
The first question is related to correlate the user’s specific tasks to their demography (especially, the function, experiences and region / country), so that I can recruit the right people and interview them with right topics and questions in the following interviews.
Descriptive Statistics
I consulted the Customer Success Team to provide me with user data reports on marketplace configuration and customization in 2021. As a result, I obtained statistics on users who actively logged in and performed tasks within the product, categorized by gender, function, region, and the company size.
I found that the active users mainly from 5 departments - Marketplace Admin, Marketing, Product Design, Engineering, and Sales.
The most representative roles include Platform Success Managers/Operation Managers, Content Managers, Web Designers, Cloud Marketing Specialists, and UX Designers. These roles can be categorized into Engineering, Marketing, and Product Design buckets.
3.4.2 Understand the Task Scenarios
Research Quesiton 2-4.
What specific tasks do users endeavor to accomplish?
How do they typically go about completing these tasks? (e.g., the sequence of steps followed)
What points of frustration arise throughout their journey?
To understand the tasks scenarios that different users try to complete across the current tool, I need them to walk me through the interfaces they used, show their actions along the way, and explain the difficult moments and the reasons behind.
User Interview
I conducted a series of 21 in-person interviews with individuals representing Small- and Mid-sized reseller companies, with the top 5 functions discovered above - Platform Success Managers/Operation Managers, Content Managers, Web Designers, Cloud Marketing Specialists, and UX Designers.
Throughout these interviews, I could gain a comprehensive understanding of their user experiences for each role. Specifically, I delved into their user journeys, examined the steps involved in task completion, and probed for any sources of pain points encountered along the way.
So that I map out the primary task scenarios and the steps to complete it, identify the essential actions that require in each step, and discover the pain points along the journey.
I analyzed all the interview transcripts within Dovetail, systematically labeling quotes related to roles, steps in the journey and key actions. Then I group them in goals, touchpoint and frequency etc.., so that I can quickly pinpoint the major task scenarios.
e.g. The majority of tasks completed by users from the marketing department including enabling the marketplace with their own brand, customising the skin, creating tailored marketing materials, and featuring products, the first two tasks align with the users from design department.
Main Task Scenarios
From interviews, I understood that the most-frequently tasks that the users need to complete include the following 6 tasks were the most-frequently required:
Presale the marketplace / key actions: being guided to MP Config and Customization tool
Enable the marketplace / key actions: configuring the GEO and language
Configure the marketplace / key actions: inserting the SSL certificate.
Customise the appearance of the storefront / key actions: creating the skin of the store, customising the layout.
Set the marketing contents/ key actions: adding Ads on homepage and product page, creating and managing banners.
Managing the service plan / key actions: setting the product category, collection and bundles.
The 6 main task scenarios outlined above represent a typical user journey for a Marketplace owner, admin, or operation engineer. Content designers, product designers, and sales specialists may interact with the interface primarily during tasks 4 through 6.
04. Customer Journey Map
By amalgamating users' task scenarios, the individual steps and actions within each task, and the pain points they encountered throughout their journey, I crafted a customer journey map. This map serves as the foundation for compiling a list of recommendations to prioritize problem-solving and project breakdown.
05. Prioritised issues
Research Question 5.
How should we determine the severity level of these issues?
Survey
I designed a survey with questions according to the main task scenarios and the pain points mentioned during the previous interviews, to explore which task scenario is most difficult to complete and which action(s) is the most frustrating step. Supported by CRM teams in the US, Benelux, Germany, and the UK, I sent the survey to over 20 CloudBlue customer companies. Finally, 63 valid results were collected.
The most challenging task scenarios are
Storefront customisation
Setting up marketing materials
Managing service plan
10 pain points were more frequently mentioned by multiple respondents with different roles
Descriptive Statistics
To cross-reference with the ticket requests sent to the Customer Success Team, I collaborated with the Product Team and checked the Feature Request report for marketplace configuration and customization in 2021.
For example, in US, 42% of the tickets were related to customization, particularly concerning customized skins. This corresponds to the finding that 34% of users identified storefront customization as the most challenging step in their journey. The data discrepancy could be attributed to repetitive iterations of colors and assets, suggesting a hypothesis for future investigation.
The following UX issues represent the critical concerns requiring prioritisation:
Problem 1. Users are not properly getting notified when an SSL certificate is invalid.
Problem 2. Users cannot flexibly customise the skins of the platform.
Problem 3. Users cannot easily hide products from the portfolio.
Problem 4. Users cannot easily add and edit banners on their own for better control over promotional content
06. Actionable Recommendations
Following a comprehensive primary research study, I identified the a list of problems that hold the most significant impact on usability.
The issue that requires immediate attention and resolution is as follows:
In the context of marketplace enablement, users are currently facing a critical incident where they do not receive proper notifications regarding invalid SSL certificates.
Users require assistance in conveniently hiding products that are not available for sale, which is impeding efficiency in product management.
Concerning the customization tool, it is imperative to prioritize the enhancement of the skin creator and banner editor functionalities. This enhancement will empower users with greater flexibility when customizing the platform's appearance.
Users cannot flexibly and efficiently showcase the marketing contents on the storefront, requiring an intuitive editor to add banners and other Ads.
Validation of the previous hypotheses
The user journeys may be influenced by user demographics and the size of their companies.
True, from the user interviews we figured out the user can be categorised by their roles, which can be categorized into Engineering, Marketing, and Product Design buckets, the roles might differ per company and depend on the size and headcount, for example, in a 10-person company, it might be the same person who enable marketplace, customise the skin and set up the products and bubbles from beginning to the end, but in a company with 200 people, each tasks would most likely completed by different person at different time.
User satisfaction levels may be influenced by their job function experiences.
False, according to the survey, there is no significant difference in satisfaction rates between new and experienced users. This suggests that the interfaces, in general, lack intuitiveness and simplicity.
Most users tend to complete only one task per login, or may not complete any tasks at all.
True,usually the marketplace owner and operation engineer could need to login at any step of the configuration and customisation, the other roles might only complete tasks of customisation and setting up the product lines on the storefront.
New hypotheses
From the study, there are new hypotheses generated based on user interview, survey result and descriptive statistics.
Marketing manager might need to repetitively iterate the colors of the skin and size of the assets for the storefront
Users at small companies tend to complete multiple tasks sequentially along the user journey, but this is not the case for users at medium-sized companies
Users at medium-sized companies, with specific roles in creating marketing content and managing product collections, may require swift navigation to their target pages, such as specific product or collection management pages.
Users with limited technical expertise may opt to upload an SSL certificate document instead of manually entering text.
In the upcoming study, the new hypotheses will require validation through research findings.
07. Feedbacks from the stakeholders
Both the Marketplace Product Owner and the Designer shows strong interests in customer journey map provides a solid foundation for reevaluating the scope and focusing on product lines. They suggest splitting the customization tool into appearance and product collection management components and prioritizing projects based on discussions with the Dev Team
The customer journey map establsih a good base to enable us to reevaluate the scope and realign our focus on product lines. To accomplish this, we should consider splitting the customization tool into two components: appearance and product collection management. Subsequently, we can prioritize projects according to the discussion with the Dev Team.
—the Marketplace Product Owner from Ingrammicro Cloud
To gain a better understanding of our target audiences, we appreciate the creation of personas with detailed profile for each, including their touchpoints with the products, will undoubtedly provide valuable guidance to our designers.
- Product Design Lead from Cloudblue UX Team
Afterward, the complete user journey was partitioned into different components, each managed by various product designers, POs and PMs, and researchers, to progress in parallel. This collaborative effort involved dig into user needs and frustration scenarios, generate new ideas, crafting concepts, conducting tests, and refining designs. I personally undertook certain components of this effort, including the enhancement of the banner editor element's design (see the case study).