Enterprise Chat Experience | UX Strategic Research
Improving Enterprise Chat through Strategic Research
Company Info
Fortune Global 50 Manufacturing Software Company — Enterprise | $5B+ revenue
Role
Lead UX Researcher
Team
1 Product Manager, 5 Software Developers
Duration
2-week Lean UX sprint
Results / Impact
Estimated the benchmark initiative could increase customer satisfaction by approximately 10-15%, if KPIs are established and tracked.
Successfully demonstrated the value of UX Research efforts and my work on a distributed team that was previously hesitant.
I feel very proud to have received such impactful feedback from stakeholders who said they learned so much about customers from the research I presented.
Project Overview
I was responsible for strategic research on enterprise Chat to prioritize and influence product goal strategy due to ambiguity surrounding the current Chat experience.
Competitive Heuristic Evaluations
Identified usability problems with the existing Chat tool, spanning from minor to catastrophic issues, considering priority as well as severity.
I wanted to compare what other companies did well and didn’t do well with their Chat experience. Providing a score for each heuristic, I presented to senior leadership and the team what areas of improvement and successes Chat exhibited. I used an adapted version of Jakob Nielsen’s 10 Heuristics called Conversation Design Heuristics.
Competitive Heuristics for Siemens, Autodesk, Amazon, Adobe
Competitive Heuristics for Siemens, Autodesk, Amazon, Adobe
Competitive Heuristics for Siemens, Autodesk, Amazon, Adobe
Competitive Heuristics for Siemens, Autodesk, Amazon, Adobe
Top Chat Usability Issues - My recommendations of top priority and/or top severity issues
Detailed Heuristics Report
Drilled down deeper into a more detailed heuristic evaluation to identify what was working well and what improvements could be made, specific to the Company’s Chat, apart from competitors.
(Selected Heuristic Evaluation Report slides below.)
Chat improvement recommendations that will bring big wins, including low hanging fruit (simple to implement quickly) and slightly more resource-heavy changes.
Heuristic #2: Match Between System and the Real World
Heuristic #3: User Control and Freedom, Score = 4 (Critical Usability Problem due to customer being unable to perform their task)
Heuristic #4: Consistency and Standards, Score = 3 (Severe Usability Problem, Potential impact on task completion, but a workaround exists)
Heuristic #5: Error Prevention, Score = 2 (Minor Usability Problem, No impact on task completion, but slows down / irritates customer)
Heuristic #9: Help Users Recognize, Diagnose, and Recover from Errors, Score = 3 (Severe Usability Problem, Potential impact on task completion, a workaround exists)
Customer Quotes
Compiled data of comments made by actual customers and identified mis/alignment with usability testing results.
Real quotes from customer surveys on the marketing site, using Refiner (survey software for SaaS and digital products).
Benchmark
The team lacked a clear method for tracking metrics, which resulted in roadmap ambiguity. I proposed benchmarking the current experience to allow comparisons with the desired experience later.
The proposal consisted of first determining appropriate metrics and a study methodology, then tracking these metrics across different releases of the product by running studies that follow the same established methodology. This would hold the team accountable and provide clear documentation of improvements. Before I joined the team, they were just “going with the flow” and had been forced to adopt a “we’ll fix it later” attitude. I was able to convince the team to put in a little more effort and work up front for a big impact down the road.
Benchmark to track metrics - Including First Response Time, First Contact Resolution, Number of chats, and more...
Benchmark to track metrics - Integrating live chat with the CRM would allow popular customer inquiries to be answered quickly and efficiently.
Benchmark to track metrics - Keeping response times low builds trust with customers, as does asking customers for feedback in a timely fashion and appropriate manner.
User Flows
Formed User Flows for three Personas to help stakeholders visualize real experiences customers have with Chat, including being essentially “hung up on” by Agents.
Chat flow of Prospective Customer Ravi who rejected cookies when first visiting site and is now unable to use Chat.
Chat flow of Existing Customer Jim who accepted cookies when visiting the site and enters Chat via Contact Us page after hours.
Chat flow of Prospective Customer Jules who accepted cookies when visiting the site, and enters Chat via the orange Chat button on home page.
Results / Impact
Estimated the benchmark initiative could increase customer satisfaction by approximately 10-15%, if KPIs are established and tracked.
Successfully demonstrated the value of UX Research efforts and my work on a distributed team that was previously hesitant.
I feel very proud to have received such impactful feedback from stakeholders who said they learned so much about customers from the research I presented.
Constraints & Limitations
What would I do differently?
This was among the final products I worked on at this Company prior to non-performance-related layoff.
While I transitioned out of the company before the solution could be executed, the recommendations I delivered remain a key resource for the team to continue refining the Chat experience.
I would have like to follow up with iterative research, including validation testing after designs were implemented by development.
I also would have loved to see the product through more iterations of research, design, and implementation to fully realize the impact on customers.