FuguUX: Automating Usability Analysis

Product Manager and UX Designer | January 2024 - July 2024

FuguUX is a Pittsburgh startup building an AI-powered platform to automate usability analysis for digital products. As Product Manager, I led our team's research, design, and product strategy to validate the problem, define the solution, and guide the team from many concepts to a set of recommendations.

Visit FuguUX↗

Screenshot of the prototype FuguUX dashboard showing usability scores, flow insights, and prioritized issues

Project Details

Team

FuguUX

Carnegie Mellon Capstone Project

Role

Product Manager

UX Designer

Tools

Figma

Airtable

Loom

Using AI to Automate Usability Analysis

FuguUX is a Pittsburgh startup by two engineering founders with deep startup and AI research experience. We joined them as they began developing a new platform to support usability analysis using AI. In our initial discussions, they weren't sure if businesses would pay to automatically detect website usability problems and what format or additional tools might be needed. We began by challenging their assumed solution, leading the team to first test whether this problem was worth solving. We needed to find the right customer segment and prove market demand before investing time and effort into building a beta platform.

A bubble diagram for website usability, starting with the website in the center, and the different roles branching off.

The potential stakeholders for website usability, all of which are the potential customer or users of FuguUX.

Design and Research Strategy

I led our research approach around provocation rather than validation. Instead of interviewing with questions like "Would you use this?" we built concepts to create emotional reactions from potential users. These ranged from fully automating the design process to generating synthetic user data – ideas that, while controversial, helped us build an honest dialogue with our interview participants.

We interviewed 100+ UX practitioners across different companies and roles, using affinity mapping to code all interviews and identify key themes. I led sessions to develop our designs using several methods, including Jobs-to-be-Done to clarify user goals and matchmaking, where we paired AI capabilities with specific problem areas. These methods helped us design, build, and test over 40 prototype iterations. We narrowed down to two core concepts for our last round of tests – one focused on tracking user experience, the other on issue identification. We tested these ideas with UX professionals in person at Config, recruiting people in lunch lines for guerrilla sessions.

UX Decisions Get Stuck in Limbo

Our research revealed something surprising: finding usability issues wasn't the real problem – most UX teams already know their sites have problems. The pain points were around sharing insights efficiently, getting stakeholders to care about usability with business impact, and making sense of massive amounts of user behaviour data. This completely shifted our approach from issue detection to decision-making support. It was one of those lightbulb moments where we realised we'd been trying to solve the wrong problem entirely. Below are two representative quotes from our interviews that capture the kinds of challenges we consistently heard.

"Is there any data that you can give me, that can help prove to our BAs and PMs that we don`'t need this button?"— UX Designer

"Our challenge lies in narrowing down to the changes that will have the most impact."— Product Design Manager

AI Methods to Prioritise UX

Our final concept integrated web analytics with AI-powered usability analysis to give UX teams enough context to act on the identified issue. The key breakthrough was grounding AI recommendations in user data and business metrics rather than abstract usability principles. This solved the trust problem we kept hearing: 'This sounds great, but can I actually trust it?' Testing showed users went from sceptical to genuinely excited – one designer said in our later testing session that they 'would rather use this than Heap personally.' The solution supported the workflow: finding issues, understanding user context, and priotising fixes with business impact data that stakeholders could get behind.

Demo video highlighting AI-driven prioritisation and actionable insights for UX teams.

Implemented Solution

Following our research and recommendations, FuguUX purchased our IP and implemented several key features that weren't in their original roadmap. They switched to numerical scoring instead of letter grades for more understandable stakeholder communication, added severity-based issue ranking for better prioritisation, and included trend tracking over time to show progress. They also added effort estimation to help with resource planning, contextual screenshots for faster issue comprehension, concrete examples with solution suggestions for actionable insights, and research citations to provide credible evidence for the recommendations. I felt rewarded to see our research translated into FuguUX's product decisions that addressed the core problems our team identified.

Screenshot highlighting new features: numerical scoring, severity ranking, trend tracking, and actionable insights

Some of the inclusions based on our reccomendations, including numerical scoring, severity ranking, and actionable insights.

FuguUX is now gathering test audiences, preparing for their product launch. Follow the link below to play with one of their beta reports.

FuguUX Wayfinder Beta Report↗