UX METRICS
SYSTEM.
Translating "Design Pillars" into data: A multi-tiered measurement system for Alaska Airlines' e-commerce platform.
SITE-WIDE MEASUREMENT
STRATEGY & OPS
EXECUTIVE MONTHLY REPORTS
Quantifying "Good Design"
Management established four new design pillars—Simple, Consistent, Relevant, and Engaging—but had no way to track them. My task was to build a system that reflected the state of these pillars alongside traditional metrics like task completion and customer satisfaction.
The challenge was to deconstruct these abstract adjectives into measurable constructs, ensuring that when a user said a page was "Engaging," they meant exactly what we intended.
-
01
THE SURVEY SUITE
Started with a general experience survey (intercepts) and evolved into URL-specific "Single Experience" surveys and "Micro-Surveys" for feature updates (e.g., Filters).
-
02
SELF-SERVE DASHBOARDS
Built automated dashboards at three levels: Director (High Level), Product Group, and Product Team, aggregating Design Scores, CSAT, and Session Replays.
-
03
THE TOP 10 REPORT
Created a monthly executive report highlighting the top usability issues, calculating "Population Impact" and tracking the "Months on Radar" to drive accountability.
CONSTRUCT DEFINITION
Conducted academic research to deconstruct the pillars (e.g., differentiating "Relevant" from "Engaging") and piloted survey prompts via unmoderated testing to ensure clarity.
SURVEY ECOSYSTEM
Deployed a mix of General Intercepts, Post-Task Surveys, and A/B Test complement surveys to capture both macro sentiment and micro interactions.
MIXED METHODS ANALYSIS
Paired quantitative survey data (CSAT, Task Completion) with qualitative Session Replay data to discover "Generative" insights beyond simple validation.