TestSprite is an AI-powered software testing platform helping developers and QA teams automate testing. TestSprite has gained 6,000+ registered users, executing hundreds of tests daily. In this project, I worked on introducing a Monitoring feature that enables users to schedule automated test runs and receive clear, actionable results.
My Role
Founding Product Designer
Timeline
4 month
Collaboration
4 Engineers, 1 PM, 1 Data Scientist, 1 QA tester
Status
Shipped
Problem
Users were actively creating tests, but very few converted to a paid plan.
Through analyzing existing data and user analytics:
Low conversion rate
Out of 6,000 sign-ups and over 2000 active users, there's only ~20 paying users.
Solution
Monitoring feature that enables users to organize tests and schedule automatic re-run to ensure product stability.
+400%
Users converting from free to paid plans
+32%
Increase in user retention
Discover
Understanding the barriers that prevented users from converting.
Talking to power users.
To investigate, I analyzed existing support ticket & joining customer calls to understand where users are struggling, and found:
Users lacked visibility into test performance over time.
There's no way for users to easily collaborate with teams when having test errors.
Phasing out delivery of value
With the user feedback, I decided to collaborate with PM and SME to map out more detailed user journey, helping us to identify opportunity and define our phase 1 MVP feature scoping.
Define
Creating a streamlined test maintenance process.
Defining the the test maintenance flow.
With the feature scope defined, I collaborated closely with the PM and engineer to map out the user flow to ensure the experience was technically feasible, aligned with user needs, and supported a smooth end-to-end test maintenance process.
Mapping out information architecture to keep track test results.
With the new test list feature added to existing concepts like tests and test details, we refined the information architecture to help users understand the relationships between elements and navigate the platform more easily.
Creating low-fi to visualize initial thoughts
With the user flow and information architecture defined, I created the low-fi wireframe. A key decision I made is to add a "Recent execution" column in monitoring, helping users to understand recent test status and provide an easier way to identify anomolies.
Design Exploration
HMW provide intuitive test report that allow users to effectively take actions?
SME feedback showed users care most about critical issues as too much detail makes prioritization harder.
Detailed test result
Highlighting critical issue
HWM provide intuitive monitoring results to keep users updated?
We combined approaches to help users scan test health quickly and track trends over time for proactive issue detection.
Monitoring Dashboard
Recent execution indicator
Test trend for each schedule
Improvement
Conducted usability testing with 5 current users.
Move test metadata to left side panel.
User mentioned that when viewing test results, they prefer to view code & instruction without scrolling.
Final Solution
Schedule automatic test re-run to ensure product stability.
Enabling users to automatically re-run & and monitor recent test results.
Monitor past execution results to identify errors and anomolies.
Provide execution history to help user effectively keep track of product status.
More flexible way in organizing test to support maintenance needs.
Providing test list to help users effectively organize and manage tests.
Test report to facilitate collaboration.
Allow users to download and share test results to inform dev teams.
Takeaways
Innovating Without a Playbook
What made this project even more exciting—and challenging—was that we didn’t have a clear competitor to benchmark against. There was no template to follow, which meant we had to create something original. That’s where collaboration became crucial. I leaned heavily on feedback from PMs and QA Tests, and engineers. Some of our best ideas came out of these conversations. It reminded me how important it is to listen, brainstorm freely, and stay open to unexpected directions—especially when building something new.
