WhollySoftware
Back to blogDesign

User Testing on a Budget: Getting Signal Without a Big Research Team

Wholly Software TeamApril 17, 20255 min read
User Testing on a Budget: Getting Signal Without a Big Research Team

Nearly every startup client we work with wants user testing but doesn't have a research function to run it, so we've built a lightweight version into our own process instead of treating it as a separate, fundable phase. Five unmoderated tests on a clickable prototype, run through a low-cost remote testing tool with the client's own waitlist or existing users, has consistently surfaced the majority of usability problems we'd otherwise catch much later and more expensively in production.

We stopped chasing statistical significance and started chasing pattern repetition instead. If three out of five people click the wrong element for the same task, that's a real signal regardless of sample size — usability problems tend to be binary (people get stuck or they don't), unlike preference questions, which genuinely do need larger samples to trust. Knowing which kind of question you're asking determines how many participants you actually need.

The cheapest testing we run isn't formal at all — it's watching a client's own support or sales team demo the product to a prospect. Those calls surface confusion in real time, unscripted, and cost nothing beyond sitting in on a call the client was already having. On one B2B dashboard project, watching four sales demos told us more about where prospects got confused than a week of scheduled research sessions would have.

Where we do spend limited budget deliberately is on recruiting the right five people rather than any five people. A convenience sample of coworkers or friends reliably fails to catch the problems that actual target users hit, because coworkers are too design-literate and too polite. We'd rather run three real target users than eight people who don't represent anyone the product is actually for.

User ResearchUX TestingStartups
User Testing on a Budget: Getting Signal Without a Big Research Team — Wholly Software