When should an experiment become a product?
Keep experimenting until the signal from real behavior gets strong enough that users are asking you to keep going.
An experiment is ready to become a product when the evidence starts pulling you forward instead of you pushing it. That evidence takes different forms: repeated use, strong retention, willingness to pay, enthusiastic referrals, or a problem people keep bringing back to you unprompted, without you having to ask whether they’d still want the thing you built.
Behavior, not optimism, is the signal
The important part is that the signal comes from behavior rather than optimism. Excitement from the team building it doesn’t count as evidence — usage does. It’s easy to mistake internal enthusiasm for external demand, especially after investing real time in something. The team’s belief in an idea is not independent evidence about whether the idea is working; it’s a byproduct of having built it.
This is why we try to separate “are we excited about this” from “is this actually pulling people back,” even though the two feelings can be uncomfortably similar from the inside. One question is about how the team feels. The other is about what’s actually happening in the world, and only the second one should decide whether to keep investing.
A lab should be comfortable with both outcomes
If the evidence is weak, stop or change direction without ceremony. If the evidence is strong, invest more. This approach keeps resources aligned with reality instead of sunk cost — and sunk cost is a genuinely strong psychological pull that’s worth naming explicitly, because it quietly argues for continuing something regardless of whether continuing is actually justified by new evidence.
Being comfortable stopping something is, in a real sense, a skill that has to be practiced. Teams that never stop anything usually aren’t disciplined — they’re avoiding a difficult conversation about resource allocation by defaulting to inertia.
It’s a shift, not a ceremony
The transition from experiment to product isn’t a single dramatic moment. It’s a gradual shift in confidence as evidence accumulates over weeks, not a launch event with a clear before and after. Looking for a single moment when “it became real” often means looking for something that doesn’t actually exist — the accumulation is the whole story, and trying to pinpoint one decisive turning point inside it usually just adds false clarity to what was actually a gradual process.
Common questions
What’s the clearest single signal that an experiment is working? Unprompted return usage — people coming back to something without being reminded, nudged, or re-engaged. Retention that requires constant prompting is a weaker signal than retention that happens on its own.
How long should you run an experiment before deciding? Long enough for a genuine usage pattern to emerge, not just an initial reaction. The right duration depends on how often the underlying task naturally recurs — daily-use products reveal signal faster than quarterly-use ones.
What if the evidence is mixed — some strong signals, some weak ones? Mixed evidence usually means the experiment needs a sharper, narrower version rather than a full stop or a full green light. Ambiguity is often a sign the test itself wasn’t specific enough, not that the underlying idea is a coin flip.
Takeaway: don’t wait for certainty — wait for a signal strong enough to justify the next investment.