The pilot worked. What is ready to grow?

During a pilot, the team may be part of the product. They explain how it works, notice when someone struggles, arrange a replacement and remind people to come back. Some of that help may be essential to the experience. Some may disappear when the offer reaches more customers.

A successful pilot shows that an offer worked under the pilot’s conditions. What is ready to grow is the part of that success that will still be true when the offer reaches more customers on different terms.

Imagine a trial of a household-refill service. Participants order cleaning products for delivery and return the empty containers. They use the products, several reorder, and feedback is encouraging. It’s an illustration, not one of our engagements.

The trial team has also been calling when supplies run low, adjusting deliveries and sorting out missed collections. That attention has helped the service work in people’s lives. If the broader launch assumes customers will manage everything themselves, the business may be proposing a different experience from the one that produced the positive response.

The conditions of success are part of the evidence

It would be a mistake to dismiss the pilot because it involved extra help. Early trials often need assistance to make an unfinished service usable. Without it, a team might learn mainly about problems it already knows how to fix.

The more useful question is which contributions created value. The phone calls might have explained an unfamiliar process that became easy after a few deliveries. They might have made up for confusing instructions. Or people might have welcomed having someone else notice when they were running out.

Those explanations suggest different development choices. The first may require good introductory support. The second may call for a simpler experience. The third could reveal a benefit worth building into the service.

Removing all assistance wouldn’t necessarily produce a fairer test. It might remove something customers value. Equally, assuming that personal attention can be retained without considering its cost would leave an important commercial question open.

A larger introduction changes more than the number of customers

The first participants may be unusually interested in refills, comfortable with the brand or willing to tolerate inconvenience while something develops. Their involvement can be valuable. It doesn’t make them interchangeable with everyone the launch intends to reach.

A broader introduction might change the price, the delivery area, the alternatives customers encounter and how much explanation they receive. Some of those changes may improve the proposition. Others may make it harder to choose or keep using.

The same problem troubles public policy. In the American Economic Review, Al-Ubaydli, List and Suskind note that it is common for effects found in small-scale studies to “diminish substantially in size when applied at the larger scale.”

This is why a result needs to travel with its conditions. A reorder tells the team something important about the experience someone had and the offer available to them. It doesn’t explain every reason for that choice, or establish that another audience will respond similarly under different terms.

Interviews and observation can help explain what happened. Transaction and usage evidence can show whether the behavior continued. Their value depends on the question each is capable of answering. Treating one type of evidence as a substitute for every other type can leave the decisive uncertainty untouched.

Give the next investment a specific purpose

An early pilot may have been designed to establish whether the product works, whether people can use it or whether an experience is worth developing. A positive answer to that question deserves recognition. The team doesn’t need to pretend it has answered every later question to justify making progress.

For the refill service, the next commitment might be a broader paid introduction with a more settled delivery model. It might be further development of the replenishment experience. The right step depends on which uncertainty could materially change the opportunity.

Rita McGrath and Ian MacMillan’s Discovery-Driven Growth makes an established case for testing a venture’s assumptions as it develops. The practical issue here is concrete: deciding which conditions behind this encouraging result the business will preserve, change or investigate next.

A pilot can reveal a stronger proposition than the one the team started with. In the refill example, the opportunity might lie as much in taking care of replenishment as in changing the container. Recognizing that could improve the offer before the company expands it.

The best use of a successful pilot is to understand what made it successful. That gives the business something more useful than a general declaration of readiness: a clearer account of the opportunity it is preparing to pursue.