I was confronted with this dilemma last week while discussing the implications of a new feature release with a product team. The aforementioned release would introduce some changes that could potentially make a small percentage of the existing user base of the product unhappy. But no one really knew the size or the extent of unhappiness caused here. Should the product team be proactive and try to fix the problem by delaying the launch, or instead, be agile and launch immediately, only to react to problems if and when they arise afterwards? What if one hour after the release, one user voices very strong dissatisfaction with the release. Do you roll back the release or wait until you receive complaints from at least x% of your users before you decide to do something about it?
I think there are 2 keys to resolving this dilemma in any organizational setting: (1) Clear Product Vision, and (2) Personal Experience. A clear product vision in the organization enables everyone involved to make proactive decisions by prioritizing those decisions against the furtherance of that overarching vision. If the organizational product vision is to provide the "best customer service in ecommerce", for instance, then the decision on whether or not to argue over a customer's desire to return a merchandise can be easily made without having to make elaborate calculations about the impact of the decision. Personal experience, on the other hand, enables the decision maker to spot trends early on and extrapolate inferences from limited data sets that may not, in and of themselves, be statistically significant.
So it seems quite important for the leadership in a startup to recognize the inherent dilemma between data-driven decision making and proactive leadership and to take steps to ensure that there is a clear product vision driving the organization forward, while at the same time leveraging prior experience of team members, advisers or consultants in making inferences from past data.
(Photo Credit: livescience)