David Brooks has a great op-ed this morning on the Philosophy of Data. He argues that data offers one major advantage and one major drawback. Data enables humans to discover patterns otherwise unobservable by our senses/intuition or patterns that violate human intuition. But the religion of data engenders a fallacy: that everything can and should be measured; and with this data, the best answer will emerge.
Belief in the power of data has become a sort of religious debate which has manifested itself in product design (data driven vs intuitive design), in politics, in baseball, in climate change debates and many others. At some level, it’s an extension of the religion/ science debate: what I feel/believe vs what my instruments indicate. Which is better?
Great innovations can come from both disciplines. Apple’s product design is built from intuition and belief - not from programmatic user testing. On the other hand, Google innovation in search focused exclusively using large scale multivariate testing and metrics optimization.
I believe the best innovations blend the two, mixing art and science. Great product ideas are borne of intuition but honed and fine tuned using data. In fact, when a company or product has reached has plateaued, when innovation has reached a local maximum, it’s often because one of these two disciplines has dominated thought. And it’s time to swing the pendulum back the other way.
Google’s recent focus on design and the reinvention of search’s user interface in Google Now exemplifies such a transition. A traditionally data driven company, Google has reinvented the feel of their brand and products using intuition.
In this debate as with most, zealotry will often command attention, but compromise will drive success. Both ways of thinking promise breakthroughs and innovation. Mastering both means balancing the two.