Be Smart: Data and Intuition

Dr Viktor Dörfler, Department of Management Science, discusses how better data analysis may not in itself be enough for better decisions, and how not to underestimate the importance of intuition.

"Data data everywhere but not a thought to think." (Theodore Roszak: The Cult of Information)

With IS/ICT development we can handle more and more data, and we can do this faster and faster. Furthermore, data is produced at an ever-increasing rate, so it is important that we have fast and capable data processing. However, it is dangerous to start believing that better data analysis is sufficient for better decisions. The essence of this problem is that we can only analyse what the databases contain, and the world is far richer, far more colourful, and far more human than this. We must not forget that for good decision making we also need those things that humans can perceive, but are not contained in databases. Don’t make the mistake of believing it is only a matter of time until everything is in databases; data will always only be just data, and never meaning.

What I would say to business executives is “Be SmArt”. What it means is: don’t underestimate the importance of intuition, and at the same time don’t throw out all the data analysis in favour of doing everything intuitively. SmArt includes both data analysis and intuition, always in a fruitful mix, achieving even more when they mutually inform each other. For instance, the intuition of an expert may help develop new analysis, which may fuel new intuition, and so forth in cycles. In order to achieve this, executives will need to create organisational culture that enables SmArt context. They need to create a culture in which analytical and intuitive thinkers co-exist, mutually respect each other, and where it is OK to transition from one to the other or to be both. It is important to understand that using intuition does not have to be untidy, unsystematic or lacking explanation. There are appropriate IS/ICT tools, primarily causal maps and expert systems, which can help make the process orderly.

I have conducted in-depth interviews with 17 Nobel Laureates, and my former PhD student Marc Stierand conducted similar interviews with 18 of the best chefs in the world. Both of these studies indicated that intuition plays a crucial role in high-level creativity. I would even risk the following assertion: no significant creative outcome can be achieved without intuition.

The other area where intuition plays a significant role is the decisions of top executives. Therefore, we need to help decision takers and experts, particularly those who engage in creative problem solving, reliably develop their intuition in addition to using analysis. For this, it is of immense importance to support master-apprentice relationships, and develop highly performing hubs that I call ‘hot spots’. Although master-apprentice relationships are today seen as inappropriate as they are very asymmetric, they are the only way we know of for transferring tacit knowledge. This gains particular significance when we realise that knowledge becomes more and more tacit with the increased level of expertise. The real challenge is that it is wrong both to follow and to abandon the master's way, however, from this struggle the new master may emerge. The master-apprentice relationships take place in workshops but a large workshop may include several masters and a number of apprentices. I call 'hot spots' those workshops that consistently deliver exceptionally high performance.

Working with executives in the KnowlegeBrief Innovation Programmes recently was an enlightening experience. Immediately, they intuitively grasped the importance of intuition. However, their analytical minds seemed to quickly kick in and question their intuitive minds. This is natural: all of our education was focused on developing our analytical mind, neglecting or even suppressing the intuitive mind. However, as the participants started thinking about implementing in their organisations what they learned, their viewpoint expanded, leading to a deeper understanding of the interplay between analysis and intuition.

In our Western societies we tend to exclusively focus on the intellectual aspect of the human mind, support it with data analysis, and more recently artificial intelligence. Intuition is usually neglected, and the technology supporting it is marginalised. I believe that for good decision making and creativity we need both analysis and intuition as well as the technologies supporting both of them.

As Henri Bergson in 'Creative Evolution' said: "A complete and perfect humanity would be that in which these two forms of conscious activity should attain their full development. [...] In the humanity of which we are a part, intuition is, in fact, almost completely sacrificed to intellect."

This is based on a blog which first appeared on the KnowledgeBrief website