Psychology Today had an article last year entitled “Why Does Overthinking Sabotage the Creative Process” where they stated:
Many scientists believe that the creative process springs as much from the subconscious as it does from a conscious thought process. Most often, creative solutions are not wrestled from your mind through sheer force of will.
This was followed up by:
“Eureka!” moments tend to occur spontaneously, almost always when the conscious mind is thinking of something else, or nothing at all.
While the entire article was interesting there was a specific point that I wanted to bring up. Apparently, you can overthink a problem. Some research done for the Journal of Neuroscience indicated that “...too much activity in specific regions of the prefrontal cortex was the likely culprit behind ‘paralysis by analysis.’ “. We’ve all heard that phrase before ‘paralysis by analysis’ or something similar. It’s basically telling us that we are analyzing things so much that we’re not moving forward.
In the IT world, this was the most common problem with the Waterfall approach to application development when the system being developed was exceedingly large or complex. So much time was spent analyzing the situation, trying to build the ‘perfect’ system, that the system is never built. Or is ten years and a jillion dollars over budget.
It doesn’t have to be this way. The Agile methodology was built around those concepts, the idea that you can build something and still make changes to it at the same time. This scares people. It most notably scares people who like control over things. One of the things that the Agile Manifesto talks about is “Responding to change over following a plan”. But isn’t that what IT has been doing for decades? Following a plan? This is a mind shift and one that many people can’t grasp.
For instance, should an organization be following a ten-year plan and not respond to change during that time period? We all realize that if things change we need to adapt to the changes. So why is it that we have ten-year plans in IT? Why is it that we think it is perfectly acceptable to build something that was designed ten years ago. Things change and we need to change with them.
We are literally overthinking things. The Psychology Today article talks about overthinking as “relying exclusively on the brain’s higher-level, executive-control centers held in the cerebrum.” That is what we are trying to do when we try to analyze a system to the nth degree. We are relying exclusively on our brain’s ability to analyze the system, we are focused on the higher-level functions of our brain when we should be relying on our creativity. Indeed, recent studies have shown that overthinking “actually impairs, rather than enhances, creativity.” Yes, all those binders of documentation? All those JAD sessions? They actually hurt us as opposed to helping us.
There are reasons why we shackle ourselves to “overthinking” and “paralysis by analysis”, but are they the right reasons? Couldn’t the same purpose be accomplished in a completely different manner? If you’re worried about the budget, doing things in smaller segments will get you a product faster and allow you to start getting those benefits without having to wait for the big bang implementation.
There are reasons for doing almost anything, but by overthinking we are doing ourselves a fundamental injustice by sabotaging our ability to think creatively.