Novelty Search - A Fresh Approach to Doing Great

A few months ago, I started a newsletter. My long-term goal—though somewhat secret, I admit—was to build an audience that could perhaps become a full-fledged activity one day. It was ambitious, but I was confident. After all, I needed to be well-identified and consistent in my efforts.

So, I structured everything:

• A method: telling stories supported by graphics and data.
• A name: Infografix.
• A tone.
• A logo.

In short, a nicely wrapped-up “package.”

I published a few stories and received positive feedback. It sounds like the beginning of a good story.

But I couldn’t keep it up.

Was it too difficult to fully commit to my job, spend quality time with family and friends, exercise, cook, eat healthily, and, on top of that, release a biweekly publication?

I don’t think that’s the real reason. We always make time for what matters, and this project genuinely mattered to me.

The real issue was that I confined myself too quickly to a narrow framework, setting constraints before fully exploring what truly suited me. I forced a goal upon myself.

Little by little, what initially excited me turned into a chore I imposed myself to complete.

It was then that I read a book that got me thinking and changed the way I see the future and the projects I'm involved in. This book is: Why Greatness Cannot Be Planned: The Myth of the Objective [1].

This article aims to share some of the book's ideas that resonated with me.

The Myth of Long-Term Goals

The book’s central point is that setting distant and challenging goals can often be the best way... to never achieve them.

Conversely, the authors argue that pursuing novelty through naive and random exploration can be far more effective.

The issue with complex, long-term goals is that it’s often impossible to predict in advance what indispensable milestones will be required to achieve them.

One example the authors frequently highlight is the computer: it could not have been developed 1,000 years ago because many essential technological building blocks were missing.

But who could have known what those building blocks would be? Initially, no one could have guessed that vacuum tubes would be a key step in creating the first computer. If the initial goal had been to “build a computer,” perhaps no one would have worked on this seemingly unrelated yet vital invention.

Worse, we often rely on performance metrics to monitor the progress toward the end goal. But, when the path to the goal is unknown, these metrics can act as misleading compasses, leading us astray rather than guiding us.

We may follow misleading compasses, but we are not aware of it.

Take a maze, for instance. It’s not uncommon that, to reach the end, you first need to move away from the objective. If you navigate solely by minimizing the distance to your goal at every step, you may actually take longer to complete the maze than if you were to choose your path at random.

Another challenge you might face when trying to stick too closely to your initial plan is the risk of becoming blind to opportunities that arise along the way. You might miss unexpected shortcuts, valuable insights, or even new paths that could lead to a better outcome than your original goal.

So, what’s the alternative? After all, we’re all profoundly familiar with setting and striving to achieve ambitious objectives. That’s how we’re educated in school, how career paths are designed, and how we operate within companies.

Setting objectives isn’t inherently bad, it’s essential for navigating the short and mid-term. However, in the long run, we might achieve better outcomes by adopting a different approach to the future.

Exploring the Unknown

Broadly speaking, innovation can be thought of as a quest in a vast space filled with rooms containing odd trinkets and seemingly useless objects. The layout of this space and its rooms is entirely unpredictable.

Sometimes, we need to clear out an apparently uninteresting area to discover, at the end of the path, an extraordinary treasure.

If you'd stuck strictly to your objective, using a monitoring rule like “find something a bit more interesting than the last item”, you'd never have stumbled across the most valuable discovery.

Not only can exploring the unknown lead to unexpected and valuable discoveries, but it is often in the shadowy, uncharted areas, those that the pursuit of precise objectives would never take us, that the greatest treasures lie.

Exploring the unknown can be incredibly rewarding, even if we cannot predict the form or nature of the reward.

Sometimes, the best way to achieve something great is to stop trying to achieve one specific great thing.

Serendipity

Newton understood gravity by watching an apple fall, Fleming accidentally discovered penicillin, and Zuckerberg created Facebook by building a college project...

These are clichés, but their slight part of truthfulness underpins the following idea: History is full of examples of sharp minds exploring spaces in a somewhat random fashion, recognizing opportunities to discover, invent, or achieve great things.

This is the concept of serendipity: discoveries that aren’t strictly accidental, as they require hard work and a good dose of genius, but that aren’t guided by a precise goal either.

What can be hard to grasp, especially when we're deeply immersed in an objective-driven mindset, is that many of humanity's most outstanding achievements wouldn't have been possible if they had been pursued with rigid and painstakingly defined goals.

If Zuckerberg had initially set out to create the world's largest social media platform, he likely would have failed—primarily because he couldn’t have known what key stepping stones would be essential to achieving such a vision at the outset.

While Facebook may not be one of humanity's greatest achievements in the sense of being universally good or beneficial, it undeniably stands as a breakthrough due to its sheer impact on our lives. As the first major social media platform, Facebook redefined how we connect, communicate, and share information on a global scale, influencing individual relationships, politics, business, and culture.

What's equally fascinating is that, despite humanity’s rich history of speculative fiction, the rise of the internet and social media as dominant forces in our lives was almost entirely unanticipated. This unpredicted transformation highlights how impossible it is to fully foresee the future, as time allows possibilities to evolve and reshuffle in ways we cannot anticipate.

Being open and flexible to opportunities is more important than knowing exactly what we’re trying to achieve.

The Power of Novelty

Returning to the central idea of the book, Greatness Cannot Be Planned, how can we then seize the future? How can we navigate a path toward a better, or at least more manageable, life?

Rather than confining ourselves to a “goal-oriented mindset,” the authors suggest prioritizing the pursuit of novelty for its own sake. By exploring what is different or surprising, we can make unexpected advances. This is how great discoveries, like penicillin or the microwave, came about: they resulted from curiosity, exploration, and serendipity—not rigorous planning.

Of course, for small objectives, the search space is well-charted, and it makes sense to make plans and follow in the footsteps of previous explorers.

But your own future has never been explored. No one knows the stepping stones that will bring you fulfillment and happiness.

By shifting from “setting goals” to “choosing areas to explore,” we free ourselves from the constant pressure of achieving objectives and instead value exploration for exploration’s sake, novelty for novelty’s sake.

Food for Thought

So, what are the practical implications of novelty search instead of objectives targeting?

How to implement this new approach in daily life?

I'll give you a simple conclusion.

Do what you genuinely find interesting without overthinking the outcomes. Stay focused on walking to the next stepping stone, and stay attentive to seize hidden treasures.

Better ask where we can get from here rather than how we can get there.