The 21st century project challenge: leading with data

As we move through the 21st century, data and the insights it gives us will change how we see the world. However, it's chaotic. It is particularly frustrating for project managers tasked with delivering projects on time, to cost and scope. As we stand at the edge of the proverbial data lake, we ask ourselves: how can we swim across?

We implore our organisations to play their part and set the stage for successful delivery. However, this can feel like an uphill battle. Traditional ways of thinking and behaving hinder our attempts at digital transformation.

I believe that a significant responsibility falls on the project delivery profession. We must take the very best of what we know, of how to comprehend scope and uncertainty, how to gauge and price work, and how we deliver the work, manage stakeholders and control change – and we must lift our game in all these respects, using data. I call it data-driven project management.

Lead through example

By leading through example and communicating the art of the possible, we can set new standards. We can set the expectation that data leads to change and delivery. I'm heartened to note that a wide range of developments is making this possible.

From a tool and technology perspective, we are now awash in distributed, computational power. The giant online retailers demonstrate just how effective modern database-driven systems are. Our telecommunication networks are not far behind either, and the impact of the internet is undeniable.

Where the gaps lie are in the processes and people making use of these technologies. Project managers will need to include a new wave of data architects, data scientists and data engineers in their teams. These professionals describe capture and reason about the world in a way that makes project delivery smarter. Project teams must overcome data bias, our tendency to cherrypick the data that supports our hypothesis and discard the elements that do not.

The same engineers will ensure the data is clean and as accurate as possible, so we do not experience 'negative training' - training in the wrong thing.

Join the dots

In parallel, we must build organisations and cultures that no longer operate in silos. The most successful organisations will be those that can exploit data across the business.

These new organisations will set the expectation that every project will be better than the last because every project learns from all of its predecessors. The word 'all' is deliberate here. Through machine learning, mathematics, statistics and careful modelling of projects, we can use the past to inform the future.

Organisations that choose not to collect and examine their data exhibit systemic data bias. They err toward overconfidence or – just as damaging - timidity in pursuing commercial opportunities that they could take.

With lack of knowledge comes false confidence in the outcome, sufficient at times to jeopardise not just the project but at times, even the business.

To overcome this, we must develop a data-driven learning cycle. For this work, we must systematically measure all aspects of project performance.

Measuring everything

We must measure much more than financials – that's the tip of the iceberg. The organisations we work within must provide the right foundations (quality systems, culture, training and support) to enable the business to be self-critical and learn in an unbiased way.

Every element of the learning cycle must operate at the same level. Another trend is emerging: machine and project manager side-by-side, each bringing their strengths to bear.

By placing and exploiting machine knowledge at the heart of project delivery, we can replace optimism bias and selective recall with a relentless focus on using past data to predict the future. We're not looking for a scenario where: 'computer says no'.

Instead, we're seeking guidance, where the algorithms operating on big data tell us where our predictions need improving, and – critically – the reasons why. When we use applied artificial intelligence (AI) techniques such as machine learning, we must insist upon 'explainable AI': systems that can give the ideas and probabilities for their conclusions.

Build better cultures

Finally, we must set the cultural expectations. Operations and portfolio staff can no longer hand-pick "high risk" projects to review.

The business must look at all projects uniformly, in-depth, regularly and in real-time. Similarly, citizens will expect the government to master data sets across all their realms. We are seeing the rise of data trusts in which data is shared for collective benefit, overcoming the commercial silos that hinder learning, optimisation and the markets in general.

If we can do all these things, then we will master the role of data within project delivery. We will move into a world of prescriptive analytics, where the data informs our decisions, learning at scale from the countless projects driving that knowledge. With uncertainty contained, the ensemble of projects will become more efficient and predictable, driven by data, in which every loss that is systematically modelled and recorded is a gain for the future.

I welcome and look forward to this future. I encourage all project delivery professionals to explore what this new world looks like, and to play your part in the 21st century, exploiting data to deliver your projects.

You may also be interested in: