Too Sexy to Fail?

“I have not failed. I’ve just found 10,000 ways that won’t work.”

– Thomas A. Edison

I’m often disheartened at how quick some people are to blame Data Scientists for the lack of success or impact of a project, and try and dismiss their efforts and skills as somewhat limited, or at least, overhyped.

There’s so much interest, investment, passion and good intention, in implementing Data Science projects that can often end poorly – so what goes wrong?

In this next instalment of The Restraint of the Data Science Beast series, I will address some of the key reasons why some projects either fail, or fall short of their expectations.

What’s important to understand is that sometimes, Data Scientist’s simply feel like they’re set up to fail. The reasons for this are understandable, and can include:

  • Lack of senior executive support for data-driven decision making
  • Poor leadership/management
  • Being given a solution, and told to find a problem

This can lead to people becoming disillusioned, losing faith in management, and questioning their value to the organisation. On rare occasions, it’s even worse, when Data Scientist’s feel like they’re being used to lend credibility to answers that others want.

More often than not, however, it’s simply ignorance and a lack of understanding that are the real issues. Three of the most common reasons include:

  1. Poor Leadership

    Technical leadership is the linchpin in most organisations for enabling success. It’s a crucial role for helping translate viable business problems into measurable technical solutions – ie focus on the right problems and measure success. Their business savvy, coupled with technical expertise, places them in a unique position to step between both domains and help enable a data-driven decision making culture – by identifying suitable business problems that can be solved via existing or emerging tech solutions. In addition, they help set realistic expectations and timeframes by being the conduit between business and tech, and ensure that projects align with strategic goals. They also focus on achieving stakeholder buy-in, which is critical.

    Further, an empowered decision maker is needed to support and action results.

  2. Lack of a Data Culture

    A data culture is imperative to ensure Data Science success. It helps break down silos and democratise data. It also helps in improving data literacy and the organisations overall data maturity, which helps with managing expectations and creating a culture of innovation and collaboration – including appropriate governance and accountability.

    Data Science is the science of change, and a fear of change, or aversion to risk via exploration and questioning the status quo, greatly inhibits any chance of success. Hence, a ‘fail fast, fail cheap’ mantra is often needed to drive innovation – and must be supported from top-down.

    The experimental nature of Data Science – exploration that can lead to innovation – necessitates flexibility and effectively ‘approval to fail’, and to take away key learnings from such analytical sojourns.

  3. Technological Constraints

    Sometimes, it’s simply the ‘tech’ that is the main bottleneck, such as lack access to data, poor quality data, inappropriate tools and systems and insufficiently qualified/trained/project suited Data Scientist’s, analysts and engineers. In addition, the focus should always remain on the problem, and not technology specifically – which, after all, is there as a means to an end.

Ultimately, success in Data Science comes down to linking viable business problems to strategic priorities, under the guidance of a suitably qualified and experienced leader, who either directly, or via influence, is able to make decisions… and this last point is where things often go wrong in enterprises where Data Science lacks impact. There’s a great deal of focus placed on the notion of ‘data-driven decision making’, with much effort devoted to employing Data Scientist’s, building the relevant tech platforms etc, but they fall short of ensuring that there’s someone in power who can actually make the decisions and action the insights!

In order to achieve this, organisations need to ensure they have the the right people in the right roles, and to put effort into creating the right culture.