Why are banks struggling to implement big data?

head in the sandThis was the topic I was asked to cover at the second annual Big Data and Analytic conference in Johannesburg earlier in July.

Of course, the event targeted Southern African banking and insurance representatives and my focus was on our market.

I do not work for a bank, although we have a number of banking and insurance clients.  My opinions are based therefore on my perceptions based on chatting to various contacts in the banking and insurance sector.

My presentation,What is stopping big data adoption in South African financial services, covered three major themes:

  1.  We still don’t understand (properly) what Big Data is. Every BI vendor, every mega-IT-vendor, every storage vendor has presented whatever it is that they do as a big data solution. No wonder we are confused. The original 3 V’s definition – a COMBINATION of Variety, Velocity and Volume – coined by Doug Laney in 2001 remains a good technical test (Simply put – if you can do it with existing SQL type technology then it is NOT big data). However, I believe it is time to move beyond this technical definition and start define big data by its primary benefit: the ability to deliver insight in business time
  2. Banks are struggling to identify suitable applications: If we look at case studies from big data leaders, like Datameer, their customers have moved beyond IT experimentation and IT driven use cases for big data (such as EDW optimization). Over the last several years, Datameer banking and insurance clients have used Hadoop to solve numerous business problems – from fraud prediction, to churn reduction, to risk data aggregation and many more. These use cases, and many others, are well documented. yet, in South Africa we are just beginning to experiment with Hadoop. The challenge: the clutter and confusion created by point 1 makes it hard to identify really good opportunities for exploiting big data in financial services
  3. Skills: Finally, a lack of skills remain a real challenge. Hadoop requires technical skills that are in short supply in the South African market, and the rapidly evolving Hadoop ecosystem makes it difficult to keep skills up to date. I looked at the evolution of data analytics on Hadoop – from technical, to self service, to visual / iterative. I strongly believe that we need to adopt simpler tools that shield us from both the complexity and the evolution of Hadoop if we are to achieve benefit – hence our partnership with Datameer

What do you think is hindering a more widespread adoption of big data analytics and techniques in South Africa?

Is it related to some of the points that I have made here, or do we simply have our heads in the sand and see no value?



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.