Data scientists must see the story behind the data.


einstein-curiousSome people look at data and see integers, booleans and strings. When I look at data, I always wonder what the story is behind it.” – Stefan Groschupf, data scientist and CEO of Datameer.

Like Stefan, I see the stories behind the data. This is what makes data interesting, what makes big data relevant and what has created my passion for ensuring quality data. [Tweet this]

And yet, this reality has been largely ignored by early big data vendors.

The early premise: That a skilled data scientist could deliver new insight simply by applying complex algorithms to vast data sources has been largely debunked. This is not science!

Science requires a hypothesis – a question to be asked if you will – and that that hypothesis is proven (or disproven) on the basis on examining the empirical evidence. For most businesses the value of big data comes when the business management and analysts are able to ask the right questions. Complex algorithms can then be applied, to vast amounts of data if necessary, in order to test these hypotheses.

The problem with big data analytics is that it has traditionally required companies to invest in scarce technical skills.

A lack of these skills and the technical complexity of early big data technologies – such as R, Python and Hadoop – has left many organisation struggling to reap the benefits that big data promises. In Data Science Lies, Sex and Videotape and How sexy are your business analysts? I suggested that big data analytics will fail to excite mainstream support until your existing analysts and managers are empowered with self service big data management tools.  Tools like Datameer, that manage the end to end complexity of taking on and integrating big data sources – from Google Adwords, to web logs, to the CRM application and everything in between -, that provide intuitive and familiar interfaces for complex analytics and visualisations, and that allow the user to focus on the story rather than on the technology.

Big data suffers from all of the data management complexities of traditional data – arguably more.  The real time nature of many big data applications – from fraud detection, to real time price adjustment, to providing relevant tokens in store – means that business users need to be able to adapt on the fly. The overwhelming trend in BI over the last five years has been towards self service – no less so with successful big data. [Tweet this]

When you empower the people in your business that are passionate about data with tools that allow them to integrate complex data sources, perform complex analytics and almost instantly see the results you create data science. [Tweet this]

I am one of those empowered people.

I am a data scientist. You can be too! [Tweet this]

For more information about how self service big data can empower you watch the video or contact us.

Image sourced from http://en.wikipedia.org/wiki/Albert_Einstein
Advertisements

One thought on “Data scientists must see the story behind the data.

  1. Pingback: Five more big data myths busted | ytd2525

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s