Just what is a Data Scientist anyway ?

The Scientist maintains the water tank, the Plumber maintains the system itself. And the Analyst takes a drink or two when the business demands to quench it's thirst for knowledge.

Dubbed the “sexiest job” of the century everyone is talking about Data Scientists. But it raises an interesting question: what’s exactly are they all about ?

The Big Data industry engine steamrolls on but some aspects never get any clearer. Let’s be serious, NASA has been playing with ‘lots and lots’ of data since the time of the Moon launches but it was just data back then, and now all of a sudden there's a gold rush on where everyone is data mining for a single nugget. Of course, what comes with a new industry interest is a new job title to come with it; the Data Scientist.

They seek them here, they seek them there

According to a recent survey by New Vantage, 70% of organizations surveyed plan to hire Data Scientists, and 100% of them said it’s “somewhat challenging” to hire a competent one. But just what is a Data Scientist anyway ?

Searching the internet tells you that they are supposed to have a distinctive set of skills, aptitudes and attitudes which distinguish them from their lowly analyst counterparts. The Guardian claims they are "the highly educated experts who operate at the frontier of analytics, where data sets are so large and the data so messy that less-skilled analysts using traditional tools cannot make sense of them."

Looking deeper it's an interesting blog post in which heard a couple of definitions;

     "…a data scientist is 1) a data analyst in California or 2) a                statistician under 35"

It's a matter of balance

But more importantly, it makes a killer point. ”Organisations already have people who know their own data better than mystical Data Scientists…learning Hadoop is easier than learning the company’s business.”

In other words, if you have a Data Analyst employed then your search may well be over. Organizations need to look internally first and invest in their existing analyst resources, train them to stand tall on the same pedestal we seem to have placed the scientists on. As withany business, understanding capabilities that exist on the inside could well be a more cost and time effective method than searching on the outside.

It's a matter of balance. Like big data begs us to ask bigger questions of it and for that to happen organizations need to do two things:

• know what the right questions to ask are and,
• know how to get the right answers

For that you need the right mix of analysts (to ask the right business driven questions) and scientists (to mine for the right data driven answers in context) who operate on the same level. It's not a case of employing one over the other.

And don't be tempted on the quick and easy path with certification. You don't become a Data Scientist overnight through multiple choice.

Maintaining the plumbing

And since we're on the topic of new industry roles let me postulate a new one, tongue in cheek: the Data Plumber.

Because data is alive; it grows, ebbs and flows as it moves and collates information. Data is in fact non-linear despite the pictures painted of information highways, it doesn’t get into a battered Ford Taurus and drives from A to B, it’s very much like a stream meandering from one point to the next, collecting stuff along the way, carving out new channels.

Scientists do the number crunching and make sense of the flood of information, but it’s the handy data plumber that makes sure organizations get the information they need, when they need it, and by how much they need, routing the flow of a constant stream of acquired knowledge and making sure there are no leaks. If the business want to understand or make a decision on something they just turn on the tap and the information flows, instantly. It’s not there when they don’t need it. It’s not constantly dripping and distracting them.

The Scientist maintains the water tank, the Plumber maintains the system itself. And the Analyst takes a drink or two when the business demands to quench it's thirst for knowledge.

What do you think ? If you're a Data Scientist I'd love to hear your thoughts.