Necessity is the mother of invention

Big data trends are sending VARs on a search to develop innovative products and services, hiring the right people in the process. Fleur Doidge asks where to look

Rod Smith - dubbed IBM's Mr Big Data by Infoworld magazine - reportedly told participants in the vendor's ExtremeBlue 2011 internship programme that innovation is a messy process that is more about collaboration and building relationships than the technology. The IBM research fellow and vice president of emerging technology used experimental rock band Frank Zappa and the Mothers of Invention as an example.

Smith said it wasn't until Zappa was shown the crybaby or wah-wah pedal, invented by music tech firm Vox in 1966, by the inventor personally that the gadget became popular. Zappa introduced it to Jimi Hendrix, and before long, it became a rock guitarist best seller. The wah-wah concept had failed to take off when pitched to trumpeters in the 1920s.

So when CRN spoke to IBM's Smith about the big data skills that IT providers need to develop on their team, it was perhaps no surprise to hear him emphasise business skills and communications as key to influencing customers and coming up with the solutions they really need.

A layered effect

Developing a successful solution for each customer without becoming bogged down in eternally changing detail is quite a task, one that requires considerable inventiveness and innovation. Building deep relationships with users and listening to their stories is as essential when making channel music, with the big data trend, as much as in anything else.

Most big data offerings are going to be bespoke for now, not least because every customer's data resources and requirements are different and many parameters appear in near-perpetual motion. Big data is handy shorthand that camouflages a mind warp of complexity, where numbers and information fly every which way, updated in real time, and where each element is layered upon and may affect many of the others.

Clearly, the right people - and the right team - for the large number of tasks potentially involved must be found. However, the term "data scientist" being bandied about in relation to big data skills could be somewhat misleading, Smith suggests - as well as potentially intimidating to smaller providers wondering where exactly they might find a spare PhD for such a role.

"But I think the point people are trying to make is that, across channels or sales, the expertise now is going to be about analytics, and understanding the power of using data and analytics," Smith says.

He adds that it is also a good idea to look at sourcing people with expertise in a particular area, rather than generalists, when it comes to big data. IBM has people working in business intelligence with products such as Cognos, Many Eyes, and Coremetrics across a wide range of areas.

"People are saying they are seeing their product working with Twitter - and that will be it for a few years and then there is something else they want to investigate. And that is what we will see more and more," he says.

"And it's not the business analyst or the IT person who knows what's needed, it's a business person."

Business people and customers will also have to be involved even more closely with projects.

The customers of channel companies, Smith says, may have plenty of information about what and who they are working with, from social media and from enterprise apps, or they may not. They may then need to collate it in compatible formats and analyse it before applying it to the business.

Alternatively, they may need help figuring out exactly what data they want and how to collect it, before they can begin conversion, interpretation and analysis. The analytics themselves may well need to be done in real time to realise the business benefits.

"So if you are talking about a ‘data scientist' competency, it's a person who can see there are a number of interesting data sources that can be used, and can think about data as a material resource. There are lots of places where that material resource is located," Smith (pictured, left) says. "It's someone who will find and seek out new sources of data and would be able to extract it and use predictive analytics with that."

He or she does not actually need to be a scientist, but rather a questing type who can think laterally and analytically and has sufficient expertise in specific apps and sources of data. Other areas of expertise, as required, can and should be contributed by other members of a team which works together on a big data project.

"When I talk to business people, they want to be in the driving seat of directing the queries and the questions," says Smith.

That is especially easy to understand when you remember that time to value for big data projects tends to be measured in days, rather than weeks or months, and that time frame is only likely to shrink as time goes on.

"And I think I would look for a sense of journey. You need someone who can think out of the box," Smith confirms. "And people who have particular expertise in relational databases, Who understand the web, and who have accreditation in Hadoop. Data visualisation specialists. Big data architects and analytics architects, who help people make the right decisions."

The will and the skill

Steve Jenkins, EMEA vice president for Hadoop software vendor MapR, says its new certification through channel partner Onepoint IQ includes training in developing, deploying and administering MapR M3, M5 and M7 installations as well as non-technical, business-related courses.

"There are two sets of resources required that are different - data warehousing offloading, and if you ran it off Oracle or TeraData it could be quite expensive, so people are looking at managing that off a Hadoop structure so you're looking at strategy," he adds.

Jenkins (pictured, right) says maths skills are definitely important. Also, one new role that may be required is "revenue officer", which sounds something like the "data explorer" some commentators have talked about: a person whose primary task is to hunt out new ways of earning revenue from data.

Useful big-data skills could be around the cloud, mobility, number crunching and visualisation. Then there is the usual skill sets of networking and communications, datacentres and compliance.

Jenkins affirms that it is not about a different direction or completely new expertise, but working to build up and knit together a range of skills the company already has.

That needs a creative and inventive, yet analytical and rigorous approach. Staff should have the will and the skill to engage in an ongoing process that adapts to the times and to shifts in customers' outlooks and requirements. The big data project should then continue to earn revenue as time goes on, never short of ideas or out of date - perhaps a bit like Frank Zappa's Mothers of Invention.