It takes tools to tango

Data warehouse solutions are not the prerogative of consultancies,says Annie Gurton. Dealers equipped with the right tools can also workdown the information mine

Selling a data warehouse, data mining or data analysis solution is always easier with the help of a real example of how it has given a business competitive advantage. Unfortunately, most businesses are reluctant to reveal how they have used their data to improve their marketing.

That, they say, would be giving away any competitive advantage they have gained. You can see their point.

Having a data warehouse and making the information it generates relevant are two separate things, say consultants involved in selling data warehouse and data mining solutions.

Bill Inmon of Prism, who likes to be known as 'the father of data warehousing', is adamant that the key to getting information that will have strategic influence is successful integration of multiple data sources.

'Integration is easily the most important aspect,' he says. 'It means consistency and covers things such as naming conventions, measurement of variables, encoding structures, the physical attributes of data and so forth.'

It is only when the fundamentals are in place that information from lots of sources, including external ones, can be integrated and proper analysis can be done. Only then can the information begin to be useful for marketing applications.

There are plenty of tools that will 'scrub' miscellaneous data being imported from multiple external sources to ensure that it matches and conforms. The process of defining the rules is something that a dealer can contribute to and manage.

IBM marketing sales solutions manager Karen Parker agrees that building a well-structured and consistent data warehouse is only the first stage towards extracting data that can be used to develop marketing strategies.

'Having a data warehouse is key, of course,' she says, 'but there are two other imperatives. One is linking and integrating data from multiple external sources.' She says a single database of information can only be cut and sliced so many ways, even with the best intelligent data mining tools. 'If you add data from another source, a whole new slant can be thrown on the main database,' Parker explains.

The problem with this is that while some external sources are obvious, the most fertile are the minds of salespeople, consultants or other employees.

Parker says: 'Many companies allow their staff to keep all kinds of information in their minds which could have a great influence and effect on the conclusions that are extracted from the data. When those staff leave, the information is lost.'

So, according to Parker, one of the first steps towards using data content for marketing purposes is to make sure that as much detail is written down as possible, and integrated into the information warehouse.

'The other key to using data content so that it creates competitive advantage is to have a mechanism in place for acting on the information,' she says.

'It's no good discovering some crucial fact if there is no way for that news to filter through to the marketing department, or the sanction process means that it takes so long for changes to be implemented that any potential competitive advantage is lost.'

Resellers, Vars and dealers have a crucial role to play in ensuring that corporates and small and medium businesses get the best competitive advantage from their data.

'IBM uses a combination of its own consultants and business partners to make sure that customers use the technology to the best strategic effect,' says Parker. 'It is no good having the IT in place if it has no effect on the business. There has to be a mechanism for action and a way of integrating all possible data sources, and that's where the dealer or business partner comes in.'

The role of the dealer in helping customers get the best out of their store of data is often underplayed by the big database vendors, which are keen for their own consultants to provide the service. But any dealer that sells the hardware or software for managing data can also sell consultancy services which help businesses get the most out of their data store.

There is a plethora of tools on the market which any competent dealer can quickly master to provide a data analysis service. IBM provides such a service to demonstrate how seemingly bland data can be used. 'We will take a company's data and spend a week cleaning it and analysing it, and then provide a report which will demonstrate the potential of our data warehousing and mining tools,' says Parker.

It is often preferable that the dealer retains some control over the data analysis process. There is a risk that users are already aware of the potential of data mining and keen to analyse their corporate data.

If they are convinced there is some earth-shattering insight waiting to be discovered, they are frequently disappointed when the results are mundane.

Phil Robinson of Sybase also warns that if too many users are let loose on a data warehouse it can fall over. 'You have to start off by having them ask prewritten, canned questions rather than having them roam around the data warehouse asking ad hoc enquiries,' he says.

Rick van der Lans, an associate consultant at database analyst Codd and Date, points out that it is advisable to let users have an offline copy of an entire data warehouse and leave them alone until they find something interesting.

'There are plenty of tools available for access to data warehouses, and they are easy to use, but in the wrong hands they can mean a hidden danger.

They can produce incomplete or even incorrect answers to queries,' he says.

Robinson points out that the tool is only as good as the underlying structure, and that the dealer can provide an invaluable role in ensuring that all the tables and data structures are sound in order for ad hoc analysis to be carried out by users.

There is a role for marketing consultants in interpreting the data. Steve Walker, a marketing manager at IBM, says that understanding basic marketing principles such as the Boston Matrix will help make the information gleaned from the data more relevant.

'Switched-on marketers and dealers selling to their customers will be looking at marketing information generated by data warehouses and data mining technology and helping their customers understand how to reach specific consumers,' he says. 'Intelligence information about marketing facts and statistics allows companies and products to get themselves properly positioned, and they can compete strongly with competitors without wasting effort.'

Parker says it is important not to leave the development of data mining and analysis products to programmers and marketers. 'Finding the right patterns within the data is a mathematical process and you need a mathematician to get the algorithms right,' she says. 'Once a trend has been detected it often does not take much marketing knowledge to turn it into a strategy, but the trick is getting the system properly tuned so that unexpected patterns can be pinpointed.'

One user says that having a consultant able to 'read' the data and set up appropriate analysis of data is the most important step in setting up a data warehouse. 'A warehouse is no good without the analysis to go with it, and we did not have the internal resources to make sure that it was happening properly. It was a service we were happy to outsource.'

The same user - who preferred not to be named for fear of its competitors following the same route - says: 'We are now able to move more quickly in our niche and be more able to respond to changes in demand from our customers. We can react extremely rapidly, and, frankly, our competitors are still wondering how we are managing to anticipate the market so effectively.

To them, it looks like magic.'

'Companies that are not implementing a data warehouse solution are already disadvantaged but they might not realise why,' says Parker. 'Those that have a good data mining system are already running rings around them.'

With few businesses having the internal staff numbers or in-house skills to take responsibility for setting up a proper data warehouse and then managing the ongoing analysis of the data, the opportunities for dealers to spread their wings are obvious.

Furthermore, because the results of good data mining can have a dramatic and quantifiable effect on a customer's bottom line, it is relatively easy to justify the consultancy charges that can be billed.

Data warehousing and effective data mining is not just a way for a company to gain competitive advantage - it is a business necessity. For dealers, it can be a crucial stepping stone to move from product supply to consultancy contract.