Perspectives: Raid: Hip Hip Array

Sean Hallahan gives three cheers to a storage system that emerged from academia in the 1960s to become one of the most popular and enduring to date: Raid

Like many technological developments, the redundant array of inexpensive disks (Raid) emerged from the academic world in the US.

The term Raid was first used by researchers at the University of Berkeley to describe the linking together of an array of disks over which files could be spread. Disk arrays themselves are not new ? IBM was shipping them in the 1960s.

IBM?s arrays, however, were expensive boxes designed for mainframes. What the researchers at Berkeley proposed in 1987 was to harness the power of smaller workstation disks, which were produced in large numbers and therefore offered a low-cost solution, and link them together. But a large number of small linked disks are inherently more unreliable than a single large disk, so an additional or redundant disk was proposed to ensure that no single disk failure would make the data inaccessible to users.

The cost of storage may be continually falling, but the proportion of IT budgets spent on storage is increasing. Storage management specialist Sterling Software estimated in 1995 that the growth in demand for storage capacity was between 30 and 50 per cent a year.

The company estimated that storage costs accounted for about 50 per cent of IT budgets in 1995 and would grow to 80 per cent by 2000. Although much of this demand for capacity is for mainframes, a considerable amount will be spent on distributed Raid systems.

The storage world has moved on considerably in the past 10 years. In 1987 only 15 per cent of IBM mainframe disk controllers, or direct access storage devices as IBM calls them, had small amounts of cache. Today not only the mainframe disk subsystems have cache but many smaller systems also support the feature.

Raid subsystems have a number of advantages but also carry with them some disadvantages. The most obvious advantages are that file management can be handled by the disk controller rather than by the central processor. Data protection can also be built into the disk sub-system, further freeing up the CPU and providing a degree of fault tolerance. Splitting the files across an array of disks also improves access times.

The main disadvantage of Raid is that there is sometimes a need to write data and parity information separately, which can affect performance. One of the other original drawbacks was that recovery from a disk failure while the system was online could also degrade performance. This has largely been overcome by the use of cache and the increasing sophistication of software in the controller.

Storage companies such as EMC and Storagetek and hardware manufacturers like IBM and Hitachi Data Systems have invested considerable amounts in the development of sophisticated controller software. There were originally six levels of Raid ? nought to five ? but the Raid Advisory Board (RAB) recently redefined the categories.

Each of the levels originally defined the functions offered by the Raid disks. Level 0 involves disk striping, the spreading of files over multiple disks at block level; Level 1 adds disk mirroring for recovery purposes. Level 2 adds error correction codes for all the data stored rather than duplicating all the data.

Level 3 is similar except that it offers a single parity disk which stores only parity information. As in Level 2, the data is spread across different disks meaning that, in the event of a single disk failure, the system continues to function by calculating the information from the failed device. The advantage of Level 3 is that it requires only one additional parity disk for four or five data disks. The disadvantage is that parity calculations are processor intensive.

Level 4 extends the process of Level 3 by interleaving data at sector, rather than bit, level extending the parallel features of the system. The problem with Level 4 Raid is that the parity disk is accessed more frequently than the data disks, and bottlenecks may arise as a result. Level 5 partly alleviates the bottleneck problem by spreading the load across the array. But this has the disadvantage of requiring greater intelligence in the disk controller.

According to a report from consultancy Bloor Research, New Computer Hardware: Options and Comparisons, Levels 0, 1, 3 and 5 are the most frequently used. ?The original levels were simply a guess, albeit an informed one, at what might be required. Levels 2 and 4 turned out to be not to be needed in most cases,? says the report.

?Definition of levels helps in so far as it clarifies the principal characteristics of the configuration. Advances in the sophistication of the controller software, which determines the level, makes the precise definition of levels increasingly complex. At the end of the day, what really matters is the characteristics of the disk system provided by the controller, not an arbitrary argument about definitions.?

One of the problems with the original definitions of the Raid levels was that they were confusing, if not meaningless, to users. Each manufacturer had its own, often different, implementation of the levels. In January, the RAB offered new definitions intended to provide ?simplified but more insightful information to customers for use in evaluating various Raid implementations?.

The three new classifications are failure resistant disk systems (FRDS), failure tolerant disk systems (FTDS) and disaster tolerant disk systems (DTDS).

The new classifications come in two guises: a basic version and a premium version. FRDS is intended to prevent any data loss caused by a single component failure or single disk failure. The premium version, FDRS+, protects against cache or external power failures.

FTDS offers continual availability in the event of a component failure and data protection against environmental failure. FTDS+ provides a further measure of protection.

DTDS must be available in two or more locations to protect against loss of data and is aimed at the large users for whom continuous availability is a requirement. DTDS+ refers to a system where zones can be located at least 1km apart and offer protection against natural disasters such as floods and power cuts.

Even those in the storage market are not convinced the new classification goes to the heart of the matter. Anton Murphy, alliances and product marketing manager at Storage Computer, says: ?We are still sceptical. The RAB is not a standards body ? it is an industry association of which we are a member.?

Storage Computer is seeking to challenge the market leader in the mainframe market, EMC, which is itself now setting its sights on the open systems market. Storage Computer considers itself a software firms rather than a hardware vendor, and claims its offerings are cross-platform products that can be used in a multivendor environment. ?We are the software compay that EMC would like to be,? Murphy says.

Dave McClelland, storage consultant at distributor Azlan, says that in practice only Level 3 and Level 5 Raid are in real demand. ?Most are Raid 5 oriented and there are some applications which use imaging or large files where Raid 3 is more appropriate.?

It is the parallel streaming features of Raid 3 that make it suitable for image processing and the storage of large files. Raid 5 is the preferred choice for networking because of the way in which the disks are independently connected.

McClelland confirms Bloor Research?s findings on the inadequacies of Level 4 Raid. ?Raid 4 never caught on because of the bottlenecks with write updates. Level 4 is very poor at write updates and Level 5 has an improved, though not perfect write update,? he says.

The problem with Raid 5 write updates is that when writing data, the system generates four I/O operations for every write. Data is updated in place and the parity array must be recalculated. This means that the system must read the old data and parity blocks and write new data and parity blocks.

McClelland says Raid systems are not necessary in every installation and that it is the nature of the application that dictates whether a Raid system or a more conventional storage solution should be installed. ?It is absolutely vital to work back from what your application demands. Raid is not for everybody,? he says.

Customers requiring Raid systems are those where the installation offers a competitive ad- vantage or where a customer interface is necessary. For example, a pensions salesman using a laptop to hold details of various schemes will probably not need to back up his system as regularly as other employees of the company.

McClelland believes that selling a Raid system requires more skill than just shifting another box. ?What we have at Azlan is a strategic storage division ? storage is strategic, and if it is not seen as such then it ought to be,? he says.

McClelland and his colleagues offer free consultancy through dealers to companies looking to install Raid systems. ?I will go out to customers? sites as a consultant, but we never sell direct to the user. We are firmly committed to the two-tier structure with our dealers,? he says.

Azlan will also train dealers in Raid. In an industry where to profess ignorance of any technology is viewed as a weakness, many dealers and users exaggerate the extent of their knowledge. ?For those dealers who require it, we offer what I call education by osmosis,? McClelland says. He adds that there are any number of issues requiring careful consideration before a Raid system is installed ? block sizes, scalability and what level of Raid to choose.

McClelland stresses that although Raid systems are highly reliable, they are not infallible. Although the inclusion of a parity disk means the failure of a single drive will not cause a loss of data, the failure of two simultaneously will certainly do so. He admits that the failure of two disks at the same time is unlikely, but it is not impossible.

?They?ve changed the name of the game so to lose data you have to lose two or more disks. You can have a theoretical mean time between failure of one million years, but that does not mean you can predict when a failure will occur,? says McClelland. He knows of at least four cases where more than one disk has failed at the same time.

His advice is simple: ?You still need to back up on to tape.?

One thing that surprises him is that many users are acutely aware of how vulnerable their data is, and by using Raid systems take great care to protect it. But they seem unaware that there are other points of failure, like the OS. ?People will put their data into Raid but not their operating system. They keep that on a system disk.?

That storage is a major area of growth for the IT industry is not in dispute. Independent analysts such as IDC predict that the open systems storage market will be worth $9 billion by 1999 ? an increase of $4 billion on the 1996 figure. The fact that companies like Sun and Compaq are entering the storage market is also an indication of the importance with which the workstation and PC manufacturers view the arena.

Raid systems are becoming de rigueur for larger installations and the likelihood is that during the next two years they will increasingly make their way into the smaller business. But choosing the right Raid system is not an easy decision and users would be well advised to seek the advice of dealers that are qualified in the installation of such sub-systems before making the decision to purchase.