Where are the Founding Fathers of RAID Today?

On the pages of this website, I’ve lamented the fact that although RAID seems to be on the back side of its useful life, nothing else seems to be on the horizon as a suitable replacement. Drive capacities are ever-increasing, as are RAID rebuild times. At some point, RAID will become intolerable as a device protection method.

But when? Probably long before mechanical storage devices disappear – can you imagine having to rebuild a 10TB or 20TB disk drive? Oh, and by the way, most SSD vendors still use RAID as the primary protection method – consuming up to 50% of the cost of these expensive devices.

In my attempt to discover what’s next for RAID, I decided to do some research into the original godfathers of RAID – Patterson, Katz, and Garth – to see what they are up to today, some 23 years after the Berkeley RAID research paper (which changed the face of storage) was published. Here is what I found:

David Patterson and Randy Katz are still at Berkeley as professors of computer science. Although distinguished staff members, neither appears to have had any inclination to continue research on RAID or any other means of storage device protection. It appears that both viewed the original project as just another research idea (funded by the National Science Foundation) that was interesting at the time, but nothing other than another project to tinker with.

Garth Gibson, however, seems to have kept his RAID interests alive. Now on the faculty at Carnegie Mellon University, Gibson is also the founder and CTO of Panasas, a company that specializes in high performance parallel storage arrays.

Looking deeper, I found that Garth and Panasas were indeed taking a different approach to RAID with their Tiered Parity Architecture (TPA):

Panasas Tiered Parity is a comprehensive architecture that enhances system reliability and availability. The three tiers are complimentary to each other and collectively provide the most comprehensive and scalable reliability architecture available for high performance storage today.

Vertical Parity maintains the reliability of the individual disk drive. It addresses the challenge of ever increasing numbers of media errors by isolating and repairing them at the disk level before they are seen by the RAID array.

Horizontal Parity maintains the reliability of the RAID group across multiple drives. It addresses the challenges associated with reconstruction times by using ObjectRAID to more quickly and efficiently complete reconstructions.

Network Parity maintains the integrity of the data path between the storage system and the clients. It addresses the challenge of silent data corruption introduced by the network infrastructure by performing data integrity verification at the client node itself.

Sounds like RAID on steroids, and in some respects it is. From the white paper published this year and available in the storage-brain library, there are some differences between traditional RAID and Panasas RAID:

1) If a drive sector media error is detected, Panasas used “vertical parity” to rebuild data across that sector only, preventing a complete drive rebuild.

2) If a complete drive rebuild is required, Panasas uses an “objectRAID” approach, where only the useful data on the drive is reconstructed, i.e. not the unused space on the drive.

3) When a drive is required to be rebuilt, Panasas uses multiple RAID controllers operating in parallel (“horizontal parity”) to dramatically reduce RAID rebuild times.

Is Panasas on to something here? Time will tell, but it is nice to know that at least one of the founding fathers of RAID is seeking new and better ways to offer protection against drive failures.

In the future I’ll investigate other innovations that are emerging to replace or augment traditional RAID.  Stay tuned!

Larry

A Marvelous Invention – The Disk Drive

In the opening pages of my book “Evolution of the Storage Brain” I make a comparison between the internal combustion engine and the computer disk drive. Both inventions changed the way that we live, both remained surprisingly unchanged throughout their evolution – and both seem to be approaching the end of their usefulness.

In the case of the internal combustion engine, this invention was an explosive fuel mixture combined with a compression chamber and an ignition source to set off a volatile mini-bomb – and viola! You had a machine capable of providing energy in many useful ways – as has been done for the past 100 years or so. The engines have become smaller, more powerful, and more reliable – but the basic premise of fuel, compression, and combustion remained unchanged.

Now let’s examine the disk drive, which contains 4 basic ingredients: aluminum platters, a motor to spin the platters, read/write heads, and an actuator that moves the heads back and forth as they “fly” above the spinning platters.

Those basic components have remained unchanged in the fifty-plus years since their original inception. Miniaturization of components, thin-film media and heads, and microscopic flying heights brought us to the extreme storage capacity of today’s disk drive, along with some creative magnetic recording properties such as “stacked” vertical bits. But the concept of the disk drive remains the same: spinning disks with magnetic heads recording bits of data in concentric circles.

Very few designs were so close to perfection as these two. Yet why am I predicting the demise of both?

Because both contain fatal flaws.

In the case of the engine, the flaw is combustion’s nasty byproducts. Sulfuric Acid, for instance, has a corrosive behavior that eventually rots the engine and the exhaust system, not to mention fouling the air we breathe. Hydrocarbons eventually float up into the Ozone where they do no good for anyone. These little “problems” of internal combustion have paved the way for emission-free, non-combusting substitutes such as Hydrogen or Alkaline-based fuel cells.

When will we see our last combusting engine? Well, that all depends upon the availability of a good economic alternative, right?

The disk drive suffers from an altogether different flaw. Computers and networks move data at the speed of light, but the disk drive, a mechanical device, suffers from Newton’s laws of physics. Spinning disks and moving actuators just aren’t built for speed, at least not in computer terms. In the 4 milliseconds it takes to make one rotation of the world’s fastest disk drives, light travels nearly 750 miles. The time spent waiting for platters to spin and heads to seek is becoming intolerable in our digital age.

No one really wants to use disk drives to store data – the trouble is that nothing better is available, at least not for a while.

However, with the advent of cheap flash memory, it’s a foregone conclusion that solid state devices (SSDs) will completely replace the good old mechanical disk drive some day. When? In my view, not any time soon.

Today, SSD’s have about 50% the storage capacity of disk drives and carry a price premium of about 10X. The inflection point will of course occur as SSD’s approach an equivalent storage density and cost per TB of traditional disk drives. 5 years, 10 years, 20 years – its anybody’s guess. It all depends on the pace of disk drive capacity growth vs the drop in SSD pricing.

It will be fun to watch this scenario unfold…and in the meantime keep an eye on those Hybrid Disk Drives.

PS One piece of trivia that seems lost to history is how the “drive” got its name.  “Disk drive” always seemed like an odd moniker to me.  Who drives it?  Where is is driving?  I seem to recall also hearing disk drives referred to in the early days as a “transport” – also a sort of an odd name.  IBM called disk drives DASD (Direct Access Storage Device) but that name never really caught on.

Can anyone shed light on how the disk “drive” got its name?

Larry

Welcome to Storage-Brain!

Hi, and welcome to the very first Storage-Brain blog. First allow me to introduce myself – my name is Larry Freeman, and I’ve been working in the data storage industry for a loooong time.   A little about my career – after a tour in the Air Force working with Airborne Navigation Systems, I landed my first real job at Data General in 1977.  From there, I worked in a variety of Engineering jobs, spent some time as a Sales Manager; and then finally worked my way into Product Management.  Every company I’ve ever worked for had some data storage focus, and for many companies, that was their only focus.

Throughout my career, it’s been interesting to watch storage technologies evolve.  I am not exactly sure why, but I’ve always found data storage to be fascinating and I could never get enough information on the topic.  When I worked as a System Engineer in the 80s, I was usually the guy at trade shows setting up and tearing down the equipment in the booth.  I was always glad to do this since it meant I could also sneak around and pick up data sheets from all my competitors (of course now the internet makes all that sneaking unnecessary).

After watching storage products go from monstrous storage controllers and disk drives to relatively tiny virtual storage arrays and associated software, I decided that somebody should document how we got where we are today, and since I witnessed this evolution who better than me?  The result was a book based on my observations called “Evolution of the Storage Brain”.  The reasoning behind the title was that as more and more intelligence was added to storage systems, I realized that what we were witnessing was akin to human evolution – storage systems were becoming more independent and much more sophisticated as machine intelligence grew.

But the book was just a start.  In today’s world of social communication, I felt there was an opportunity to make the topics in the book more interactive.  I also knew that there was no central collection point for the thousands of books, papers, and articles that have been written on the topic of data storage technology over the recent past.  That’s the purpose of this website, to bring people together and discuss various thoughts and ideas about storage evolution and to give folks a place to find other opinions and research that’s been done on the topic.

This site is brand-new and far from perfect, but I promise it will get better as it evolves.  Check it out and feel free to send me your comments or better yet start your own discussion topic – and please check back often as we add more content to the reference library!

Thanks,

Larry