Could what happened to flight engineers apply to data storage and its management?

By Charles Brett

Flight engineers in cockpits began to disappear in the 1980s as aircraft manufacturers rolled out new plane designs that were more automated and thereby cost efficient. Arguably the same should be happening with data storage in enterprises. Why is this not taking place?

Until the 1980s most large passenger aircraft carried a flight engineer to assist and support pilots: the latter flew the plane and the engineer managed the plane’s systems. This was true of 707s, VC10s, Concorde and early 747s, to name only a few of the best known models. In the late 1980s the flight engineer’s position disappeared as automated flight management systems on these new aircraft provided tools direct to pilots — which delivered most of what a flight engineer had previously done (and anything that fell outside expected parameters required real time communication back to base).

Many current approaches to storage are today similarly people and skill intensive. As the need to store ever more data continues to expand (see below, from the Freeform DynamicsStorage Anywhere and Everywhere report which comes from a survey of 300 medium size enterprises but is applicable to larger and smaller ones), so storage solutions have become more sophisticated and expansive — to the point at which there are multiple different approaches (some dependant on the size of enterprise):

Advanced solutions are often based on technologies from the likes of IBM, EMC, Dell, HP, HDS and others; here the complexities of virtualisation, availability, tiering (whether for performance or for reliability or for archiving or other reasons), along with the people to manage, are deployed
Intermediate solutions, where traditional approaches to storage predominate (if only because the resources or business demands have not required new approaches)
Less traditional solutions, from the likes of HDS, HP, NetApp, IBM’s acquisition of XIV and a number of start-ups where advanced automation of storage makes management simpler and arguably more profound in its impact (for example these often end up with the redeployment of skilled storage people).

The analogy with flight engineers applies if you think about what might change if enterprises were able to run an economic (as in cost effective) lens over their approaches to storage and its management. To imagine what highly automated storage systems could offer, take IBM’s XIV as an example.

The XIV design approach seeks to automate nearly everything that is automatable about storage, to the point that there is relatively little — once running — that storage administrators need do. In effect XIV designed a highly reliable (99.999999%) solution where the storage system software is able to manage better and faster than is possible or practical using traditional approaches to storage.

Modern passenger aircraft dispensed with the need for flight engineers and airlines reduced their flight crew costs. The question for IT is why so many enterprises choose to persist with storage specialists when highly reliable, automated storage solutions now exist and the undoubted IT skills of those storage administration people could be redeployed to support other parts of the business?

Looking at the Freeform Dynamics’ analysis there are at least six broad reasons to explain this, which are not mutually incompatible:

The first is simple inertia; often it is easier to do nothing….
The second is that, like flight engineers before, those already expert in managing storage often have little incentive to change (and this is often aided or abetted by internal management politics and/or fear of admitting that something could be done differently)
The third, and possibly most important, is that moving between storage platforms at scale is complex and involves risk; while the XIV team (for example) argue that moving need not be difficult or risky, most organisations prefer only to change when they must do so (plus they have already lined up the necessary resources for investigation and testing of the ’better’ solutions that they hope will meet their needs)
The fourth is simply financial: write off times and depreciation on existing storage purchases provide an obvious non-technical impediment, especially when IT budgets are constrained
A fifth is the lack (as yet) of additional alternative vendors offering XIV-type highly automated solutions which deliver value through lowering overall operating costs; Freeform Dynamics knows of at least one example where a financial customer that had moved to XIV then partially returned to a combined IBM XIV and Hitachi storage configuration – to ensure that it was not dependent on XIV as its only storage solution
Storage administrators do more than just feed and water storage arrays; they help with enterprise data management over its complex lifecycle.

Much of the over-complexity in storage exists because it has always been complex and, until relatively recently, was often designed around a storage platform per application, usually as a sub-component of a specific physical server. That need no longer apply. Indeed, with mass virtualisation, it no longer applies and enterprises, therefore, need different approaches to storage.

Even with examples already in the market place, of what can be done with automated storage today, storage conservatism (and data ’stickiness’) tend to prevail. Storage management conservatism does have a basis for its existence: data is valuable and most organisations want it to be adequately, sometimes excessively, protected to ensure it is always available when required.

That said, it is the view of Freeform Dynamics that automation of storage is going to happen. When is less clear but, as soon as business execs in enterprises realise how far storage automation and its vastly improved reliability has moved forward, then what occurred with flight engineers will apply to storage-only specialists. Economics combined with pragmatism tend to win in the long term.

This does not mean, however, that enterprises will just eliminate the management of storage. Flight engineers became part of the larger engineering function. So the storage specialty will be subsumed into a managed systems environment where storage automation is the norm. Thereafter the focus on data will be less about storage platforms and more about ensuring that relevant data for the business is available to those who need it, when they need it and at an acceptable cost.

What should enterprises do? Experiment now in order to build confidence that they understand what automated storage will eventually deliver to their businesses. This does not necessarily mean rushing out to buy an XIV frame (currently both top end and expensive); there are other entry points. But it does mean that enterprises should think through their data priorities. Trying to ignore trends in automation that will in time become inevitable tends to be counterproductive.

CLICK HERE TO VIEW ORIGINAL PUBLISHED ON