Enterprise Flash – The New Normal?

In a nutshell All-Flash Arrays (AFAs) have come a very long way in a short time. First generation AFAs were designed more as point solutions, using fast storage to solve specific proble ms. Now though, a new breed of AFA is claimed ready for enterprise use, supporting business applications with differing performance and service needs. But what characteristics must an AFA possess before you can trust it with a broad range of enterprise workloads, both routine and business-critical? Enterprise SSDs and the All-Flash Array In 2008, the first enterprise-grade solid-state drives (SSDs) were used as fast caching others on specially designed flash modules, but each typically targeted only a single high-value application or a small group of applications with similar requirements. Most first generation AFAs were not well suited to support a broad range of enterprise applications and services, however, as they lacked key storage management features. Some vendors took the alternative approach of retrofitting an existing enterprise-class disk array with a layer of SSDs. Even with new software to cater for the different performance characteristics of disks and SSDs, few of these early retrofitted systems could take full advantage of the benefits flash has to offer. Still, they often enjoyed a more complete range of storage services such as data protection. A few vendors even developed a whole new class of purpose-built disk/flash hybrid arrays, only to see AFAs evolve more rapidly than anticipated and turn their hybrids into niche solutions. To understand why AFAs have been so successful so fast, let’s run through some of the advantages they can provide. Flash advantages – obvious and unexpected Flash is often seen as just a super-fast disk, but it is much more than that. Some are well known and widely marketed, while others are more subtle, yet still important. Low latency: Non-mechanical flash doesn’t just read and write data faster, it delivers it with shorter and more consistent wait times, since there is no need to wait for the desired sector of the disk to spin underneath the read/write head. Quality of Service: With relatively little contention and no mechanical delays, flash can deliver more consistent performance than disk. This is important when it comes to implementing effective quality of service (QoS) management. Data reduction: Having latency to spare allows AFAs to run data reduction processes (for example deduplication and compression) ‘in-line’. This was not feasible on disk arrays as it degraded overall performance too much. When data reduction works well, it can dramatically increase the effective capacity of an AFA, bringing its cost per gigabyte close to that of a hard disk array and making the AFA easier to justify financially. Be aware though that data reduction technologies do not suit all data types – some are already compressed, for example. In addition, turning on data reduction inherently adds a degree of latency back to the storage, which may be a bad idea if absolute performance is key. Even more importantly, it is essential to check that the business and its regulatory environment permit data reduction to be used. Some jurisdictions require certain data to be stored completely unmodified. Less tuning needed: Compared to spinning disk, flash’s performance is much more predictable. With its performance largely determined by software, not mechanical considerations, you can have a much reduced need for manual tuning or for tricks such as altering the width of the data striping on-disk to improve performance. Better storage utilisation: The mechanics of hard disk technology mean that the performance of an array typically declines as it begins to fill up. More files are spread across more disks, each with its mechanical delays, and more users are trying to pull data through each finite ‘pipe’ linking a drive to the array. Flash does not suffer like this as it fills, so AFAs can be pushed much closer to the envelope than disk arrays. This can make AFAs considerably more cost-effective in reality than they look on paper, especially if you plan to use most of the raw capacity. Rebuild time: Most IT professionals who have rebuilt arrays following disk failure will recognise the pain involved. As hard disk capacity increases, so does the time needed to rebuild a RAID system. This is turn greatly increases the business risk, because having another disk fail during the rebuild can result in data loss. Flash based systems do not have such problems and most use fundamentally different data protection mechanisms. Plus they write data much faster, making rebuilds quicker and reducing the associated business risk. Less power, space and noise: Often, storage engineers must provision an array with more disk capacity than needed. Each drive has only a limited input/output capacity, so adding more drives is the only way to get additional performance. AFAs do not have this limitation, plus flash is intrinsically denser storage than hard disk, so solving the same problem requires fewer SSDs or flash modules. This can significantly reduce the need for rackspace, power and cooling. AFAs are also non-mechanical and vibration-free, so should be quieter and more reliable. IT and business transformation: Software has traditionally been written to cater to the foibles of hardware. In particular, programmers avoided going to disk whenever possible, as that was the slowest part of the system. With flash, applications can be built in new ways, for example, to manipulate more data and do it faster. This can in turn allow a business to accelerate its processes, or work in ways that were not possible before. Potential downsides? So far it sounds like AFA systems are full of potential benefits. But does flash have no disadvantages, real or perceived? Cost: Flash today still costs more per raw terabyte than hard disk, but the difference is narrowing rapidly. And when data reduction is added to the mix the price premium can be greatly reduced, or even removed entirely for some workloads. Flash lifetime and wear: Early examples had a limited life expectancy, dictated by the number of writes each flash ‘cell’ could withstand. However, today’s flash is far more robust, thanks to new chip designs and improved management software to control write placement (known as wear levelling). This allows some vendors to offer extended lifetime guarantees on their AFA systems. In addition, the use of data reduction in-line with data ingest can considerably improve a system’s operational lifetime by reducing the absolute number of writes. This is most pronounced in shared enterprise environments, where data reduction can profoundly reduce the volume of data to be stored. Market Noise: The AFA marketplace is awash with hype, and many vendors are working ferociously to raise the profiles of their solutions. The technology has evolved rapidly, yet older niche technologies are still marketed alongside newer enterprise-grade systems that offer a much wider range of capabilities. On top of that, the market is not standing still, or even showing any sign of slowing down. As a result, it can be very difficult to understand the pros and cons of any particular solution or vendor, and what capabilities an AFA system needs in order to support a variety of different workloads and be enterprise-ready. What do we mean by enterprise-ready? An enterprise AFA must work well with a broad range of simultaneously active workloads. The challenge is that each workload is likely to have different performance characteristics and demand different levels of data protection and security, and that these demands will also vary over its natural work cycle and lifetime. It’s important that you check your potential enterprise AFA solution to see that it has the capabilities you need. These requirements will likely include a number of the following, any or all of which might be appropriate to your specific environment: General-purpose shared storage: An enterprise-class AFA should not be specific to or optimised for any single application or group of applications, or need skilled manual intervention for tuning. Architecture and Design: An enterprise-class AFA should be designed as a complete system, ideally with a non-blocking architecture and without internal bottlenecks. This enables consistent response times and consistent QoS. Fast and consistent response times: For enterprise use, a reliable response time is more important than raw performance measures such as IOPS, which are more relevant to uses such as high-performance computing (HPC). Simple or automatic tuning: The system must accommodate different levels of service quality within the platform. It should be easy to administer workload performance characteristics in a straightforward fashion – ideally the system will automatically tune itself to ensure it operates effectively and efficiently. QoS management: If resources are constrained, then in a shared environment you need the ability to prioritise key applications by adjusting quality of service (QoS) characteristics on a per-workload basis. Setting QoS must be simple and straightforward, without the need for complex operational processes and skilled staff. Data protection capabilities: In a shared environment any enterprise class AFA must offer a wide range of durable data protection options to cater for different business services. For example, as well as backup and snapshots it needs to support asynchronous replication and synchronous mirroring. It is also essential that it can protect each workload individually, rather than a ‘one size fits all’ approach. HA & DR failover: Whether you need non-stop operations or simply the ability to recover quickly, uncomplicated high availability (HA) and disaster recovery (DR) capabilities are essential. Preferably, they should not require you to buy much extra hardware or software. Watch out though for failover schemes that require identical storage at each site. With many applications you can make savings if your failover scheme allows you to use a less expensive hybrid array, say, at the DR site. Flexible data reduction: Not all data sets benefit from data reduction. In addition, regulatory, legislative or business requirements may demand that certain data be stored in native form. On the other hand, inline data reduction improves an array’s life expectancy and reduces the effective cost per GB. You may therefore want the ability to toggle data reduction on a per-workload basis. However, many AFAs today do not support this, and with some it is impossible to turn data reduction off. Enterprise class lifespan: In order to have an operational life of at least five or six years, an AFA must come with appropriate warranties on all elements of the system, and with upgrade options compatible with long term usage running multiple workloads. Integrated management: Not only must an AFA be simple to administer in daily operations, it should work with and within your existing storage management framework. Whether it is for administration, storage policies, automation, data protection or QoS, you do not want to create a storage silo. Future Proofing: To support the modern virtualised enterprise data centre, an AFA should support a variety of virtualised storage schemes (e.g. VVols). In addition, for a multi-year operational lifetime, an enterprise AFA must be able to incorporate key new technologies as they are developed. More importantly, updates and any system repairs should require minimal or no service interruption. Lastly, flash comes in many varieties. For example, today’s standard flash is better for reads than writes and may not be suitable for very write-intensive uses (e.g. log volumes). Fortunately, there are other types of non-volatile memory with different characteristics (and more are in development), so an enterprise AFA will most probably need to support multiple varieties of memory type. Support: Nothing works perfectly forever. Flexible, reliable and responsive support is a key feature of any enterprise AFA. The bottom line Early, first generation, All-Flash arrays (AFAs) were designed as point solutions, accelerating a few specific workloads. They lacked key features that are essential in an enterprise-class array. As newer AFA systems appear, this is no longer the case. Infrastructure changes slowly, and hard disk arrays still have niche and secondary roles to play in the grand scheme of enterprise storage, and hybrid (disk/flash) arrays may still have a place at the low end and in entry systems. But both are likely to come under pressure from AFA on one side and deep storage technologies such as tape and cloud-based archival on the other. As we speak, there is considerable variety in the AFA market, ranging from storage systems originally designed as hybrid arrays but upgraded to all-flash, through first-generation AFAs that have had enterprise-grade storage management tools added, to modern systems designed from the ground up as enterprise-class AFAs. As a buyer, you need to consider the feature-sets on offer and match them against your business requirements today and the likely needs you will have tomorrow. Overall, it is clear the age of the enterprise All-Flash Array has arrived. In the majority of use cases, flash’s benefits considerably outweigh its few remaining caveats, with even the price supplement decreasing. All-flash is well on its way to becoming the new storage normal. Today it is possible to acquire enterprise class second generation AFAs that are fast, reliable and attractive from a cost perspective and ready to run multiple business workloads concurrently. AFA is now ready for primary storage deployment supporting non-stop critical workloads.

Bryan Betts

Click here for more posts from this author

Bryan Betts is sadly no longer with us. He worked as an analyst at Freeform Dynamics between July 2016 and February 2024, when he tragically passed away following an unexpected illness. We are proud to continue to host Bryan’s work as a tribute to his great contribution to the IT industry.