Capacity planning in an age of agile and on-demand IT

Have we all been caught asleep at the capacity planning wheel? Business users today want, and expect new IT services to be delivered in the blink of an eye, the necessary resources provisioned instantly, and changes made “on demand”. But such IT flexibility requires that physical resources, server, storage and networking are ready to be allocated when required. The need for capacity planning has never been greater, yet a recent survey tells us that few organisations have the capabilities they need.

Furthermore, ‘overprovision and forget’ remains a common approach that elevates IT procurement and operational costs at a time when money is tight.

Business services at risk

Every organisation relies on instant availability to a wide range of IT services, from relatively predictable essential everyday functionality provided by key business applications to customer facing systems whose usage may be highly variable. In some environments, such as development and test systems, they also have to operate on a more ad hoc basis with unpredictable resource requirements. For some IT solutions, such as DR, the hope is that the resources required will never be used, but the potential impact of them kicking in needs to be accounted for.

The results of a recent online survey, during which feedback was gathered from over 100 IT professionals, suggest that only a minority of organisations have effective planning capabilities in place to ensure that the necessary capacity is available as business demands fluctuate (Figure 1).

Click on chart to enlarge Looking beneath such business applications and functions to the physical and virtual components and systems that underpin them reveals a similar picture. In the majority of organisations elementary resource monitoring capacity planning capabilities in relation to servers, storage and networking are shown to be less than ideal or worse. And relatively new platforms such as converged systems and cloud-based solutions and services appear to suffer the same planning frailties (Figure 2).

Click on chart to enlarge

Even something every executive publicly claims to be in their thoughts “twenty four – seven”, namely security, shows a majority of respondents declaring they are without effective capacity planning capabilities. This is worrying, to say the least. So what’s behind some of these gaps and shortfalls? How, exactly, are capacity planning matters currently being dealt with among our research respondents?

Best Practice or Instinct?

IT resources have never been available without limits, but today there is immense pressure to ensure that infrastructure costs are optimised while the business expects any request for new IT services to be delivered as rapidly as possible, often immediately. So what approaches to resource management are being taken to ensure these contradictory requirements can be met? The answer is that we see a number of different approaches, that are often combined (Figure 3).

Click on chart to enlarge

Looking at this, it’s no surprise that the long established method of capacity management born thirty years ago, namely the significant over provisioning of resources on day 1 of a new project so there is little chance of running out, is still often relied on in many organisations. We then see a number of ‘informal’ approaches whereby resources may only be looked at in the event of some form of alarm being raised. Some then simply rely on IT staff to keep an eye on things.

Turning to technology solutions in this space, it is notable that only small numbers of organisations utilise modern monitoring tools and intelligent software to ensure resources are optimised or to carry out proactive resource monitoring. But with so much dependence on alarms and alerts, how is service availability ensured? Again, we see a range of approaches (Figure 4).

Click on chart to enlarge

The important role played by IT professionals in many organisations is clear when we look at such data, meaning that ‘keeping the lights on’, never mind responding to new and changing demands, often relies on the experience of sysadmins, their instincts and, crucially, their vigilance. And when tooling is in place to help them, it’s often a mishmash of ad hoc and embedded capability that almost certainly isn’t that well integrated even with the help of spreadsheets. This approach can work, but doing so means it is almost impossible to optimise IT costs, never mind being able to prevent service interruptions due to insufficient IT resources being available.

Challenges and Impacts

Standing back, the survey findings we have looked at so far suggest that important aspects of IT resource planning and management have not evolved significantly in many organisations over the course of two decades, despite IT undergoing profound architectural change. Given that capacity planning is, at least to some degree, essential for the smooth and effective running of every IT system and business service, it’s worth taking a closer look at some of the challenges (Figure 5).

Click on chart to enlarge

Highlighted here are a number of technical problems that are seen to be major issues or significant challenges. Many of these revolve around the complex nature of modern IT systems and the often fragmented nature of the systems that must be monitored. This is an area where specialist software tools should be well positioned to help.

But the results also illustrate that many of the challenges faced by organisations are people oriented. Lack of time and problems keeping things up to date can often be an issue, but a majority of respondents also state they do not have access to the skills and experience necessary to do a good job. Again, there is potential to improve matters through the use of specialist capacity monitoring and modelling tools, as most clearly aren’t exploiting such technology at the moment.

These tooling shortfalls are easily explained from a quick glance at the top two items on the above chart that we haven’t yet mentioned. A huge majority have problems securing funding to implement and operate specialist capacity monitoring and planning solutions, and this is undoubtedly related to the profound lack of executive appreciation of the issues that many also report. Given this, there is clearly a need for IT professionals to find ways to raise awareness of the importance of capacity planning and modelling, preferably before something goes catastrophically wrong.

As part of the executive education process, it makes sense to have an open discussion about the consequences of inadequate capacity planning. (Figure 6).

Click on chart to enlarge

As we can see, inadequate capacity monitoring and modelling can have very visible repercussions. Some of the consequences highlighted are very tangible in nature, with almost half of those surveyed saying they have experienced costly emergency procurements. Others arguably fall into the ‘irritation’ category, such as reports of hard-pressed IT teams who regularly ‘get it in the neck’ when problems occur.

Overarching all of this, it focuses the mind to see significant numbers saying their organisations have already experienced commercial loss or brand damage as a direct consequence of poor capacity planning, or that it’s only a matter of time before they do. This makes it hard to understand why so few IT professionals are ever praised for doing a good job managing IT resources; yet another example where IT is only visible when things go wrong.

The bottom line

With business services dependent on complex, multi-platform environments, even a single element running out of capacity can lead to slowdowns and failures. The resulting commercial impact may be felt long after service is restored.

Beyond such service level considerations, the frequent reliance, even today, on overprovisioning to avoid capacity problems has a direct impact on IT costs. Perhaps the persistence of this traditional ‘blunt instrument’ approach reflects the view that IT resources nowadays are too cheap to worry about. Use of cloud resources that are notoriously easy to overprovision can reinforce this mind-set, as the cumulative cost of innocuous-looking on-demand or subscription-based arrangements is not always immediately obvious.

An executive-level focus on cost management and the alignment of spending with specific business needs accentuates the issues and imperatives. Unnecessary waste will increasingly be frowned upon, but steps to avoid it cannot put business services at risk. Against this background, a well-managed approach to capacity planning is critical, and an ability to implement ‘just in time’ resource allocation within this will be extremely useful. From a practical perspective, relying on Excel spreadsheets and judgement calls made by over-stretched IT staff in their spare time is probably not the best answer.

Whatever your intended future mix of cloud versus on-premise IT, you are likely to need a more robust set of capacity planning processes and tools.

Tony Lock

Click here for more posts from this author

Tony is an IT operations guru. As an ex-IT manager with an insatiable thirst for knowledge, his extensive vendor briefing agenda makes him one of the most well informed analysts in the industry, particularly on the diversity of solutions and approaches available to tackle key operational requirements. If you are a vendor talking about a new offering, be very careful about describing it to Tony as ‘unique’, because if it isn’t, he’ll probably know.