Agile Development’s Secret Sauce – It’s all in the recipe

One criticism levelled at Agile methodologies for software development is that of scale. “It’s all very well for very small projects,” we have been told, “but for anything above 10 people, you can forget it.” So, how true is this?

We took this question to the Reg reader panel, and the results make for interesting reading indeed. It’s worth stating our standard “self-selecting” caveat up front, but this time we’ll add a caveat to the caveat. Sure, the majority of respondents are likely to have an interest in Agile, as illustrated by the fact that 90 per cent claim some kind of direct experience. But equally, it’s important to think about what we can learn from such seasoned types.

This applies particularly to the question of whether Agile can scale at all. As can be seen from the chart the results were pretty much across the board – which suggests it really does come down to experience, good or bad.

There will be those who have hands-on, practical proof that Agile couldn’t scale its way out of a paper bag; yet a quarter of respondents were OK with the premise that it would be well suited to development projects of 100 people or more. Both answers are perfectly valid, as they lead us to ask – what is it about the different groups that makes success more or less likely?

From the overall picture, the number one criterion is communications. This point is as vital to grasp as it is, on the surface, self-evident. Put it this way: a long time ago, one of the big names in software development (I think it was Ed Yourdon, but I’m not sure) suggested how he could look at how a software project was starting up and have a good impression of whether it was going to succeed or fail. Good communications is one of those fundamental factors which, to old hands like Mr Yourdon, are as good an indicator as any of good practice. Fail to sort out bad communications, and even with the best will and most honed methods in the world, things are still going to fall flat.

Let me be blunt. If you don’t have a collaborative, communicative organisation in place, you might as well forget Agile. There, I’ve said it. That’s not to say that you can’t improve communications, and use Agile as the catalyst for that – but it’s worth facing up to this key issue up front.

Good communications may be key, but we can see from the chart a number of other criteria which are seen as important; notably the need for a clear management overview, and the importance of continuous integration. What’s interesting here is what happens if we compare the two criteria, for different sizes of project.

While communications remains the number 1 priority across all sizes of project, the need for a clear management overview is seen as more important for smaller projects. As project sizes increase, it is integration which becomes more important. It is straightforward to see how these three characteristics need to work in tandem: communications within and across teams, plus a coherent management overview are prerequisites for continuous integration of the application sub-parts into a single deliverable – which, to be fair, is the litmus test of Agile.

Given this, it’s interesting to drill down into what are seen as the challenges of implementing such continuous integration. Top of the list overall is build performance, which does become more prevalent as the scale of the project goes up – as does the importance of having tools to manage the configuration items that need to be integrated. This is one of those areas which may well be common sense to anyone who has struggled to keep up with a rapidly changing pool of source code – but such common sense is often hard earned!

When considering tooling, we did ask which tools would make more or less sense to helping Agile projects. Unsurprisingly, communication and collaboration tools were top of the list (but back to the point about communications failures, we would add a caveat of our own – there’s no point in having the slickest set of collaboration tools in place, if nobody has the slightest interest in using them). The second most important type of tooling was to do with testing and specifically, with supporting quality and integrity testing prior to integration. While we haven’t shown this on the chart, we do know that such tools only become more important as the size of project increases.

This makes a lot of sense. It’s not that other tools are unimportant; rather, that for Agile to work, there is very little room for error. Building on the continuous integration point above, one can imagine what might happen if too many errors started to appear in the process – like pieces flying off the fast-moving car in Wacky Races, ultimately leaving Dick Dastardly sitting on his singed backside in the desert, holding only the wheel.

It’s less funny, though not so very different in effect, when a poorly configured, buggy software application is spewed out of the end of an ill-conceived process. “Yes, but we stuck to the timescales!” goes the project manager, even as he turns and pats the smirking Mutley on the head.

So, there we have it. Agile can work, and it can scale – according to a fair number of people who have real experience of success. But, like our presidential candidates are so proud of saying, “You can’t put lipstick on a pig”. Agile is not some silver shot that can be fired into the middle of a poorly-run development organisation; it requires real discipline, second to none communications, and a tenacious grasp of what it means to develop, test, integrate and deliver the application elements.

There’s a number of specific pointers that experienced Agile types can offer, but without these basics in place, they will be difficult to apply. As goes the Irish adage: “If you want to get there, don’t start from here.” ®