When Agile was new and its case was pleaded more frequently, the following graphic was quite common:
NOTE: The above graphic is intended to represent to relative shape of the cost curves, and not convey exact scales for your organization. Your own organization's cost curve mileage may vary.The Cost of Traditional Development
The essential proposition is that the cost of a unit of software development effort gets exponentially higher as a project progresses. The same curve represents the cost of change of existing features equally as well as the cost of adding new features.
The underlying principle is pretty straight-forward: the more stuff you have, the more stuff that will go wrong that you can't predict and that you don't detect. The more stuff going wrong, the more rework in-progress. Rework is the crumbly raw material that makes all foundations shaky - be they the foundations of software, or the foundations of the plan.
The number of modules in a software project increases as a project progresses. The number of inter-relationships between modules grows as the number of modules grows. The number of unpredictable repercussions of any change to existing modules or addition of new modules, as well as the extent that repercussions dissipate throughout a system of modules, increases as the number of inter-relationships increases.
When there are fewer modules, it's much more straight-forward to add new modules. However, at a critical turning point, this ease rapidly erodes and it becomes exponentially more difficult to get work done without undoing work that was done previously.
The more modules in a system, the more likely that any new additions or changes will cause adjacent modules to no longer work as they once had, and to no longer work correctly. The longer that they stay in this incorrect state, the more that other adjacent modules will be built incorrectly, or built in a way that reduces development productivity.
The more modules that are in a system, the less likely it is that its programmers can assess any detrimental effects brought about by changes. The system becomes an increasingly unsolvable puzzle that must be solved every time a change is made to add new code or to change existing code. Early on in the life of a system, the puzzle is dead-simple to solve. As the number of modules grow, the possibility of solving that puzzle with every new change closes in on improbable. This is the source of the extra effort that accretes around software development as time goes on.
How Agile Flattens the Traditional Costs
The Agile cost curve is fundamentally different. It was such a radical re-imagining of managing the factors that influence the cost curve that it was frequently dismissed out-of-hand, despite affirmations and reports from people who had given it an honest empirical and experiential trial.
Agile development's power rests in the drive to recognize the conditions that cause the cost of effort to increase over time. It uses techniques that keep effort leveled over the length of a software project, rather than allowing them to spiral toward the inconceivable, unpredictable, and unmanageable.
The cost of change is controllable only when you can predict whether any change will have detrimental consequences. You need to be able to apply countermeasures immediately when any of these detriments are created. The longer the wait to detect a design flaw, the faster the cost grows. This ultimately leads to the vicious circle of rework, where problems are found after the work is delivered to the next phase in the development process, while developers continue to build on that work, believing it to be a sound foundation.
In other words, Agile's radical re-shaping of the traditional cost curve comes from the ability for workers to easily and continuously check that their work isn't a cost magnet while that work is still open on their workbenches. They embrace a principled understanding of the kinds of software designs that are directly and easily tested, and a diligence paid to recognizing, avoiding, and remediating designs that aren't.
And of utmost and paramount concern - concern that can make or break an Agile effort: that organizations realize that its the rigidity of organizations and organizational systems and behaviors that obstruct software development from being done in a way that allows software workers and teams to rehabilitate the traditional cost curve. The impact to organizations - especially entitlements to silos and rigidity - can be profound. Organizations adopting Agile have a responsibility to shift from defensible, garrisoned blame cultures to proactive, optimized learning cultures. If the organizational and management system locks the steering wheel into one, largely immovable position, there should be little surprise when projects end up in the productivity ditch.
The Agile cost curve also suggests that there's a bit of a startup tax to pay up front, but that once the initial bump is navigated, productivity stabilizes and becomes much more predictable.
This is in contrast to traditional methods where the inherent absence of complications early-on lulls developers and managers into a false sense of success. They fail to practice productive countermeasures from the very start by failing to recognize the technical and organizational naivete inherent in the work. The success doesn't last. It's outlived by the realities of the exponentially-increasing number of complications that are natural side-effects of the progress of a software development effort.
The Agile Project Startup Tax
The graphic depicting the Agile cost curve overlaid on the traditional cost curve tells this whole story at a glance. This is what makes it so compelling. And yet, while the Agile cost curve is incredibly compelling, the most interesting point of the graphic isn't so much the benefits that Agile methods avail, but the point on the curve where the lines intersect:
Before that point, Agile development has a higher cost. Before that point, traditional methods should have a lower cost. The time spent using by-the-book, colloquial Agile development techniques before the point where traditional techniques become too costly is a likley volunteer mission for exacerbated costs.
The Best (or Least) of Both
A methodology that uses aspects of traditional development up to the point the costs of traditional methods are on the verge of spiraling out of control, and then makes a methodological "pivot" to techniques that can be considered "Agile" can reap the benefits of each approach's strengths.
Such a "blended" approach takes advantage of traditional methods that, while unsustainable, can deliver more bang for the buck up-front. The resulting path has the least amount of cost if the pivot is well-executed.
The resulting methodology is both traditional and Agile based on where work is in the project timeline and in the smaller development cycles. Neither its traditional aspects nor its Agile aspects are left unchanged by the imperatives of blending what is usually seen as incompatible methods into a continuum. But the most effective and salient principles of development productivity remain in-play.
My own practice of that continuum is informed by Lean Development, and more recently by the Lean Startup movement and its understanding of up-front phases that are often not mentioned by predominating methods like Scrum, and traditional methods that came before.
The Least Way
I was bemoaning failures with Agile and Agile community to Jason Cohen a couple of years ago. He offered the "The Least Way" as a name for the approach that I was embracing to deal with a number of the issues with Agile stasis and orthodoxy that I was trying to work around. I'll ultimately add more meat to those bones on LeastWay.org, and talk about how it underlies some tooling that I'm working on, and how it has affected work on projects. Maybe you'll find it as useful as I have if packaged up in a consumable form.
One of the key differences between an approach that doesn't just cherry pick techniques from different methodologies, but executes a full methodological pivot, is the explicit management of that switch. Planning for it, preparing for it, recognizing the right time to make the pivot, and making a clean entrance and exit from it is a bonafide part of the practice.
Missing the pivot is like missing the last exit before entering the Lake Pontchartrain Causeway. By the time you've realized that you've zoomed past your best chance to take the turn off, you've likely been captured by the gravity well that will start you toward the acceleration of costs. The cost curve begins it's rapid incline immediately after the pivot. The longer it takes to realize that you've missed the turnoff, the faster you'll be traveling, and the more you'll have to spend to put on the breaks and get back on-track toward managing the cost-per-effort.
Shaping the whole methodology around the pivot changes some of the conditions that drive staffing and work management at different phases of a whole software effort. It's not just "changing gears", but more like changing vehicles while they are still moving: you'd want to have vehicles designed for this specific purpose. Modifying the existing vehicles is a good place to start.
With the pivot being a full-fledged part of the methodology, with practices in place to recognize its approach and to execute it cleanly, the cost curve should be smoother, representing the best of what traditional methods have to offer - up to the point that they become unsustainable, and the best of what Agile has to offer - but not before it is sensible.
It's easy to see the methodologies as "one vs. the other" issues - "traditional vs. agile", for example. This perspective can even be helpful when more radical approaches are needed to shake up a rigid, entrenched, silo culture. However, plotting methodologies along a continuum helps us to see those parts of the spectrum where one blends into the other, and to learn to recognize and take advantage of the distinct transitional phases between them that we might otherwise miss or ignore.
Our tendency to look at some colors as "primary" and those in-between as "secondary" is largely an arbitrary side-effect of perceptual bias. The spectrum is one, fluid continuum of material that is neither primary nor secondary. When we choose to see transitional states as secondary, we naturally tend to tune them out. The powerful continuum of abilities that flow smoothly from each other is turned into a checkerboard of disconnected, shallow monotones.
Rapid feedback is the lifeblood of software development that doesn't spiral out of control. Agile protects vital feedback mechanisms from becoming calcified after a projects initial phases. Traditional methods provide rapid - but unsustainable - access to product and customer feedback early-on. Leveraging both and planning for and executing the transition as a fully-fledged part of the methodology rather than a secondary interest is a good part of navigating a "least way" of development obstructions (and costs).
Working with software developers and organizations to help realize the potential of software product development through higher productivity, higher quality, and improved customer experience
Learn more about my work and how I can help you at ampgt.com
