Has the Rise of the Agile SDLC Completely Replaced Waterfall Development?
Has traditional IT code development seen its last day with the establishment of Agile as a better way of doing business? It may very well be the case as the metrics on projects are proving on the performance side. And project managers are paying attention.
The Term Waterfall Often Relates More to a Straw Man Argument
For years, early software development has been about following a sequence. One step is completed and then the team moves onto the next, and the next, and the next. Each phase was a milestone, as well as a measurable metric that could be used to show success. No surprise, the sequence accomplishment oftentimes seemed to be more important than the creation of the software itself, a common criticism. This created a straw man myth about the Waterfall method, a fear that wasn’t real but took on a mind of its own. No surprise, Agile proponents jump on Waterfall’s blunders today wholeheartedly, saying it’s the worst development method ever, at least since AMPM gas station burgers were created (really, they’re not real food at all).
Generally Attributed to a Paper by Winston W Royce In Which He Purposefully Showed a Flawed Methodology
The above sequential methodology for development was laid out by Winston W. Royce when he laid out the principles of the Waterfall Method. However, he also pointed out that risks need to be managed and mitigated with five steps providing critical help in the matter. These were:
- Program design comes first.
- Document the design.
- Do everything twice to make sure it’s right.
- Plan, control and monitor testing.
- Involve the customer.
Somewhere along the way, one or another of these risk controls keeps getting lost which often then leads to failure.
Waterfall Often is a Term Used for Any Linear Software Development Methodology
The waterfall method is a practice of reaching a step, and falling into the next one. Software coders found great structure in it by first creating an idea and then moving to a conceptual framework for an application. Then the code would be written, and it would be tested. A professional approach would apply some quality testing to make sure thing the application worked under pressure, and then it would be released for full use. The approach provided a predictable, formulaic path that became quite easy to attach buzzwords to. Over time, this path became better known as Conception, Initiation, Analysis, Design, Construction, Testing, Production and Maintenance.
A key limitation of waterfall approaches, however, was their sequential nature. A team did not engage in a later phase until the current phase was completed. And in a professional setting, that completion needed to be confirmed or verified. As can be expected, the waterfall method had the strong potential to become a very rigid foundation for bureaucracy, something creative software developers really can’t stand. In addition, stakeholders rarely saw the software until it was completed, resulting in a solution that may not have met the expected outcome. Changes along the way were cumbersome to deal with and created conflict between development teams and those who were expecting the perfect solution. Thus, the ground was already set for a different approach to come into play long before Agile started becoming a well-adopted replacement methodology.
The Primary Difference Between Waterfall and Agile is that Agile is Iterative
Agile offers an extremely flexible and less rigid approach to development, which is often the reason why it is favored so much more than the waterfall method. Collaboration is the name of the game, and cross-functional synergies as well as groupthink development is commonplace. The silo’d culture that was so common in the waterfall approach is broken and replaced by rapid and flexible creativity, often releasing steps early and then refining them again and again for honing. One the best analogies of the Agile method is how swordsmithing is handled; a sword is heated, hammered, cooled, and heated again. The repeat process hardens the steel, making it extremely strong with each cycle or iteration. Done right, the end product is a sword that cuts through other metal like a knife through butter. Obviously, software development is not swordsmithing, but the idea of accepting mistakes in early achievement and redoing the development again and again oftentimes produces a far better product than on release from a sequential production like Waterfall.
Iterative Design Accepts that a Project and its Specs May Change During Production From the Initial Outline
Agile has its drawbacks; the approach doesn’t typically put in near the amount of final testing and quality checks needed to ensure that an app is not vulnerable with each small release. Testing is ongoing and iterative just as new features are developed and the applications will improve over time. As expected, many apps and tools are finding they are open and easy to hack because the bugs in the code simply haven’t been discovered or addressed in the quicker release to market. This kind of thinking is driven by a "results matter" perspective as problems only occur a small part of the time, and most production will be successful - a bit of a reverse on the 80/20 Paretto Rule. In some cases, development isn’t even concerned about the matter, shoving off data protection to a later iteration instead of the initial one. That may work fine if the data is simply game records or texting about one’s favorite singer, but it’s not such a simple thing if the data ends up being a financial transaction or private information in one’s phone or tablet.
Iterative Development Has Multiple Feedback Points from Stakeholders Throughout the Life of the Project
Clearly, Agile offers those working away on a project a success point far faster than Waterfall, which is likely why it is far more attractive a development approach. Instead of waiting long months and years to get to one release and, after intensive testing and quality checks, to find out it doesn’t work well (sound familiar like old Microsoft releases?) Agile gets to the point quickly. And if the project doesn’t work right, it goes quickly into the next cycle for more repair and another quick jaunt to a second release and so on. This approach has other side benefits too.
On the other hand, Agile has a tendency on big projects to keep adding sprints. It is challenging to meet an agreed-upon schedule. Since development happens in iterations, resolving immediate problems with one cycle to then find more with a next cycle, the project can easily get caught in a “broken record” syndrome where the work just seems to continue and never really gets to a completion point. This can be addresses by involving QA /Testing in each iteration and firmly deciding what criteria is used for acceptance. It should be mentioned that there has been more than one conference room verbal brawl over when exactly a project has been completed versus a contractor trying to spin out one more cycle. Agile has proven itself to be a consultant’s best tool and an IT executive’s biggest challenge trying to figure out what “done” really means.
Each Iteration is a Chance to Test and Alter the Existing Product, Simultaneously!
For the fast-moving mobile (and web) application side of technology, Agile works extremely well. Apps are fickle tools, going in and out of popularity very quickly. As a result, a Waterfall approach is simply too slow for a digital economy and marketplace that expects to see changes, updates and new options literally every three to six months at the most. Because the coding and testing schedule are compacted together under an Agile approach, development puts out a product far faster and has far less wait to see how the product plays in the real world. That’s important because as soon as one loses their audience in the mobile market, they fall to the bottom of the barrel for a long climb back up again.
In the Modern Day and Age Iterative Development Seems to Fit Many Clients' Needs Better
Agile also makes the accountants and budget folks far happier. Instead of dealing with long projects and open-ended cost drains as in Waterfall, Agile is tactile – it can be seen from beginning to end in a reasonable time period and costed out with far more accuracy as a result.
Agile can also entertain the gambler-types who want fast returns and results. Unlike Waterfall, which under traditional IT theory, one should have a very clear plan and scope laid out to get from point A to point B, Agile is a crazy train ride of experimentation and coalescing of ideas into a tangible concept and then a product and then a release to see if it sticks on the original target. No surprise, Agile allows very quick software releases which makes clients far happier with a real product on the table, albeit one that continues to need work many times, but it’s there and tangible.
The World’s Pace of Change is Not Slowing Down, So Why Would Our Development Life Cycles?
Agile is simply better suited to the dynamics of what is expected by the market in product delivery. One would expect that Waterfall and traditional IT development approaches still provide better long-term management and delivery on high profile large enterprise projects and with a lower project failure rate. But the truth is, Agile is still better even on large projects at least in term of those reaching completion. The devil is in the numbers.
In early industry reports from 2012 to 2014 Agile was touted as a breakthrough approach that worked great and had notably better finished project results.
Speed up another three years to 2015 and the pattern began to emerge confirming Agile was on small, medium and large IT projects the better way to go in terms of results.
However, overall the industry is still put to shame in terms of those projects that go from start to finish and stay on track. So, when we say that Agile performs better than Waterfall, that’s in reference to being successful in 39 percent of projects versus Waterfall with only 11 percent of projects being completed on time. So, taken another way, Agile produces an on-time win 4 out of 10 projects while Waterfall only works well in 1 out of 10 projects. In either case, the success rate is challenging and the industry will continue to try and make improvements. With Agile producing four times the success rate than, it makes sense to shift over.
A Sign of Things to Come
Notably, Agile works far better on smaller scale projects than big, large enterprise initiatives. That has created a push for breaking projects down into digestible bites for higher success performance and completion versus big megalith challenges. So, in some respects, a shift in how projects are planned is occurring due to Agile, which in turn is creating lots of small projects which work better under Agile, which in turn promotes more use of Agile. It's a self-fulfilling prophecy in action. Waterfall doesn't have a chance of staying under these conditions.
