Which is Better – Agile or Waterfall

Marianne Hang, Senior Project Manager

March 29, 2018

As a newcomer to project management in the early days, there was no set project management methodology. Projects either worked or they didn’t – we either completed them successfully or we abandoned them. Over the years, we learned and developed a set of best practices we could count on to help ensure project success, and this was likely the start of the “lessons learned” process. That body of knowledge was wrapped up into what we now call the Waterfall methodology and has become the basis for the PMBOK (Project Management Body of Knowledge). As you’d expect for a pioneering methodology, Waterfall is linear in nature (i.e. flows from stage to stage through the project lifecycle). This methodology met with varying degrees of success. One of the biggest drawbacks was, and remains, that few projects are of a linear nature in real life. This is because projects are characteristically non-linear and non-static in nature. For example, technical solutions can change as better, faster, and stronger solutions are identified. Solution requirements can also change during times of organizational upheaval, buy-outs, and management replacement. People behind project deliverables can change, so skill sets and sponsorship of deliverables may come and go during the life time of the project. And when they leave, they rarely come back. We have learned that we need flexibility and calculated adjustments in a project management methodology to effectively manage the change inherent in projects. During my project management career, I’ve seen methodologies come and go, and undergo many revisions. I’ve used everything from Waterfall to Agile to EXtreme methodologies, with varying results. Sadly, I’ve also worked for organizations that rely on the “cowboy” approach where there is little to no methodology in favor of a mad dash to the finish line. I hope to shed some light on the Agile method of project management versus Waterfall, and provide some recommendations to help make a decision between them.

One of the most common questions that I’ve been confronted with as a Project Manager lately is whether or not I’ve managed Agile projects. The answer is an obvious yes. Sometimes it’s not a choice. There are organizations that get on the Agile bandwagon because they’ve read great things about it. They are hazy as to what methodology should be used and the need of one over the other. The rationale regarding which one to choose is complex. When it comes to Agile, I’ve found that it’s worth an in-depth discussion with the folks asking this question to ensure that we’re all on the same page and understand what Agile project management is. It’s one thing to read about and research Agile methodology, but a completely different thing to actually manage a project that way. This is because theory is just that – something to be proved. Applying theory into a real-life setting that is changeable requires experience, maturity, transparent communication, and a solid foundation in project management essentials. So the first step in determining whether to apply Agile vs Waterfall methodology is confirming alignment in understanding of each methodology. Alignment goes beyond the Project Manager and the manager of the Project Managers. To be successful, alignment must also be between the Project Team, clients, stakeholders, governance, and PMO. Nothing is worse than a Project Team and stakeholders all pulling in a different direction based on different thinking about what an Agile methodology can do. Everybody involved should take the time to openly discuss and document what Agile means to their organization and the different type of projects within the organization. We seem to have this crazy notion that one size fits all. If alignment is indeed needed, ensure this alignment process is completed and decisions are made before starting a project. Having a signed decision document in place is a good place to start and will likely see a lot of use in maintaining alignment throughout the initiative.

It may end up that an Agile methodology is discovered not to be the best way to proceed for a given initiative. This decision is extremely important to have in place before starting the project. Even in the early stages of a project, there are significant differences in using an Agile vs Waterfall methodology. Going down the wrong road in these early stages can mean time and money wasted, and a sense of not knowing what’s going on with Project Team and stakeholders. There’s nothing worse in life than not knowing what you’re doing and how you’re going to get to the end. The Project Management Office (PMO) must also be considered in the early stages in terms of its ability to govern and support an Agile vs Waterfall project. All PMOs should be well positioned not only to govern and support a Waterfall methodology but one of an Agile nature as well. Unfortunately, few PMOs are mature enough to do this for Agile methodology because it doesn’t necessarily align to embedded components of a PMO such as stage gates, and it doesn’t necessarily require all of or the same artifacts and deliverables as expected with Waterfall methodology. It’s key to acknowledge that we don’t use the same processes to govern and support both types of methodologies – sure there are similarities and crossovers, and sharing of tools and artifacts – but the methodologies are significantly different and require distinct expertise on the part of the PMO to understand and manage effectively.

Another aspect of a project that must be clear in order to determine whether to use an Agile vs Waterfall approach is the deliverables that are required for the job. Its common knowledge that Agile methodology works well with software development initiatives, and initiatives where the team is small (say under 10 people). Or does it? Depending on the solution to be delivered, can development work be done in short sprints, often only a week or two in length? This process may work well if technical and business folks are dedicated to the Project Team. They both need to be engaged while prioritizing the work effort to manage the short sprint timelines effectively and keeping the momentum going. Business stakeholders, in particular, must be available for ongoing testing that is required as a vital part of working with solution developers, and demonstrating progress as the solution is being built. How many technical and business people do you know who can devote that kind of dedicated time to one project? In my experience, not many because most are highly matrixed on various initiatives and there is often a push/pull relationship between projects vs ongoing operational work.

Another question regarding deliverables is how established are the requirements driving them. If requirements are fully established and signed off, an Agile solution development based on those requirements may lend itself well to a sprint process, especially if requirements are well understood and a line in the sand has been drawn so that developers know exactly what they’re developing to. Without clearly established requirements, the sprint process is in danger of going on forever because it tries to develop to an ever-changing horizon. This is why we need the chickens and the pigs, but we’ll leave that topic for another time when we talk about an Agile framework called Scrum. This leads to frustration for everyone involved and little chance of being successful. As an experienced PMO Lead, I then have to question if we’re managing a project that can show progress and be accountable in terms of its KPIs or if the project has turned into a series of incoherent work orders. The challenge includes how to accurately forecast in this situation. Managing using the Waterfall approach can certainly present the same issues around quality of deliverables, but the Agile approach is especially at risk of succumbing to an endless cycle of sprints that never produce a workable solution because of its focus on that sprint period (often in isolation from the higher purpose of the project, and lack of an experienced governing body to provide support and expect accountability).

There are other considerations when making the decision to go Agile vs Waterfall but these are the important ones. Sure, we can talk about having a mature practice, experienced people, and the discipline to pull off an Agile project, but that comes with proper management and governance. This is why we have PMOs, and it’s important they can support both Agile and Waterfall. I’ll caution at this time that where you may find yourself after these discussions is not so much being able to make a decision in regards to which is better – Agile or Waterfall – but rather having a sense of which methodology may be the best fit for your project. And here is where my years of experience have taught me and brought me – I’m not sure that there is any such thing as a pure Agile or pure Waterfall project. I agree that the theories and various methodologies may be pure but it’s that translation to managing in real life that is the challenge, with all the changeable parts and uncertainty that we discussed earlier in this article, and the need to become much more flexible in order to be successful. As I reflect upon the many, many initiatives I’ve managed, as well as the PMOs I’ve implemented, and all of the teaching and coaching and mentorship I’ve provided – I believe that I’ve come full circle to not so much managing in an Agile vs Waterfall manner, but rather both. I no longer believe that we need to make a choice between the two methods – it no longer needs to be a one versus the other. Instead, a seasoned Project Manager and a mature PMO that understands the realities of project management will see and implement the best of both methodologies, with a close eye to the needs of the stakeholders and organization, the state of the requirements, and the skill sets of the Project Team. In my mind, this is a hybrid approach using a philosophy of fit for purpose, while also ensuring the right level of controls and accountabilities are in place. In this way, we better manage those changeable pieces that are part of every initiative in a way that moves forward flexibly, while still relying upon the best practices embedded in the foundations of project management.