Agile vs Waterfall — How to Choose the Best Delivery Method

Agile vs Waterfall — How to Choose the Best Delivery Method

By Jibility Co-Founder Chris Benthien

Agile evangelists may argue that pitching Waterfall methodology against Agile is a foregone conclusion — in the latter’s favour, of course. Waterfall is the rusty old man of project execution approaches; Agile is the “new” kid on the block.

This is a debate that rears its head every time organizations reach the stage in the strategic planning process where they need to decide how to execute. We’ve seen the Agile evangelists often win, even when there is a lack of clarity as to what they mean by ‘Agile methodology’. Scrum? Kanban? Lean Startup?

But, roll back – should Waterfall really be retired?

Not necessarily.

You need a tool or technique that enables a consistent approach for assessing Agile versus Waterfall. There is no definitive answer; each project or initiative needs critical consideration to identify the best methodology.

Jibility is our preferred tool as it leverages a built-in 2×2 matrix in which the quadrants are defined by the degrees of certainty and uncertainty associated with each initiative.

Decision Criteria: Degrees of Uncertainty

The originator of this method was Jez Humble, and it’s a variation of the Stacey Matrix by Ralph Stacey. We’ve adapted it and use it as a broad guide when deciding on Agile approaches versus Waterfall.

The core of the method is the determination of the degree of uncertainty for two key aspects of your project or initiative: the clarity or certainty of the problem and the certainty of the solution.

Problem Certainty

The problem that your initiative is intended to solve is often crystal clear. Perhaps your organization is losing customers, and you know for a fact that the key issue is the poor response time to customer queries. That is a clear and succinct problem. How you solve it may be up for grabs, but it’s a well-defined problem statement for your initiative.

But if your objective is more broadly to reduce customer churn, are you certain that the fundamental issue is the slow response time? Perhaps the challenge is being driven by other issues of which you’re unaware. Maybe your customers can’t easily find and access your help material and manuals, necessitating a need to submit queries. This could be a simpler and more effective problem to solve.

Before you rush out and buy a CRM system to manage customer queries, you need to be certain about the problem or challenge you are solving for.

Solution Certainty

In some cases, the solution to your problem will be clear and well-defined; in other cases, not so much.

The solution to a customer churn issue could unequivocally be the installation of a CRM and implementing its capabilities out of the box. There is a high degree of certainty that the system and the proven business processes it encapsulates will solve your problem. But sometimes the solution is not so clear. A new system may well solve the problem, but could it be better solved through some simple people and process tweaks?

How to Choose: Agile vs Waterfall Methodology

By considering these dimensions of problem and solution certainty, you can gain a broad indication of the best approach to take to successfully deliver your initiative.

• When the problem and solution are both unknowns, a Lean Startup approach is often the most effective, a.k.a. an experimentation model.
• When the problem is known but the solution is not, it’s time for Scrum, a.k.a. an iterative solutioning model.
• When the solution exists but the problems to be solved are still unknown, take a Kanban approach, a.k.a. the factory model.

As you can see, Agile approaches are suited to helping organizations deal with uncertainty – but what about when both the problem and solution are known and well-understood? That’s when the linear model of Waterfall can still play a part.

Let’s walk through the four approaches in more detail.

Experimentation Model

The Experimentation Model is where the problem and the solution are less certain. This lends itself to starting with a Lean Startup approach, including Design Thinking and Human Centred Design.

Beginning with a clear exploration of the problem, you ideate a solution hypothesis. Then, you progressively test and refine it for the least cost/effort until you have more certainty about the true underlying issues and the feasibility and viability of the proposed solution. Once this is clear, you may move on to using Scrum, Waterfall, Kanban, or any other approaches suited to the clarity of the problem or solution.

Continuing with the high customer churn example, you could choose to immerse yourself in customer feedback to investigate why they are leaving. From this, you may formulate a hypothesis as to the problem, define a potential solution, then develop a series of solution prototypes and experiments to prove that your problem/solution is certain.

Iterative Solutioning Model

When the problem is reasonably certain, but the solution is less so, an iterative approach such as Scrum is best suited.

In our customer churn scenario, it may be clear that failures to handle customer queries and complaints in a timely fashion is the problem. However, the shape and complete specification of the solution is not clear.

You could progressively establish an implementable solution (technology, people and process changes), testing and seeking feedback after each increment. In this way, you can continually shape your solution as you learn more about its effectiveness in addressing the problem.

Factory Model

The Factory Model is where the solution exists (or is known), but the problems to be solved haven’t yet been discovered. Or in other words, we have an approach to solve a set of unknown problems, within a known space, using a set of known solution patterns.

We call this the Factory Model. You have an existing solution or capability and you just keep feeding it the next piece of work or problem to solve Kanban style. The Factory’s goal is to constantly optimize its throughput to get more done more efficiently and for less cost.

An example of this could be workflow automation, where you have an existing workflow automation technology platform, team and delivery approach (“The Workflow Factory”), and you just keep feeding it business workflows to automate. No need to think about the solution – you only need to identify the business opportunities to feed it and ensure the prioritization of the input aligns to organizational priorities.

Linear Model

Last but not least, we reach the context where Waterfall may remain a better fit than any Agile approach.

In a linear model, the problem and solution are both certain and the Waterfall methodology is valid as long as:

• You’re clear about the solution requirements
• The definition of the solution is stable
• The technology or solution components are mature

In relation to our customer churn example, you may know that query handling is a problem and that the lack of a system and processes is the underlying cause. You have chosen a SaaS solution and an initial proof of concept has shown that it alleviates your problem. You could then implement this solution out of the box (well, out of the cloud in this case), clarifying your requirements, configuring the software and deploying it to your teams in a linear fashion.

The problem is clear; the solution requirements are known; the solution definition is stable (bounded by the SaaS product chosen); and the technology and solution components are mature. The Waterfall methodology fits.

What Happens When You Choose the Wrong Approach?

Things can become very inefficient when you find yourself executing in the wrong quadrant.

The example that always sticks in our minds is an old client who had established a workflow automation team and solution. It was progressively responding to internal customer requests to digitize and automate business workflows to improve efficiency. However, the standard framework for executing any work in the business unit was a traditional PRINCE2^® framework, mandated by the Project Office. This team was therefore executing in the Waterfall methodology quadrant.

When we analyzed the value chain, the actual tasks performed to deliver the customer output (i.e. the automated workflow) were a small portion of the overall process, which was largely consumed by delivering project management artefacts. We helped them move to a Factory Model by developing an optimized and streamlined value chain (Kanban style), which meant they were able to achieve more throughput with happier customers for the same overall cost. Management overheads were reduced, and the value chain was now dominated by activities associated with actually producing customer value.

Agile vs Waterfall in Practice

The 2×2 framework is a broad guide only; determining the certainty of the problem and solution is subjective, and also depends on a number of unrelated factors, such as:

• Existing organizational standards and approaches
• Availability of skills
• Stakeholder understanding of the preferred approach
• Commercial models (e.g. fixed price)

A challenge can also move through the quadrants. Perhaps you start in Experimentation to diagnose the problem and formulate the most viable solution; then move to an Iterative Model to establish the solution; and then finally move to a Linear Model to roll out the solution.

There is no single optimal approach to delivering initiatives or releases on a strategic roadmap. Agile approaches have their place and are extremely effective in many situations, but Waterfall should not be disregarded by default.