Why are Agile Teams 25% More Productive?
Written by Ned Kremic
What if you could provide such environment that your software development team is 10-20% more productive?
The potential benefits could be enormous, from saving achieved on labour cost, to faster time to market.
But what if you could boost productivity 25% or more!?
Yes, you can!
The independent study done on sample of over 8,000 projects have shown that agile teams are on average 25% more productive than their industry peers.
Before we analyze this phenomenon, let us define what the productivity is:
Higher Productivity = More scope delivered within the same time frame
Consider the following situation: You are the project manager on traditional well-defined, fixed scope, waterfall software project that was estimated to last 6 months. Have you noticed any difference in pace (and productivity) along the project timeline? When was your team most productive: 6 months; 2 weeks; or 2 days from the deadline?
If your experience is the same as mine, here is possible explanation.
There are many theories that could help us answer this phenomenon, and my favorite is the “four quadrants of time management matrix” introduced over 20 years ago by Stephen Covey in his book “The 7 Habits of Highly Effective People”.
We can classify any activity we undertake in one of the four quadrants based on how urgent and important it is.
Quadrant 1 (Q1) - represents activities that are both important and urgent. They cannot be ignored. Q1 is the crisis quadrant. No matter how important work you are doing, or important meeting you are attending, if you get a phone call that the member of your immediate family is having a medical emergency, you will immediately prioritize the family matter first and go to the hospital. At work project whose deadline is few weeks from now is in Q1. If a deadline is few days from now project is in Q1 and crisis mode.
Quadrant 2 (Q2) - represents activities that are important but not urgent. Your important project, whose deadline is 6, 8 or 12 months from now is in Q2
Quadrant 3 (Q3) - represents activities that are urgent but not important. Most phone calls and emails are in Q3. Honestly answer yourself how many times you have interrupted some important work to answer the phone call, or check the new email. That is how many times you have unconsciously prioritized the urgent but non-important phone call (Q3), higher than your important but no so urgent work (Q2)!
Project Timeline
6+ Months to the Project Deadline - Project is in Q2
The project you are working on, with approved funding must be important for your company. Depending on the scope, most medium sized or larger projects will be longer than 6 months. With plenty of time ahead, clearly the urgency is low, therefore the important but not-urgent project is in Q2. However if your project team and you are like most people in the modern world, than 6+ months before the project deadline you have probably spent most of your time in quadrant 3 or 4, in countless long meetings, answering non-important calls or emails, with some portion of your time spent on leisure activities in quadrant 4. Important but not urgent work in quadrant 2, usually at this stage of the project may take less than 50% of effective time.
Clearly the target deadline is still either invisible or very small to trigger any urgency at all.
Also our estimates for such a long range cannot be that accurate. Accuracy of our estimates significantly improves with the shorter time-line. There is a close resemblance between the accuracy of the project estimates to the accuracy hitting the bullseye of the shooting target.
- Accuracy of the 8 month project estimate is like hitting the bullseye target 80 meters away.
- Accuracy of the 4 month project estimate is like hitting the bullseye target 40 meters away.
- Accuracy of the 2 weeks agile iteration estimate is like hitting the bullseye target 2 meters away.
The closer you come to the target, the bullseye becomes bigger and bigger, until finally there is a very low chance that you will miss it!
Few Weeks to the Project Deadline - Project Moves to Q1
The closer we come to deadline, the target becomes visible and all outstanding important work all of the sudden gets urgent. At this point we cannot ignore the urgent project related activities, and the portion of effective daily work dedicated to them gets progressively higher, avoiding non-essential (non-urgent) ones.
At this stage of the project, you will see many people coming early and leaving late their work and having lunch at their desks.
Few Days to the Project Deadline - Project is in Crisis Mode
Finally, few days before the deadline, the project is in crisis mode (Q1: Urgent and important). People are focused only on quadrant 1 activities, which are urgent and important work committed 6 months ago!
Very frequently at this stage, project team is working late nights and weekends, and even urgent quadrant 3 phone calls are ignored in order not to cause interruptions. Ask any medical imaging company about their November schedule before the annual RSNA (Radiology Society North America) trade show in Chicago. All of them will answer, “Not late nights, rather overnights and all the weekends in November!”)
If everything goes well, the project team will deliver the scope on time and project will be declared as success. Congratulations!!!
However, in this scenario to be honest, is our productivity really higher at the end of the project, or due to urgency the amount of actual work hours just got higher?
Delivering the Scope
To simplify the fact, this scenario considers that the scope is fixed, and that planned effort equals the amount of actual hours, something you will rarely find on the real software project.
The following diagram shows the “ideal scenario”, where the team works effective 6 hrs/day to deliver the fixed scope (area under the line).
However due to human psychology explained in “time management matrix”, no team will work effective 6-8 hrs/day during the entire project time-line.
The following diagram shows the “real scenario”, where in the first few months of the project the team works less than effective 6 hrs/day to deliver the fixed scope (area under the line).
Real Waterfall Scope and Effort
As the actual work on important activities in the first few months of the project is less than nominal 6-7 hrs/day, we will have to make up for deficiency later in the project stage, in order to meet the deadline (which is to have actual hours = estimated effort). Those extra hours do not necessarily reflect the higher productivity, even less the process effectiveness. Unfortunately in many cases the scope will not be met within the time-limit, and either it will be cut or the deadline extended.
As we can see in the time-matrix diagram, the important project work is in quadrant 2. It is important but not urgent, therefore very often neglected when the project deadline is far away.
Agile Approach - Short Iterations
How do you eat an elephant? One bite at a time.
That is exactly how agile treats the large scope, as an elephant. If you view the elephant as one giant scope of your project, why not enjoy the bites along the way?
The major advantage of this approach is to have a small scope, that could be easily and accurately estimated, which deadline is clearly visible. As per Covey’s time-matrix, due to such short deadline, it always verges on the brink of important and urgent quadrant 1 that cannot be ignored.
Many agile processes adopted the short time frame iterations.
When the target deadline is always near and visible, it is urgent. It also puts a healthy pressure on the project team to focus on important things. The most optimal iteration length based on industry results is 2 weeks, however the iterations could be anywhere in the range from one to four weeks.
The team is supposed to deliver the scope that is doable in that time-frame. The small but important project work is still in quadrant 2. However any time spent in quadrant 3 or 4, would immediately put those important activities in quadrant 1, and we won’t be able to ignore the immediate crisis.
This may seem too harsh on the project team, but consider the following: The scope of the two-week iteration is relatively small and equal to amount of work that waterfall team is supposed to do in the same time period, based on the traditional effective work effort somewhere around 6 hrs/day. With that work pace there is no over-burn!
The only difference is that due to near deadline, the agile team will treat the important project work as urgent and will work steady pace every day.
The following diagram shows the typical agile scope delivery. Agile team on 2 weeks iterations will also spend some time in Q3 and Q4 in first few days of the project (human nature), however near deadline will very soon correct it. As a result of this phenomenon, the agile scope delivery is much closer aligned with the ideal one, than the project whose deadline is several months down the project time-line, resulting in higher-productivity.
On the other hand, as explained in this article, when the deadline for the same scope is remote as in traditional waterfall projects, the less important but urgent tasks will take priority over non-urgent important project work, therefore the effective work hours will slip well below required 6hrs/day.
Conclusion
Iterative development, with short (~2 weeks) time-boxed iterations, almost always guarantees the higher productivity.
Always near and visible deadline is powerfull self-correcting mechanism, where important and non-urgent project activities (Q2) will immediately become urgent (Q1), if time is lost on non-important activities (Q3, Q4) in first few days of the iteration.
If we superimpose all three diagrams: the ideal one, the waterfall and the agile, it is clearly visible that within first few months, agile productivity would closely match the ideal one and will perform better than the traditional one.
The results from QSM Associates study done on 8,000+ projects confirms that findings and shows that agile productivity is 25% higher than traditional ones.