The State of Agile Development Adoption

Executive Summary

To gain a deeper understanding of the key drivers of success, challenges and relationships that various factors play in the successful adoption of Agile, Intelliware designed a survey on the state of Agile adoption in organizations today.We surveyed 117 individuals representing a broad range of industries, job roles, company sizes, and experience with Agile. Given our local presence in Toronto, Canada, the majority of our respondents – approximately 85% – were in Canada. There are several key findings from this survey:

• “The ability to adapt to changing priorities” is the most common benefit of Agile.Close collaboration leads to more adaptive organizations. There is a significant positive correlation between the Agile benefits of “greater collaboration” and “ability to adapt to changing priorities” with an organization’s success with Agile.

• Organizations want to adopt Agile, but often aren’t having a great deal of success. There’s a significant gap between the importance organizations place on Agile and their capability with it.
• Augmenting your existing team by hiring full-time staff with Agile is only part of the equation. Agile experience alone cannot be relied on, as we found that experience drives Agile capability but only to a limited extent.
• Lip-service doesn’t work. “Approach – using Agile elements in a non-Agile framework” was the most frequently selected barrier in our survey; 24% of respondents stated this as the lone barrier that is inhibiting further adoption of Agile.
• Lean Startup is driving Agile adoption, but it’s mostly with organizations that have been practicing Agile for some time.

Purpose & Methodology

• What is inhibiting further adoption of Agile practices in your organization?
• How much influence have “Lean Startup” concepts influenced your organization’s desire to adopt Agile development practices?

The survey was short enough to help generate a sufficient number of survey responses but long enough to capture enough data that would allow for a meaningful analysis. These questions provide the opportunity for us to get a better understanding of the business drivers, capabilities and challenges companies are facing in their adoption and use of Agile practices. We also included a question on Lean Startup, since this method is gaining traction as a viable alternative to traditional product development and employs several Agile practices. To add to the analysis, we wanted to learn more about the companies represented by the survey respondents. We researched the respondents’ companies to identify company size and industry. Company size was categorized by number of staff: Small-to-medium sized businesses (SMBs) = <100, Midmarket = 100-999 and Enterprise = 1000+. While we saw some variance by industry and company size, our sample size was not large enough to report conclusive findings based on these variables. Throughout the report, we refer to several statistical techniques. For definitions of these techniques, please see the Appendix.

*Books such as Lean Marketing for Startups: Agile Product Development, Business Model Design, Web Analytics, and Other Keys to Rapid Growth by Sean Ellis and Marketing Agencies that tout Lean or Agile Marketing practices are appearing more frequently than ever before.

Respondents

• A variety of job roles and companies of all sizes
• Multiple industries, such as banking, insurance, ICT(information, communication and technology), retail
• Organizations with varied Agile experience. On average, respondents represent companies with just over two years of experience with Agile

Findings

We believe that importance can be seen as a proxy for expected results from Agile. Therefore, this could also have been interpreted as the gap between the desired outcome from Agile practices and the ability to achieve this outcome.

The following pages provide an overview of this gap as well as an exploration of the influences of organizational capability based on these 5 variables:

• Benefits of Agile
• Barriers to Agile
• Practice Makes Perfect? The Role of Experience
• The Influence of Job Role on Agile
• Agile Segments – Two Groups

The Agile Capability Gap

Figure 1: The gap between organizational capability with Agile versus the importance of Agile

The Benefits of Agile

Understanding the relationship between two benefits of Agile is helpful to get a sense of which benefits go well with one another. But alone, they leave an unclear picture of how these various benefits of Agile relate to broader business benefits.   Consequently, we grouped these benefits together to create related “factors”, or broader categories of benefits.   Using factor analysis, we found that the benefits listed in our survey could be loosely grouped into the following three factors:

• Software quality
• Speed and agility
• Resources: saving time and money

“Software quality” includes four variables (or benefits):  better fit of features of product needs, reduced risk, greater collaboration, and ability to adapt to changing priorities. “Speed and agility” includes time to market and ability to adapt. “Resources” include lower costs and faster time to market, which could be interpreted as saving staff time as well as satisfying the market with releasing products faster.   These three factors show the true breadth of Agile benefits: effectiveness (software quality), efficiency (speed and agility) and money (resources).***   “Greater Collaboration” has a 0.30 correlation and 0.001 significance with Agile importance and has a 0.35 correlation and 0.000 significance with Agile capability. The “ability to adapt to changing priorities” has a 0.35 correlation and 0.000 significance with Agile importance and has a 0.31 correlation and 0.001 significance with Agile capability.

**Statistically significant at the 0.05 level, i.e. p<.05.

***To see the factor scores, see the Appendix.

The Barriers to Agile

* Percentage of respondents who agreed that these barriers are inhibiting further Agile adoption in their organization

One of the most interesting findings is that the barrier of “management – not supportive of Agile” is negatively correlated with both the perceived importance of, and capability with, Agile.*   In other words, those who see management as a barrier to further Agile adoption in their organization are 

more likely to say that their organization places lower importance on Agile and are less capable with Agile relative to their peers. This is an unsurprising yet interesting finding because it shows how critical it is to have management support for Agile. A grassroots Agile effort may be the easiest way to implement Agile in an organization but it is by no means guarantees success if management isn’t involved.   The barrier of “approach – using Agile elements in a non-Agile framework” is noteworthy. Interestingly, while it was the most frequently selected barrier in our survey (46% of respondents), it is the only barrier that does not have any significant correlation (p<0.05) with another barrier. Sixty three percent of those who selected “approach” as a barrier selected two or fewer barriers that are inhibiting further adoption of Agile practices at their organization, with 24% selecting approach alone.   In other words, there are many people who find that their organization’s approach to Agile is the only or just one of a few barriers they face to improve Agile adoption.   Using factor analysis again, we found that the barriers listed in this survey could be loosely grouped into the following two factors: 1.Practice barriers 2.Environmental barriers “Practice barriers” include only two important variables (or barriers): approach and culture. “Environmental Barriers” include four important variables: staff, office layout, management, and culture. In essence, what’s getting in the way of Agile adoption appears to be either the environment in which one is trying to be Agile or the approach that one is taking with Agile.**

“Management – not supportive of Agile” has a -0.39 correlation and 0.000 significance with Agile importance and has a -0.37 correlation and 0.000 significance with Agile capability.

**To see the factor scores, see the Appendix.

Does Practice Make Perfect? The Role of Experience

Given the gap between capability and importance with Agile, we looked into the role of experience. Results indicate experience drives Agile capability but only to a limited extent. In the first year of implementing Agile, the average respondent rated organizational Agile capability at 5 out of 10. After more than a year of experience with Agile, the average increased to over 6 out of 10. Then, it appears organizations remain stuck at this level for a few years before they moderately improve to 7 out of 10.

Figure 3: Capability and importance ratings by Agile experience

While greater experience with Agile generally leads to better Agile capabilities, the rate of improvement is underwhelming. One might expect capability to increase more steadily with experience.  It seems as though organizations cannot rely on Agile experience alone to improve their capabilities. One theory is this is due to the common barriers to Agile, such as culture, approach, and staff (as described in “Barriers to Agile” section).   Regarding the perceived importance of Agile, the good news is that experience and average perceived importance are positively correlated. That is, the greater one’s experience with Agile, the more important it becomes. The bad news is that the rate of growth is rather slow – only about 7 percent per year after the first year with Agile. This may be due to the results or challenges from gaining experience with Agile.   It’s worth noting that this finding is based on the respondents’ perceptions. Consequently, it is possible that organizations are improving their capability at a steady and greater rate but are overestimating their organization’s Agile capabilities when new to Agile and are underestimating it once they are experienced. It is also possible that there is more than meets the eye with Agile and that those who are new to it underestimate what it entails.

The Influence of Job Role on Agile

People at all job levels share common views about Agile (see Figure 4). While it’s encouraging that the results show that Agile is seen as at least moderately important across job levels, this may be the result of response bias. After all, those who respond to an Agile survey likely think that Agile is somewhat important.  The findings are similar for organizational capabilities with Agile; moderately important across job levels but likely influenced by response bias (see Appendix for details). While we saw some variance in these ratings by job role, our sample size was not large enough to report any conclusive findings, as the margin of error is too large.

Figure 4: Capability and Importance of Agile to Organizations Strategy – Perceptions By Job Role

Lean Startup: Driving Agile Adoptions Further, Not Encouraging New Adoptions

The Lean Startup concept of validated learning goes well with the Agile principles of regularly reflecting on how to become more effective and of welcoming change.  Both Agile and Lean Startup use iterative processes, from development to product releases. Given these similarities, we were curious to understand how much influence Lean Startup concepts are having on the organizational desire to adopt Agile development practices.

We found that Lean Startup is influencing Agile adoptions, but to a limited and specific extent. When we asked survey respondents how much influence, on a scale of 1 to 10 with 10 being the highest, Lean Startup concepts have had on their organization’s desire to adopt Agile development practices, the median response was 4 and the most common response was a 1. However, despite this minimal impact, when we considered experience with Agile, the effect of Lean Startup on Agile became more evident.

Experience with Agile is positively correlated with the perceived influence Lean Startup is having on Agile adoption. This implies that those who have been Agile for longer are more likely to belong to an organization whose Agile adoption rate is influenced by the Lean Startup concept. Figure 4 

illustrates that for every experience category, the longer one has practiced Agile, the higher the level of influence that Lean Startup tends to have on their organization’s Agile adoption.

Because the sample size was not large enough to report any conclusive findings, it seems as though industry sector may be an influence of Lean Startup on Agile adoptions. On average, ICT respondents provided a 4.22 rating whereas Financial Services respondents provided a 2.70.*

Agile Segments – Two Groups

We explored whether there are any natural groups of people with similarities related to Agile. Using cluster analysis, we determined that while there were no tightly-knit clusters in our sample, there were two loosely-connected groups that can be described as follows:

• The Veterans: Those who consider Agile to be among the top priorities for their organization, are very good with Agile, and are influenced by Lean Startup
• The Rookies: Those who consider Agile as only having moderate importance to their organization, are subpar at it, and are not influenced by Lean Startup

The Veterans:

• Experience:  about 3 years
• Importance of Agile: 9 of 10
• Capability with Agile: 7 of 10
• Influence of Lean Startup on Agile: 6 of 10

• Believe Agile helps their organization have greater collaboration
• Do not believe staff or company culture are inhibiting Agile adoption

 The Rookies:

• Experience:  about 1.5 years
• Importance of Agile: 5 of 10
• Capability with Agile: 4 of 10
• Influence of Lean Startup on Agile: 2 of 10
• Do not believe Agile helps their organization have greater collaboration
• Believe both staff and company culture are inhibiting Agile adoption

Conclusion

The response data from this survey reveals several insights into the adoption of Agile in today’s organization. There are two main areas that should be considered by anyone in an organization that is Agile or is considering this approach:

1. Agile is enabling a more adaptive organization, and,
2. Despite a will to be Agile, there is a gap between good intentions and actual capability.

Organizations are recognizing the imperative to be nimble and action-oriented in order to manage changing priorities. The results of this survey indicate that Agile is a key enabler for this: from “the ability to adapt to 

changing priorities” to “greater collaboration”, organizations are coming more adaptive because of Agile.

Although Agile is helping companies become more adaptive, the Agile capability gap needs to be addressed. Organizations are headed in the right direction in terms of placing strategic value on Agile. Now they need to address their actual ability to implement and maintain a high-performing Agile culture. Hiring experienced Agile practitioners is only part of the solution. Other aspects of the organization must be addressed, most importantly, meaningful support from the top – not only in embracing the idea, but also by removing the barriers that inhibit Agile and creating an environment that supports Agile.

Appendix

Definitions of Statistical Terms

Correlation: A statistical measure that indicates the extent to which two or more variables fluctuate together. A positive correlation means that the variables tend to be either large or small together. For example, education and income are positively correlated. A negative correlation means that as the value of one variable increases, the other(s) decrease(s). For example, TV viewing and grades (for students) are negatively correlated. Correlation ranges from +1 between (perfect correlation) to –1 (perfect negative correlation).

Statistically significant: The probability that an effect is not likely due to just chance alone. P-values are often coupled to a significance or alpha (α) level, which is also set ahead of time, usually at 0.05 (5%), as they are in this report.

Factor Analysis: a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. It helps identify groups of related items (called factors). For example, in a decathlon, each athlete competes in 10 events, but several of them can be thought of as “speed” events, while others can be thought of as “strength” events, etc. Thus, a competitor’s 10 event scores might be thought of as largely dependent on a smaller set of 3 or 4 types of athletic ability.

Cluster Analysis: the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters).