This is Part 1 (go to Part 2 >> | go to Part 3 >>)
Three weeks ago I initiated The Big Agile Practices Survey. In these three weeks, with the help of many bloggers and twitterers, my survey attracted 341 participants. The survey was actually divided in six mini-surveys, for requirements, design, construction, testing, process, and organization.
Not everyone bothered to answer all questions, which is not surprising. The survey was indeed quite BIG: spanning 67 software engineering practices, with six questions per practice. That's 402 checkboxes in total! I am very, very thankful to all those participants who had the stamina to fill out the extensive forms. I won't ask this again any time soon, I promise. :)
As is often the case with surveys, this one got some criticism from both famous and less famous people. Johanna Rothman wrote that surveys like this may be harmful. And James Shore told me that the survey suffers from selection bias. And they are both right! Every poll on the Internet suffers from selection bias, and if data is not interpreted and used correctly the results can indeed be harmful.
Of course, the same applies to the selection process for Agile 2009. It was biased from the start (but I don't care). And the selection of practices in agile books, like the Art of Agile Development, also biased and potentially harmful. And don't forget my own Top 100 Blogs for Developers. Very biased indeed. However, there IS value in knowing what other people think. Though you should never let those opinions dictate your own actions. Which is why I ignored the critics and went my own way... :)
Fortunately, I had many more supporters than critics. Here's a big thanks to the bloggers who helped me with the promotion of this survey: Dave Nicolette, Corey Ladas, Mike Cohn, Mike Cottmeyer, Alberto Brandolini, Alvin Ashcraft, Robert Dempsey, Kirk Knoernschild (DZone), Raven Young, Amr Elssamadisy (InfoQ), Lee Henson, Dahlia Bock, Artem Marchenko (AgileSoftwareDevelopment), and all those on Twitter who tweeted about this survey. My hard work would have been in vain without you! Thank you.
Note: you can also analyze the data yourself in big on-line sortable tables: here (created by Mario Menger) and here (created by Maciej Gren).
And now, the results...
| Which practices are REALLY AGILE, or... highest level of agility? |
| Area |
Best practice |
Score |
| Process |
Iteration Planning / Planning Game / Sprint Planning |
98.8% |
| Process |
Velocity |
98.1% |
| Process |
Sprint Backlog |
98.1% |
| Process |
Daily Stand-up Meeting / Daily Scrum |
97.6% |
| Organization |
Self-Organizing Team / Scrum Team |
97.5% |
| Process |
Story Points |
96.9% |
| Organization |
Scrum Master |
96.3% |
| Process |
Sprint Review / Iteration Demo |
96.3% |
| Process |
Timeboxing / Fixed Sprints / Fixed Iteration Length |
96.0% |
| Requirements |
Product Backlog |
95.6% |
Comments: This is the top 10 practices that the participants in this survey most often associated with agile software development. It is clear that Iterative Planning is the clear winner for this question. And it's interesting to see that the practices from the Process and Organization categories are the ones that are strongest associated with agile.
| Which practices are NOT REALLY AGILE, or... lowest level of agility? |
| Area |
Best practice |
Score |
| Construction |
Configuration Management |
25.2% |
| Construction |
Issue Tracking / Bug Tracking |
27.4% |
| Requirements |
Use Cases |
28.2% |
| Process |
Risk Management |
29.5% |
| Design |
Design by Contract |
32.2% |
| Construction |
Software Metrics / Code Metrics & Analysis |
32.5% |
| Requirements |
Usage Scenarios |
37.9% |
| Construction |
Code Reviews / Peer Reviews |
42.3% |
| Process |
Root Cause Analysis / 5 Whys |
42.6% |
| Organization |
Move People Around |
42.6% |
Comments: This is the top 10 practices that are least associated with agile software development. Configuration Management is the winner for this question, and interestingly enough four practices are from the Construction category. Agile software development is clearly more associated with organization and processes than with tools/practices that support the construction of code.
| On which practices was the LEAST AGREEMENT regarding level of agility? |
| Area |
Best practice |
Score |
| Design |
User Interface Prototyping |
1.1% |
| Construction |
Coding Style / Coding Guidelines / Coding Standard |
1.9% |
| Construction |
Source Control / Version Control |
4.1% |
| Requirements |
Personas |
5.4% |
| Testing |
Integration Testing |
6.8% |
| Testing |
System Testing |
6.9% |
| Design |
Domain Driven Design |
7.9% |
| Testing |
Smoke Testing / Build Verification Test |
9.0% |
| Testing |
Exploratory Testing |
11.3% |
| Organization |
Move People Around |
14.8% |
Comments: For some practices the answers were split almost evenly, which indicates lack of agreement on what it is to be agile. This is the top 10 of practices with the least agreement among participants. User Interface Prototyping is the winner here, with an (almost) even split among Yes and No answers. And it is a bit surprising to see no less than four practices from the Testing category.
| On which practices was the LEAST CONFIDENCE regarding level of agility? |
| Area |
Best practice |
Score |
| Process |
Lead Time / Cycle Time |
49.2% |
| Process |
Value Stream Mapping |
54.6% |
| Requirements |
Personas |
59.5% |
| Design |
System Metaphor |
60.6% |
| Requirements |
Usage Scenarios |
61.9% |
| Process |
Root Cause Analysis / 5 Whys |
62.2% |
| Design |
CRC Cards |
64.8% |
| Requirements |
Defer Decisions / Real Options |
65.1% |
| Process |
Kanban Board |
65.9% |
| Construction |
Behavior Driven Development |
66.3% |
Comments: Not everyone answered every question. In fact, some questions were ignored by many, which indicates a lack of confidence in being able to answer the questions (due to the practices being not well-known or not well-described). Here it seems that people had the most trouble with lean practices, like Lead Time, Value Stream Mapping, and Kanban Board.
| Which practices are REALLY IMPORTANT, or... highest level of importance? |
| Area |
Best practice |
Score |
| Construction |
Source Control / Version Control |
100.0% |
| Process |
Definition of Done / Done Done |
99.4% |
| Construction |
Refactoring |
98.9% |
| Organization |
Sustainable Pace |
98.6% |
| Construction |
Frequent Delivery / Frequent Releases |
98.3% |
| Requirements |
Product Backlog |
98.2% |
| Requirements |
Requirement Prioritization |
98.2% |
| Testing |
Unit Testing |
98.2% |
| Testing |
Storytesting / Acceptance Criteria / Acceptance Testing |
98.1% |
| Process |
Retrospective / Reflection Workshop |
98.0% |
Comments: When talking about importancy, participants unanimously agreed that Source Control is an important practice. More suprisingly, there is not a single design practice in this top 10 of most important practices.
| Which practices are NOT REALLY IMPORTANT, or... lowest level of importance? |
| Area |
Best practice |
Score |
| Design |
System Metaphor |
38.1% |
| Design |
CRC Cards |
40.7% |
| Process |
Kanban Board |
52.1% |
| Requirements |
Use Cases |
52.2% |
| Design |
Design by Contract |
52.9% |
| Organization |
Move People Around |
53.4% |
| Requirements |
Personas |
56.0% |
| Construction |
Software Metrics / Code Metrics & Analysis |
57.1% |
| Requirements |
Usage Scenarios |
60.8% |
| Organization |
Scrum of Scrums |
64.2% |
Comments: In the top 10 of least important practices we see a significant number of requirements and design practices. (Of course, this does not mean that design and requirements in general are unimportant.)
| On which practices was the LEAST AGREEMENT regarding level of importance? |
| Area |
Best practice |
Score |
| Process |
Kanban Board |
4.2% |
| Requirements |
Use Cases |
4.3% |
| Design |
Design by Contract |
5.9% |
| Organization |
Move People Around |
6.8% |
| Requirements |
Personas |
12.0% |
| Construction |
Software Metrics / Code Metrics & Analysis |
14.3% |
| Design |
CRC Cards |
18.6% |
| Requirements |
Usage Scenarios |
21.6% |
| Design |
System Metaphor |
23.9% |
| Organization |
Scrum of Scrums |
28.3% |
Comments: This is the top 10 of practices with the least agreement among participants, regarding their importance. What is most intriguing here is that these are exactly the same results as in the previous question, only in another order. The practices that people find least important are apparently also the ones for which public opinion varies the most!
| On which practices was the LEAST CONFIDENCE regarding level of importance? |
| Area |
Best practice |
Score |
| Process |
Lead Time / Cycle Time |
46.5% |
| Process |
Value Stream Mapping |
47.6% |
| Requirements |
Personas |
51.3% |
| Process |
Kanban Board |
51.9% |
| Design |
System Metaphor |
53.1% |
| Design |
CRC Cards |
55.4% |
| Design |
Design by Contract |
55.9% |
| Requirements |
Usage Scenarios |
56.9% |
| Organization |
Move People Around |
57.2% |
| Process |
Root Cause Analysis / 5 Whys |
57.8% |
Comments: This is the top 10 of practices that people ignored the most, when asked about importance. Again, the lean practices dominate the results, meaning that many people in agile software development haven't made up their minds yet about the importance of specific lean practices.
| Which practices are REALLY APPLIED, or... highest level of application? |
| Area |
Best practice |
Score |
| Construction |
Source Control / Version Control |
100.0% |
| Requirements |
Requirement Prioritization |
94.4% |
| Testing |
Unit Testing |
93.4% |
| Requirements |
Product Backlog |
93.2% |
| Process |
Iteration Planning / Planning Game / Sprint Planning |
92.9% |
| Construction |
Refactoring |
91.8% |
| Process |
Daily Stand-up Meeting / Daily Scrum |
90.8% |
| Process |
Timeboxing / Fixed Sprints / Fixed Iteration Length |
90.8% |
| Construction |
Daily Builds / Automated Builds / Ten-Minute Builds |
90.3% |
| Testing |
Integration Testing |
88.2% |
Comments: This is the top 10 of practices most often applied by the participants in their own organizations. Again, Source Control is the clear winner, with a significant lead. And again, there are no clear winners from the Design category of practices.
| Which practices are NOT REALLY APPLIED, or... lowest level of application? |
| Area |
Best practice |
Score |
| Design |
System Metaphor |
19.0% |
| Process |
Kanban Board |
20.9% |
| Design |
CRC Cards |
22.8% |
| Process |
Value Stream Mapping |
24.2% |
| Design |
Design by Contract |
25.3% |
| Process |
Lead Time / Cycle Time |
29.4% |
| Requirements |
Usage Scenarios |
30.3% |
| Requirements |
Personas |
32.6% |
| Construction |
Behavior Driven Development |
35.1% |
| Requirements |
Use Cases |
36.2% |
Comments: In this list of top 10 practices that are not applied, we see both lean and design practices dominating the results. Interestingly enough, the Testing category is not present in these results, which might be good news for quality assurance people around the world. :)
| On which practices was the LEAST AGREEMENT regarding level of application? |
| Area |
Best practice |
Score |
| Construction |
Software Metrics / Code Metrics & Analysis |
1.2% |
| Construction |
Pair-Programming / Pairing |
4.5% |
| Design |
Domain Driven Design |
7.1% |
| Process |
Root Cause Analysis / 5 Whys |
8.0% |
| Process |
Risk Management |
14.7% |
| Requirements |
Product Vision / Vision Statement |
16.5% |
| Construction |
Code Reviews / Peer Reviews |
18.1% |
| Organization |
Move People Around |
18.5% |
| Requirements |
Minimum Marketable Features |
18.6% |
| Organization |
Scrum of Scrums |
22.9% |
Comments: This is the top 10 of practices with the least agreement among participants. It is interesting to see Software Metrics topping this list for the first time, meaning there are about as many people applying software metrics as there are people not applying software metrics. The Testing category is also conspicuously absent in this case.
| On which practices was the LEAST CONFIDENCE regarding level of application? |
| Area |
Best practice |
Score |
| Process |
Lead Time / Cycle Time |
64.3% |
| Process |
Value Stream Mapping |
67.0% |
| Requirements |
Defer Decisions / Real Options |
68.0% |
| Design |
System Metaphor |
69.0% |
| Requirements |
Personas |
71.0% |
| Organization |
Move People Around |
72.2% |
| Process |
Root Cause Analysis / 5 Whys |
74.1% |
| Design |
Design by Contract |
74.2% |
| Requirements |
Usage Scenarios |
74.5% |
| Process |
Kanban Board |
75.1% |
Comments: This is the top 10 of practices that people ignored the most, when asked about their own application of practices. Not surprisingly, the lean practices dominate the results again. Either people don't know what the practices are, or they don't know whether they're being applied in their organizations.
This was Part 1 (go to Part 2 >> | go to Part 3 >>)
Note: you can also analyze the data yourself in big on-line sortable tables: here (created by Mario Menger) and here (created by Maciej Gren).
Twitter - 
Subscribe - 
Newsletter - 
Bookmarks
Latest, greatest and favoritest posts:
The Big List of Agile Practices
Top 100 Blogs for Developers (Q1 2009)
Top 50 New Software Development Books
Twitter - 
Subscribe - 
Newsletter - 
LinkedIn - 
SlideShare