big

NOOP.NL | The Creative Networker

The Big Agile Practices Survey Report (Part 1)

11/05/2009

Pie This is Part 1 (go to Part 2 >> | go to Part 3 >>)

Three weeks ago I initiated The Big Agile Practices Survey. In these three weeks, with the help of many bloggers and twitterers, my survey attracted 341 participants. The survey was actually divided in six mini-surveys, for requirements, design, construction, testing, process, and organization.

Not everyone bothered to answer all questions, which is not surprising. The survey was indeed quite BIG: spanning 67 software engineering practices, with six questions per practice. That's 402 checkboxes in total! I am very, very thankful to all those participants who had the stamina to fill out the extensive forms. I won't ask this again any time soon, I promise. :)

As is often the case with surveys, this one got some criticism from both famous and less famous people. Johanna Rothman wrote that surveys like this may be harmful. And James Shore told me that the survey suffers from selection bias. And they are both right! Every poll on the Internet suffers from selection bias, and if data is not interpreted and used correctly the results can indeed be harmful.

Of course, the same applies to the selection process for Agile 2009. It was biased from the start (but I don't care). And the selection of practices in agile books, like the Art of Agile Development, also biased and potentially harmful. And don't forget my own Top 100 Blogs for Developers. Very biased indeed. However, there IS value in knowing what other people think. Though you should never let those opinions dictate your own actions. Which is why I ignored the critics and went my own way… :)

Fortunately, I had many more supporters than critics. Here's a big thanks to the bloggers who helped me with the promotion of this survey: Dave Nicolette, Corey Ladas, Mike Cohn, Mike Cottmeyer, Alberto Brandolini, Alvin Ashcraft, Robert Dempsey, Kirk Knoernschild (DZone), Raven Young, Amr Elssamadisy (InfoQ), Lee Henson, Dahlia Bock, Artem Marchenko (AgileSoftwareDevelopment), and all those on Twitter who tweeted about this survey. My hard work would have been in vain without you! Thank you.

Note: you can also analyze the data yourself in big on-line sortable tables: here (created by Mario Menger) and here (created by Maciej Gren).

And now, the results…


Which practices are REALLY AGILE, or… highest level of agility?
Area Best practice Score
Process Iteration Planning / Planning Game / Sprint Planning 98.8%
Process Velocity 98.1%
Process Sprint Backlog 98.1%
Process Daily Stand-up Meeting / Daily Scrum 97.6%
Organization Self-Organizing Team / Scrum Team 97.5%
Process Story Points 96.9%
Organization Scrum Master 96.3%
Process Sprint Review / Iteration Demo 96.3%
Process Timeboxing / Fixed Sprints / Fixed Iteration Length 96.0%
Requirements Product Backlog 95.6%

Comments: This is the top 10 practices that the participants in this survey most often associated with agile software development. It is clear that Iterative Planning is the clear winner for this question. And it's interesting to see that the practices from the Process and Organization categories are the ones that are strongest associated with agile.


Which practices are NOT REALLY AGILE, or… lowest level of agility?
Area Best practice Score
Construction Configuration Management 25.2%
Construction Issue Tracking / Bug Tracking 27.4%
Requirements Use Cases 28.2%
Process Risk Management 29.5%
Design Design by Contract 32.2%
Construction Software Metrics / Code Metrics & Analysis 32.5%
Requirements Usage Scenarios 37.9%
Construction Code Reviews / Peer Reviews 42.3%
Process Root Cause Analysis / 5 Whys 42.6%
Organization Move People Around 42.6%

Comments: This is the top 10 practices that are least associated with agile software development. Configuration Management is the winner for this question, and interestingly enough four practices are from the Construction category. Agile software development is clearly more associated with organization and processes than with tools/practices that support the construction of code.


On which practices was the LEAST AGREEMENT regarding level of agility?
Area Best practice Score
Design User Interface Prototyping 1.1%
Construction Coding Style / Coding Guidelines / Coding Standard 1.9%
Construction Source Control / Version Control 4.1%
Requirements Personas 5.4%
Testing Integration Testing 6.8%
Testing System Testing 6.9%
Design Domain Driven Design 7.9%
Testing Smoke Testing / Build Verification Test 9.0%
Testing Exploratory Testing 11.3%
Organization Move People Around 14.8%

Comments: For some practices the answers were split almost evenly, which indicates lack of agreement on what it is to be agile. This is the top 10 of practices with the least agreement among participants. User Interface Prototyping is the winner here, with an (almost) even split among Yes and No answers. And it is a bit surprising to see no less than four practices from the Testing category.


On which practices was the LEAST CONFIDENCE regarding level of agility?
Area Best practice Score
Process Lead Time / Cycle Time 49.2%
Process Value Stream Mapping 54.6%
Requirements Personas 59.5%
Design System Metaphor 60.6%
Requirements Usage Scenarios 61.9%
Process Root Cause Analysis / 5 Whys 62.2%
Design CRC Cards 64.8%
Requirements Defer Decisions / Real Options 65.1%
Process Kanban Board 65.9%
Construction Behavior Driven Development 66.3%

Comments: Not everyone answered every question. In fact, some questions were ignored by many, which indicates a lack of confidence in being able to answer the questions (due to the practices being not well-known or not well-described). Here it seems that people had the most trouble with lean practices, like Lead Time, Value Stream Mapping, and Kanban Board.


Which practices are REALLY IMPORTANT, or… highest level of importance?
Area Best practice Score
Construction Source Control / Version Control 100.0%
Process Definition of Done / Done Done 99.4%
Construction Refactoring 98.9%
Organization Sustainable Pace 98.6%
Construction Frequent Delivery / Frequent Releases 98.3%
Requirements Product Backlog 98.2%
Requirements Requirement Prioritization 98.2%
Testing Unit Testing 98.2%
Testing Storytesting / Acceptance Criteria / Acceptance Testing 98.1%
Process Retrospective / Reflection Workshop 98.0%

Comments: When talking about importancy, participants unanimously agreed that Source Control is an important practice. More suprisingly, there is not a single design practice in this top 10 of most important practices.


Which practices are NOT REALLY IMPORTANT, or… lowest level of importance?
Area Best practice Score
Design System Metaphor 38.1%
Design CRC Cards 40.7%
Process Kanban Board 52.1%
Requirements Use Cases 52.2%
Design Design by Contract 52.9%
Organization Move People Around 53.4%
Requirements Personas 56.0%
Construction Software Metrics / Code Metrics & Analysis 57.1%
Requirements Usage Scenarios 60.8%
Organization Scrum of Scrums 64.2%

Comments: In the top 10 of least important practices we see a significant number of requirements and design practices. (Of course, this does not mean that design and requirements in general are unimportant.)


On which practices was the LEAST AGREEMENT regarding level of importance?
Area Best practice Score
Process Kanban Board 4.2%
Requirements Use Cases 4.3%
Design Design by Contract 5.9%
Organization Move People Around 6.8%
Requirements Personas 12.0%
Construction Software Metrics / Code Metrics & Analysis 14.3%
Design CRC Cards 18.6%
Requirements Usage Scenarios 21.6%
Design System Metaphor 23.9%
Organization Scrum of Scrums 28.3%

Comments: This is the top 10 of practices with the least agreement among participants, regarding their importance. What is most intriguing here is that these are exactly the same results as in the previous question, only in another order. The practices that people find least important are apparently also the ones for which public opinion varies the most!


On which practices was the LEAST CONFIDENCE regarding level of importance?
Area Best practice Score
Process Lead Time / Cycle Time 46.5%
Process Value Stream Mapping 47.6%
Requirements Personas 51.3%
Process Kanban Board 51.9%
Design System Metaphor 53.1%
Design CRC Cards 55.4%
Design Design by Contract 55.9%
Requirements Usage Scenarios 56.9%
Organization Move People Around 57.2%
Process Root Cause Analysis / 5 Whys 57.8%

Comments: This is the top 10 of practices that people ignored the most, when asked about importance. Again, the lean practices dominate the results, meaning that many people in agile software development haven't made up their minds yet about the importance of specific lean practices.


Which practices are REALLY APPLIED, or… highest level of application?
Area Best practice Score
Construction Source Control / Version Control 100.0%
Requirements Requirement Prioritization 94.4%
Testing Unit Testing 93.4%
Requirements Product Backlog 93.2%
Process Iteration Planning / Planning Game / Sprint Planning 92.9%
Construction Refactoring 91.8%
Process Daily Stand-up Meeting / Daily Scrum 90.8%
Process Timeboxing / Fixed Sprints / Fixed Iteration Length 90.8%
Construction Daily Builds / Automated Builds / Ten-Minute Builds 90.3%
Testing Integration Testing 88.2%

Comments: This is the top 10 of practices most often applied by the participants in their own organizations. Again, Source Control is the clear winner, with a significant lead. And again, there are no clear winners from the Design category of practices.


Which practices are NOT REALLY APPLIED, or… lowest level of application?
Area Best practice Score
Design System Metaphor 19.0%
Process Kanban Board 20.9%
Design CRC Cards 22.8%
Process Value Stream Mapping 24.2%
Design Design by Contract 25.3%
Process Lead Time / Cycle Time 29.4%
Requirements Usage Scenarios 30.3%
Requirements Personas 32.6%
Construction Behavior Driven Development 35.1%
Requirements Use Cases 36.2%

Comments: In this list of top 10 practices that are not applied, we see both lean and design practices dominating the results. Interestingly enough, the Testing category is not present in these results, which might be good news for quality assurance people around the world. :)


On which practices was the LEAST AGREEMENT regarding level of application?
Area Best practice Score
Construction Software Metrics / Code Metrics & Analysis 1.2%
Construction Pair-Programming / Pairing 4.5%
Design Domain Driven Design 7.1%
Process Root Cause Analysis / 5 Whys 8.0%
Process Risk Management 14.7%
Requirements Product Vision / Vision Statement 16.5%
Construction Code Reviews / Peer Reviews 18.1%
Organization Move People Around 18.5%
Requirements Minimum Marketable Features 18.6%
Organization Scrum of Scrums 22.9%

Comments: This is the top 10 of practices with the least agreement among participants. It is interesting to see Software Metrics topping this list for the first time, meaning there are about as many people applying software metrics as there are people not applying software metrics. The Testing category is also conspicuously absent in this case.


On which practices was the LEAST CONFIDENCE regarding level of application?
Area Best practice Score
Process Lead Time / Cycle Time 64.3%
Process Value Stream Mapping 67.0%
Requirements Defer Decisions / Real Options 68.0%
Design System Metaphor 69.0%
Requirements Personas 71.0%
Organization Move People Around 72.2%
Process Root Cause Analysis / 5 Whys 74.1%
Design Design by Contract 74.2%
Requirements Usage Scenarios 74.5%
Process Kanban Board 75.1%

Comments: This is the top 10 of practices that people ignored the most, when asked about their own application of practices. Not surprisingly, the lean practices dominate the results again. Either people don't know what the practices are, or they don't know whether they're being applied in their organizations.


This was Part 1 (go to Part 2 >> | go to Part 3 >>)

Note: you can also analyze the data yourself in big on-line sortable tables: here (created by Mario Menger) and here (created by Maciej Gren).

Twitter TwitterRss SubscribeEmail NewsletterDelicious Bookmarks

Latest, greatest and favoritest posts:
The Big List of Agile Practices
Top 100 Blogs for Developers (Q1 2009)
Top 50 New Software Development Books

This article is written by on in Top Lists. Jurgen Appelo is at Happy Melly. Connect with Jurgen Appelo on .

This article was posted in:

  • http://www.hericus.com/ Mr. Hericus

    I’m glad to see that Daily Builds / Automated Builds are something that people (at least 90%) are really implementing because I think it’s one of the real key’s to freeing the development team to focus on the code and quality without having to worry about some of the process.

  • http://profile.typepad.com/6p01156fb131c2970c gbcambridge

    Hi,
    thanks for some very useful information. It agrees with what I see in industry, but with just one issue. To get a better handle on what is actually going on in industry, being used and so on, you need to get an idea of the type of project. So, basically, you need to be able to break down the work by type of Application and Project. Clearly different approaches can be used for modifying a running system, where there is likely to be in-house experience, from a green-field project. Similarly for mission-critical vs. web page apps. I think this break down would add greatly to the value of such metrics and should probably explain why Use Cases are so low… how many green-field projects are there? There are certain core principles which will apply everywhere.. iteration/continuous build-test and so on. There are others which depend on the level of risk… the lower the level, the less process and procedure you need. This would be very useful information if you could pull it out of the feedback that you have.