APPENDIX X3

AGILE SUITABILITY FILTER TOOLS

X3.1 INTRODUCTION

Agile literature contains many agile suitability filter tools to help assess under what circumstances an agile approach is appropriate to use. In 1994, the Dynamic Systems Development Method (DSDM) developed an Agile Project Suitability Questionnaire and an Organizational Suitability Questionnaire to help gauge likely fit and potential problem areas.

The Crystal family of approaches also employed suitability criteria, ranking projects by team size and the criticality of the product or service being developed. Crystal recommends that smaller, less critical projects be undertaken with lighter controls and simpler approaches. Large, mission or life critical projects were recommended to use more rigor and validation.

Since the development of these approaches, there have been many more models created to help determine where and when to employ agile approaches. Boehm and Turner adopted some of the elements from DSDM and Crystal to develop a popular assessment model to help determine if projects should be undertaken with agile or more traditional approaches.

Based on these previous models and expanded to consider the middle ground of hybrid approaches, the following model is proposed. It represents a synthesis of several suitability filter attributes to help organizations assess and discuss whether projects should be undertaken using predictive, hybrid, or agile approaches.

X3.2 OVERVIEW OF THE MODEL

Organizational and project attributes are assessed under three main categories:

• Culture. Is there a supportive environment with buy-in for the approach and trust in the team?
• Team. Is the team of a suitable size to be successful in adopting agile, do its members have the necessary experience and access to business representatives to be successful?
• Project. Are there high rates of change? Is incremental delivery possible? How critical is the project?

Questions in each of these categories are answered and the results plotted on a radar chart. Clusters of values around the center of the chart indicate a good fit for agile approaches. Results around the outside indicate a predictive approach may be more suitable. Values in the middle portion (between agile and predictive) indicate a hybrid approach could work well. An example is shown in Figure X3-1.

X3.3 INSTRUCTIONS FOR USE

X3.3.1 COMPLETE THE QUESTIONNAIRE AS A GROUP

For small projects, this group may simply be the sponsor, technical lead, and a customer. For large projects, this may include representatives from the sponsoring group, project execution team, impacted business group(s), project governance group(s), and customer community. The idea is that just as no single stakeholder should estimate or plan a project because of representing only one viewpoint and having personal bias; so too should no single person assess the suitability of an approach since any one person will also have a limited view with a bias.

Instead, the value of the tool is the conversation it encourages with the invested parties of the project. Even if the results point to a hybrid approach, but the stakeholders want to proceed with a largely agile or predictive approach, follow the stakeholder consensus. This tool is a high-level diagnostic only, the final decision should rest and be supported by the people involved.

X3.3.2 SCORE THE QUESTIONS FROM 1 TO 10

As a group, discuss and agree (or compromise) on a score that most accurately reflects the subjective evaluation of the question. While definitive options are only provided for the start, middle, and end points of the answer spectrum representing scores of 1, 5, and 10, it is fine (and desirable) to use scores such as 2 for “almost a 1, but not quite,” or 7 for “somewhere between a 5 and a 10.” Again, the assessment is a discussion tool—views will be subjective and shades of gray are to be expected.

When the group cannot agree on a score, discuss the issues openly and honestly. Before suggesting compromises (i.e., using average scores or marking PMO scores with a blue “X” and the development team with a green “O”), consider how successful is the project likely to be when the participants cannot agree on completing a simple assessment? When discussing the issues, if the differences of opinion can be identified—then great, it is working; now come to an agreement. Likewise, if the assessment indicates a predictive approach but everyone wants to try an agile approach (or vice versa) that is fine too, just understand the issues and discuss how the impacts of the approach will be handled.

X3.3.3 INTERPRET THE RESULTS

Mark the answers from the questions on a blank suitability assessment chart and connect the points. Results clustered around the center in the agile zone indicate a good fit for a purely agile approach.

Results predominantly in the hybrid zone indicate some combination of agile and predictive approaches might work best. However, it is also possible that an agile approach with some additional risk reduction steps such as extra education and training or extra validation and documentation rigor in the case of high criticality projects may suffice. Alternatively, a predictive approach with some proof-of-concept work or extra processes could also work.

Results predominantly in the predictive zone indicate a good fit for a purely predictive approach. As mentioned in Section X3.3.2 (Score the Questions step), this diagnostic tool is aimed at starting meaningful conversations with the impacted parties about the most appropriate approach to use. If the approach suggested by the tool is not acceptable it is allowed to use a different approach. Use the results as inputs to the risk management process, since the tool indicates mismatches that will need to be managed.

X3.4 SUITABILITY FILTER QUESTIONS

X3.4.1 CATEGORY: CULTURE

X3.4.1.1 BUY-IN TO APPROACH

Is there senior sponsor understanding and support for using an agile approach for this project? See Figure X3-2.

X3.4.1.2 TRUST IN TEAM

Considering the sponsors and the business representatives who will be working with the team. Do these stakeholders have confidence that the team can transform their vision and needs into a successful product or service—with ongoing support and feedback going both directions? See Figure X3-3.

X3.4.1.3 DECISION-MAKING POWERS OF TEAM

Will the team be given autonomy to make their own local decisions about how to undertake work? See Figure X3-4.

X3.4.2 CATEGORY: TEAM

X3.4.2.1 TEAM SIZE

What is the size of the core team? Use this scale: 1-9 = 1, 10-20 = 2, 21-30 = 3, 31-45 = 4, 46-60 = 5, 61-80 = 6, 81-110 = 7, 111-150 = 8, 151 – 200 = 9, 201+ = 10. See Figure X3-5.

X3.4.2.2 EXPERIENCE LEVELS

Considering the experience and skill levels of the core team roles. While it is normal to have a mix of experienced and inexperienced people in roles, for agile projects to go smoothly; it is easier when each role has at least one experienced member. See Figure X3-6.

X3.4.2.3 ACCESS TO THE CUSTOMER/BUSINESS

Will the team have daily access to at least one business/customer representative to ask questions and get feedback? See Figure X3-7.

X3.4.3 CATEGORY: PROJECT

X3.4.3.1 LIKELIHOOD OF CHANGE

What percentage of requirements are likely to change or be discovered on a monthly basis? See Figure X3-8.

X3.4.3.2 CRITICALITY OF PRODUCT OR SERVICE

To help determine likely levels of additional verification and documentation rigor that may be required, assess the criticality of the product or service being built. Using an assessment that considers loss due to possible impact of defects, determine what a failure could result in. See Figure X3-9.

X3.4.3.3 INCREMENTAL DELIVERY

Can the product or service be built and evaluated in portions? Also, will business or customer representatives be available to provide timely feedback on increments delivered? See Figure X3-10.

X3.5 SUITABILITY ASSESSMENT CHART

Figure X3-11 is the radar chart used for the suitability assessment.

X3.5.1 CASE STUDIES

To illustrate how the radar chart works, here are two examples of using the model to score very different types of projects. The first is an example of an online drug store project (see Figure X3-12) and the second (Figure X3-13) is an example of a military messaging system. These two case studies illustrate some of the variances seen on projects. Central clustering indicates a good fit for agile approaches, peripheral scores indicate predictive approaches might be more suitable. Some projects are centered around the middle but then spike out on one or two axes. These projects may be best solved with a hybrid approach.

X3.5.1.1 DRUG STORE EXAMPLE

The project was to develop an online drug store to sell cheaper Canadian prescription drugs to (primarily) U.S. customers. The sale of these drugs is a contentious subject in Canada as well as the U.S. and as a result the industry is characterized by swift regulation changes and fierce competition. The project faced extremely volatile requirements with major changes week on week. It used very short (2-day) iterations and weekly releases to tackle the high rates of change.

As shown in Figure X3-12, high levels of buy-in and trust are evident for those who worked in an empowered way. The visual nature of the website made it easy to show new increments of functionality, but the system criticality was fairly high with essential funds for the pharmacy at stake. As mentioned, there were very high rates of change, but the small experienced team handled them well and had easy access to a knowledgeable business representative. The approach was very successful and extremely agile.

X3.5.1.2 MILITARY MESSAGING SYSTEM EXAMPLE

Contrast the first example with a large project to develop military messaging system that had already been running for 5 years when the assessment was made. See Figure X3-13.

Buy-in for an agile approach was lacking because an agile approach was not being considered. Trust in the vendors was mixed but generally respected. Decision making was not local, but instead made by architecture and requirements committees. While elements of the design could be tested incrementally in a laboratory, they could not be gathered together for an end to end demonstration of functionality. Many lives were potentially at risk, so criticality was very high. Requirements were locked down because changes impacted so many subcontractor organizations.

The project was large with more than 300 people from one vendor alone, but each role had many experienced practitioners. Finally, access to the business/customer was not possible, but contract analysts were available to ask specification questions to and they usually replied or asked clarifying questions within 10 days. Parts of the project could have been carved off and run as agile projects, but at the heart of the initiative was a single large project.

X3.6 SUMMARY

Agile suitability filters are useful tools for identifying potential fits and gaps for agile approaches. They should not be used as definitive inclusion or exclusion gates, but instead as topics for objective discussion with all interested parties.