15
Which Projects Should Be Agile?
Globally, many organizations have had struggles with troubled information technology software development projects. As a result, some of these organizations have made attempts to experiment with newer approaches such as agile methods with the expectation that value could be delivered early alongside superior quality features. The hope was that this change in approach would certainly result in more satisfied stakeholders. The author believes that agility has fulfilled its promise. Many projects have obtained successful outcomes with agile implementations. On the other hand, there are some organizations that are very cautious about forging ahead with agile. This guarded approach is based upon the acknowledgment that the agile framework is not appropriate for every project. This chapter identifies those types of projects that are suitable for agile and those that are not. We also provide simple techniques that can be used to make formal assessments when contemplating the move to agile methods.
AGILE VERSUS WATERFALL
How does an organization know when to select agile rather than the waterfall model for software development? The answer is not always simple. According to Ambadapudi Sridhara Murthy (2013), there are alternatives to waterfall and agile. Often a combination of both methods will yield the desired models as outlined below:
• Agile in Waterfall: The waterfall method does not typically allow for an immediate return on investment (ROI) and results are generally not seen until after the project has been completed. Once the product is finally made available, sometimes results are not as expected. The costs required to correct defects late in the life cycle are quite expensive when compared to fixing these issues earlier. Some organizations are not completely satisfied with the waterfall model because they want to see the result of the product more frequently, just like with the agile model. The solution would be to mix the waterfall method with agile and here’s how this occurs. The project would begin with the waterfall method and then the requirements broken into components during the traditional schedule work breakdown process. Next, individual features would be broken down and developed with the agile method. Table 15.1 provides an example of how agile can be integrated with the waterfall methodology.
TABLE 15.1
Agile in Waterfall Agile in Waterfall
• As-Needed Hybrid Model: This model represents a combination of waterfall, agile, rapid application development (RAD), Extreme (XP), and so on. The idea behind the hybrid model is that an organization can continue to use their usual models and then gradually begin to adapt and move to the new model(s). A particular model is selected based upon its perceived execution speed, low risk ranking, and the expected quality level of the delivered product. It must be clear that although an organization makes plans to use the hybrid model, it is recommended to continue to use the methods they have become accustomed to and then slowly add agile into the mix along with other methods as appropriate. Table 15.2 depicts the structure of the hybrid model.
TABLE 15.2
Hybrid Model
• Waterfall within Agile: There are some organizations that criticize agile methods for lack of documentation and the quality of the product. Say what? Some folks feel that agile projects are quality and documentation deficient and that this would leave the organization in a place where they would be unable to take the proper corrective actions at later times. The remedy for this perceived dilemma can be rectified by the usage of the waterfall model during the iterations. See Table 15.3 for a description of this adapted model.
As most of us are aware, organizations are under tremendous pressure to deliver projects faster within budget and scope. In addition, the expectation is to provide all of the functionality that a customer desires within the smallest number of releases possible. As a result of projects with compressed time frames, companies feel that they are required to forge ahead with agile methods without doing proper evaluations to determine whether this process is a best fit. It must be clear that agile is not a fit for every project and it is the responsibility of an organization to conduct the proper analysis prior to the selection of this method.
TABLE 15.3
Waterfall in Agile
AGILE NOT A FIT
According to Barrington Rowe (2014), agile is not a proper solution for the following types of projects:
• A project that requires changes to complex code that has been written with unstructured programming practices is not a good agile candidate. This would be the case of older legacy systems with tightly coupled code such as functions or classes that are very dependent upon each other and require a lot of effort to modify. In contrast, code that is loosely coupled is not dependent upon other code.
• A potential project where the release management process is very costly. In addition, the time and effort required for a production deployment is extreme. As a result of cost concerns an organization is led to believe that a single release is the best option. However, limiting a product to a single release because of cost concerns is not practical.
• A project where the team lacks experience and skills required for complex user stories. This would mean that some user stories would need to be implemented over multiple iterations. The recommendation is that a user story be completed within a single iteration. In addition, a skills gap is a real risk to any project.
• A project where the risk of having a third-party vendor as a major stakeholder is high. This is the case of the vendor having responsibility for providing products or services that are critical. If a vendor does not deliver as required, a dreadful scheduling mishap would have the potential to affect the project negatively. Another risk is user stories that depend upon this type of vendor and they are delayed until later iterations: a definite no-no.
• A project that has user stories that are inadequate for whatever reason (untestable, no value, not estimatable, independent, or non-negotiable). This would be a huge risk to the project.
Furthermore, an organization is advised to use decision-making tools and techniques as discussed in this chapter to determine which projects are proper candidates for agile implementation. Just as a project is evaluated for its potential to be chosen carefully over others, this same concept should be applied to determine if a project is an “agile candidate.” During the analysis process of determining which projects are good candidates for agility, a determination should be made as to what changes need to take place so that impediments can be removed. Once changes are made, the project can move forward to being categorized as “agile appropriate.”
With regard to changes that may need to be made, organizations need to consider whether their existing IT unit is grounded on the waterfall lifecycle model. An organization should never attempt to place traditional IT personnel in the agile environment without proper training and expect an automatic success story. It would be a drastic change for these workers to become cross-functional without guidance and understanding of the agile requirements. It is well known that organizational changes are often prolonged and painful for many companies. Change is not easy for everyone to accept. We must understand that agile implementation requires a cultural change within an organization that should not be undertaken lightly.
AGILE FIT OR MISFIT?
Many organizations are not able to comprehend fully some of the challenges that occur during their transition to the agile framework. Most are attempting to make a direct change from the waterfall to agile methods. There are other situations where agile is a good fit and other circumstances where the method is considered to be a misfit. We now describe the best project type for agile methods.
Murthy (2013) provides a very specific circumstance for agile success: web-enabled technologies. He goes on to restate that agile methods support changing requirements throughout a project. The fact that defects are revealed during continuous iterations means that there is never a wait to review these types of discrepancies. Changes and fixes are included in the code during the next iteration after being discovered. This ultimately results in a reduction in costs that are associated with resolving defects. In addition, there is no impact on the product’s delivery date with this approach.
Murthy (2013) further states that agile is a dynamic model and when used properly typically results in a success story. In spite of the fact that most agile projects are undoubtedly successful, there are occasions where agile projects sometimes fail. The question is why? The answer is quite simple: if an agile project is not implemented properly, the end result is a “sunken” project that undoubtedly would need to be redone. Ouch! Murthy (2013) goes on to describe several reasons why agile projects fail:
• People—attitude and beliefs: In very simple terms, team members with the “me” attitude are a contributing factor to failure on the agile project. The attitude should always be “we” because the entire team is equally responsible for the success or failure of the agile project. Being motivated as a single entity is a driving force behind successful product delivery and the attainment of the project’s goals.
• Team expertise, servant leader: It is critical that the agile team is established with the right level of expertise. Inexperienced developers are not appropriate for this type of team. This team needs enthusiasm, exceptional skills, and self-motivation. Concerning the servant leader (a ScrumMaster, e.g.), this role needs to understand how to serve the agile team as well as provide leadership.
• Product owner—understanding of role and responsibilities: A major success factor for the agile project is the Product Owner’s role and the level of understanding required in terms of engaging with stakeholders and managing the product backlog. If this role is not able to provide clarification for the user stories so that the development team can build the product, the agile team will undoubtedly experience failure and frustration.
• Documentation: Murthy (2013) asserts that there are critical documentation areas with the potential to harm an agile project in the long term. In cases where a developer(s) might leave the project, then all of the information could be lost if there is no documentation to pass on to new team members. We know that there is some documentation that is required on the agile project such as user stories and design artifacts. What if team members on the agile project have the false belief that no documentation is ever required? The author believes that this misconception would be disastrous.
• Schedule, budgets, and contracts: According to Griffiths (2012), agile projects typically work under fixed time and costs. Scope has to be dynamic and can never be static in this case. We must understand that whenever scope is static, there is a risk that time or money can be exhausted prior to the product being delivered. This can also increase the probability that the product will be of insufficient quality. As a result of frequent and unplanned changes, an increase in scope would cause an upsurge in the budget and time frame for the agile project. This would mean that a traditional fixed price contract should never be used. This is a major factor of agile project failure because some organizations are not clear on the implementation and have frequently utilized the incorrect contract type. A fixed price contract is not flexible enough for agile methods because the team would be limited by the budget. This constraint has the potential for the team to lose sight of delivering a quality product.
LIMITATIONS OF AGILE SOFTWARE PROCESSES*
According to Turk, France, and Rumpe (2002), there are definite limitations associated with some of the agile software processes. Notice that we have referred only to the processes and not the actual agile methods. This distinction is clearly important and Turk et al. (2002) acknowledge that agile methods differ in their values, practices, and application domains. As a result, it is not an easy task to evaluate all of the agile processes in a general sense and apply these limitations across the board. This study and analysis originated using several XP projects as a source. The assumptions were made from the following agile methods:
• Extreme Programming (XP)
• Scrum
• Agile Unified Process
• Agile Modeling
• Agile Alliance Principles
Following is a list of the limitations that were uncovered by Turk et al. (2002) for agile software processes:
• Limited provision for dispersed development environment: Agile methods support colocated environments. This can be alleged as a hindrance to those organizations desiring to develop software in globally dispersed environments. An alternative solution is video conferencing, however, this can be very expensive depending upon the number of people involved.
• Restricted support for subcontracting: The subcontracting process requires that requirements in the contract be clearly defined in adequate detail so that costs can be accurately estimated. Agile methods are adaptive and are not predictive enough for some subcontracting purposes. In Chapter 14 we have spoken of the need to have agile contracts on agile projects. Turk et al. (2002) recommend a two-part contract: fixed and variable.
• Fixed Part: This is the portion that outlines the framework limiting how the subcontractor includes changes into the product. It also includes what the subcontractor must do, and what must be delivered.
• Variable Part: This is the portion that outlines the requirements and deliverables that will change within the boundaries of the fixed part of the contract. Variation is expected but within the boundaries of what has been established as fixed. Prioritized items should be included in this section during the time the contract is executed.
• Limited support for the construction of recyclable items: Agile methods are concentrated on building products for specific solutions that are not generic enough to be used on other projects. In the past, it was very desirable for software components to be developed that could be used in other software products. There is no clear understanding as to how agile processes could be adapted for recyclable software components when the goal is to develop a certain product for a specific customer.
• Limited development support when using large teams: The agile process supports small-to medium-sized teams rather than large teams. Communication becomes less effective with a large-sized team as a result of face-to-face practices. Large teams need less informal agile and more formal processes to tackle bigger projects. The level of agility has the potential to be smaller in large-team environments and bigger in smaller team environments.
• Limited support for building safety-critical software: According to Turk et al. (2002), in the case of safety-critical software where failure would cause the loss of human life or catastrophic economic repercussions, the agile process does not have adequate quality control processes in place that could provide assurances of protection from disaster. Critical components of software should be developed under more formal software development methods.
• Limited support for building large software systems: Code refactoring can be very expensive for large systems. Some systems have components that are not very easy to modify because they are critical to the essential services that are offered. Some systems have functionality that is so tightly coupled and intertwined that it may be impossible to develop software incrementally. Code that is developed using an iterative methodology may be fit for use, however, when used in each iteration, all of the components will be in dissimilar and incomplete stages.
AGILE CHALLENGES*
We now focus our attention on agile challenges: perceived, imagined, or real. Gandomani et al. (2013) have identified several aspects of agile methods that could be viewed as challenges which need to be overcome prior to moving to agile methods. The author would like readers to be cognizant of the fact that a subjective analysis has been provided for the ratings of each challenge (perceived, imagined, or real). Other experts may have different views on these areas of concern.
People.
Senior-level developers have a particular mind-set that is often perceived as being difficult or at times impossible to change (perceived and real).
Difficulties with implementing pair programming in XP with senior-level developers (real).
Code ownership is a challenge from the developer’s viewpoint when compared to other types of teams (perceived).
Overly ambitious people have been known to alter a customer’s priorities that make the transition to agile more difficult (perceived and real).
Cultural differences can be barriers when transitioning to agile methods (real).
Communication complications within some organizations could affect the move to agile methods (real).
Management.
A self-organizing team is sometimes difficult to establish for some organizations (both real and perceived).
Lack of solid agile change management strategy (perceived).
Agile team roles may be very difficult for some companies to fill (real and/or imagined).
Some project managers who have backgrounds in traditional software development are not able to let go of former roles and responsibilities easily and may find the transition to the servant leader role “uncomfortable” (real).
There are some who believe that a perfect customer is a fairy tale. The customer is required to take on a servant leader role and must drive the agile process by taking responsibility for the prioritized product backlog. Some feel that this is a stretch to get a customer to move into this role easily (perceived).
Lack of qualified agile coaches can be a challenge (real).
From a management point of view, little or no documentation can be a hindrance to project success (perceived).
Team decision making may be flawed as a result of:
Lack of readiness to commit to decisions because of prior reliance on the project manager (real).
Conflicting priorities (real).
Unstable resource obtainability (real).
Lack of implementation decisions by team members (real). Unwillingness to take ownership (real).
Lack of team empowerment by blocking experts from making decisions (perceived).
Team decision making with regard to resource provision, strategic product line positioning, development, and maintenance work requirements can be obstacles during the adoption of agile methods (real).
Process.
Moving from a traditional model to an iterative model can be challenging for some organizations. This transition can be a genuine obstacle because this is a big change that has a huge impact on business strategies, tools, methods, and roles (real).
Colocated teams can be an obstacle for distributed organizations (real).
Different ways of measurement in agile and traditional practices can be perceived as issues for anyone prejudiced against traditional processes (perceived).
Technology.
Differences in the degrees of agility are perceived to be unequal across the agile methods (real).
Agile methods focus on either software development or project management (real).
Lack of knowledge regarding the capabilities of specific agile methods (real).
TABLE 15.4
Agile Suitability Construct
IS AGILE SUITABLE?
Van Dijk (2011) acknowledges that there is only a small amount of theoretical information available to assist practitioners during their analysis and selection process for an appropriate agile methodology. An assertion was made that an appropriate framework is not one that should be modified to fit all circumstances. The right framework should be flexible in such a way that it is inclusive of various points of view that have been established upon the existing methods as required. In order to actually determine if a particular method is right, the framework would need to include:
• The project characteristics
• Agile characteristics
• The relationship between the project and agile characteristics
• A technique that can be used to evaluate these characteristics (the project and agile) so that a decision can be made
In order to accomplish the objectives listed above, the following suitability construct has been established and recommended by Van Dijk (2011) as identified in Table 15.4.
Agile Suitability Construct
Let’s begin our discussion of the suitability construct so that the proper understanding is obtained as to how we utilize it for actual projects. According to Van Dijk (2011), whenever we examine the available agile methods that are currently available, we must consider the following contingency factors:
• The capabilities of agile methods are expressed in terms of limitations.
• The capabilities of organizations are expressed in terms of culture and personnel capabilities (Table 15.4).
The suitability construct can be used to make a decision by conducting an evaluation of the environment for the project by providing answers to the following questions (Van Dijk, 2011):
What are the limitations that apply to the situation? Van Dijk refers to the same limitations that have been identified by Turk et al. (2002) in the section “Limitations of Agile Software Processes” in this chapter.
Is the organizational culture a hierarchical one? Van Dijk asserts that a hierarchical organizational culture does not favor agile methods. Descriptions of his four cultural types are given below:
• Group culture: Transformation and internal focus.
• Development culture: Transformation and external factors.
• Rational culture: Constancy and external focus.
• Hierarchical culture: Constancy and internal focus.
Does the organization presently use or has it had any past experience with agile components?
• An information system development (ISD) method component is agile if it contributes to one or more of the following:
– Creates change.
– Acts in anticipation of change.
– Reacts to change.
– Learns from change.
• To be considered agile, an ISD method component must make a contribution to one or more of the following and must not take away from:
– Alleged economy.
– Alleged simplicity.
– Alleged quality.
• To be considered agile, an ISD method component must be continuously ready to prepare the component for usage (i.e., being ready refers to minimum time and costs).
Is the capability of the organization’s personnel suitable (Van Dijk, 2011)? Following are descriptions and categories that identify the skill levels of an organization’s personnel.
• Level 3: Capability to modify a method (modifies the guidelines in order to apply the method to a new situation).
• Level 2: Capability to modify a method to fit a new situation.
• Level 1A: Capability to conduct optional method steps with training (estimate user stories to fit iterations, refactoring, etc.); with some experience, has the potential to be Level 2.
• Level 1B: Capability to conduct routine method steps with training (coding, simple refactoring, test execution, configuration management, etc.); with experience can perform some Level 1A skills.
• Level 1: Potential technical skills but unwilling to collaborate or adhere to shared methods.
Furthermore, Van Dijk (2011) states that a particular agile method is only superior to another when there are requirements to be agile and an organization has the capability to be agile. As we have previously stated in this chapter and Van Dijk concurs: organizations need to conduct proper assessments of their capabilities and prerequisites prior to the selection of an agile method.
AGILE IMPLEMENTATION: RISKS AND ISSUES
According to Veneziano, Rainer, and Haider (2014), there are several risks (or issues) that should be addressed prior to making a decision to “go agile” for a particular project. After examination of the items, the question would then be: “How could this potential risk factor affect our project?” Although the list does not include all possible factors, the items in Table 15.5 should be seriously considered. As previously stated in this chapter, an organization does have the option to make changes and remove impediments so that a project can be moved forward to the agile list. The reader should observe that the limitations asserted by Turk et al. (2002) are once again in force as shown in Table 15.5.
Agile Adoption Decision Model
Veneziano et al. (2014) implement an agile decision-making model (WAINGE) that uses each issue/risk factor (r), that has been weighted against a suggested value (w)* that makes a contribution to the calculation of the overall risk ranking for the implementation of an agile method. The overall specific risk (OSR) is a value between 0 and 1 and it can be defined as the sum of the (n) risk factors associated with each issue from the critical items list (again please refer to Table 15.5). The result should be a number that can be used to make a decision on the overall risk ranking for adopting agile for a particular project. The author of this book was able to obtain the actual WAINGE model and found it very useful and easy to comprehend. We now provide more detail regarding the agile usability model.
TABLE 15.5
WAINGE Agile Decision Maker v1.8
WAINGE AGILE DECISION MAKER v. 1.8 |
|||
Item # |
Risk Factor (r) |
Description |
Weights (w) |
1 |
1 |
Customer cannot be available for the entire duration of the development process. |
0.8 |
2 |
1 |
An incomplete user acceptance test is highly probable. |
0.8 |
3 |
1 |
An unreliable estimate on time and budget is expected. |
0.7 |
4 |
1 |
Documentation is not thoroughly developed and not a critical asset. |
0.8 |
5 |
1 |
Some agile methodologies and practices have the potential to be less flexible than anticipated by an overall agile approach. |
0.6 |
6 |
1 |
Lack of experience and the relaxed discipline of agile have the potential to affect project management negatively. |
0.5 |
7 |
1 |
Lack of support for distributed development environments has the potential to occur. |
0.0 |
8 |
1 |
Outsourcing or subcontracting will probably be more problematic to accomplish. |
0.0 |
9 |
1 |
Support and opportunities for developing reusable objects is limited. |
1.0 |
10 |
1 |
Limited or unsustainable support for development with large teams. |
0.0 |
11 |
1 |
Limited support for developing safety-critical software is expected to occur. |
0.5 |
12 |
1 |
Limited support for developing large multifarious software is expected to occur. |
0.8 |
13 |
1 |
Insufficient focus on architecture has the potential to produce less than desired design decisions. |
1.0 |
14 |
1 |
As a result of the social values that agile embraces, there is a potential for decision making to be less than desired. |
0.6 |
1 |
High turnover is expected for agile developers and consultants. |
0.9 |
|
16 |
1 |
There is a perception that agile methods only wrestle with trivial components of a project and leave the complex ones unaddressed. |
0.8 |
17 |
1 |
The agile development process and results have the potential to rely on the quality of the people that have been hired. |
0.7 |
18 |
1 |
There is a risk that adaptive planning could be interpreted as lack of long-term planning. |
0.3 |
19 |
1 |
Planning poker, (a variation of wideband Delphi) and other decision-making approaches have the potential to be affected by unwilling team members who could be unfocused on actual development. |
0.8 |
AVA |
0.4 |
Explanation of Calculations |
DEC |
0.576583616 |
||
Note: AVA value is an input value and is not calculated. |
No. of Issues |
19 |
Input Value only (no calculations) |
The final answer is .57 which rounds to 60%. Recall that any number greater than 50% is considered to be a high-risk agile project. |
||
(∑r)/n |
0.610526316 |
= SUM (all weights (w) divided by the total number of weights (19)) |
||||
Δ = (1 – ∑r/n) |
0.389473684 |
= 1 – 0.610526316 |
||||
MAF |
–0.0339427 |
= LOG((0.5 + AVA)/(1.5 – AVA)) * MIN(ABS(0.610526316), ABS(0.389473684)) |
DEC = MAF + 0.610526316 |
|||
Source: Derived from V. Veneziano, A. W. Rainer, and S. Haider, When Agile Is Not Good Enough: An Initial Attempt at Understanding How to Make the Right Decision. University of Hertfordshire: School of Computer Science, pp. 1–10, 2014. With permission.
WAINGE Model
This model has been developed in MS Excel and, according to Veneziano et al. (2014), can be requested directly from the creator. The name WAINGE is derived from the title of their paper, “When Agile Is Not Good Enough: An Initial Attempt at Understanding How to Make the Right Decision.” The model’s usage starts with the input of the AVA value.
Attitude Value toward Agile (AVA)
The attitude value toward agile (AVA) is to be measured against an entire development team; the values obtained are between 0 and 1 where:
• 0 = An extreme agile critic
• 0.5 = An ideal neutral view
• 1 = An extreme agile supporter
Risk Factors for Agile Adoption
For usability purposes, and during the research, the risk factor (r) value for the adoption of agile on projects was always assumed to be 1 (Veneziano et al., 2014). In addition, weights (w) were assigned for all 19 risk factors and are based upon the following:
• 1 = Very high impact
• 0 = Very low impact
Mitigation Amplification Factor (MAF)
We now describe the mitigation amplification factor (MAF) as the value that represents the team’s attitude (as a myth) and is used to standardize the decisional value that has been derived from the overall specific risk. The question used to obtain the MAF is, “Do you describe yourself and your team as agile supporters?” The resultant answer should be between:
• 1 = Absolutely yes
The response provided from the MAF above is used to calibrate the decisional tool against the attitudes of stakeholders. It must be understood that the researchers somewhat transformed the curve (i.e., inverse hyperbolic tangent) in order to better accommodate the values of the MAF to fall within the expected ranges of 0 to 1 for the final DEC value (Veneziano et al., 2014).
Final Decisional Value
The final decisional value (DEC) will always be a value between 0 and 1, which can be converted into a percentage. A DEC > 0.5 represents the threshold that would indicate that the adoption of any agile method is very risky; however, this premise should be further authenticated by observable evidence. In addition, the DEC is calculated by the sum of the overall risk plus the mitigation factor depending upon whether the team’s attitude is for or against agile (Veneziano et al., 2014). Once again, the reader is encouraged to study Table 15.5 in order to gain an understanding of how the WAINGE model is used to provide an agile ready decision. Better yet and as previously mentioned, the MS Excel spreadsheet that contains the WAINGE model can be requested from its creators. Readers are encouraged to make this request for complete analysis. The model’s implementation is very straightforward.
CHAPTER SUMMARY
We have identified two methods of determining whether a project is an agile candidate. Our intent was to provide relatively simple tools and techniques that can be used on actual projects. Even though tools are helpful, it is up to the organization to decide how and when to move forward with the agile methods. We hope that readers have found the information useful.