Shift your software delivery methodology into high gear with Agile

Yet, software development projects continue to fail at an alarming rate. The Standish Group’s 2011 CHAOS Report found that more than 50 percent of all software development projects conducted between 2002 and 2010 were classified either as challenged or as complete failures, with only 37 percent classified as successful. It’s no surprise then, that software development is often viewed as a risky and costly venture.

The root cause of this problem is in the use of predictive software development processes in complex, unpredictable and sometimes even chaotic business environments.  The majority of projects today are still using the traditional waterfall planning methodology, which is best suited for predictable, repeatable work. Years of project work is developed based on assumptions and past experiences. The reductionism approach, which is often used to explain and predict a system’s behavior by reducing it to the interactions of its parts, or to simpler rudimentary units, is still practiced in many classical hierarchical team structures and management techniques..  These types of traditional project planning and management methods are becoming less effective and increasingly obsolete in today’s fast-paced business setting.

We see several problems with the assumption-based and reductionism approaches practiced in traditional waterfall software development:

• Waste of money
  

The traditional “waterfall” software development methodology calls for the entire system design upfront.  It is based on the assumption that one knows the entire system functionality and can plan for it at once. However, that is hardly possible in today’s world of rapidly changing priorities, trends and context.  In addition, traditional change management is too cumbersome and inert to address changes mid-cycle.  As a result, companies are often left with engagements that focus on the project plan execution, rather than developing functionality according to the latest business priorities and feedback from early system users.  This, in turn, leads to budgets being spent on features  that are rarely if ever needed by the end users.

• High costs of change
  

Complex project plans, loads of upfront documentation, burdensome document-based review and approval processes, and upfront architecture design; all these parts of traditional waterfall development-based projects result in an enormous effort to release a system (manual build/deployment procedures) and ensure its quality (longer test cycles due to manual software testing ). As a result, CIOs are frequently left with long, inflexible and costly change cycles.

• Wrong features prioritization
  

With an overwhelming planning stage, traditional software development models don’t account for the software demonstration to end-users until three to six months into the project. Additionally, engineers often tend to start the development process with its more appealing technical parts, such as architecture, frameworks, base classes, and libraries, steering away from core business features development.   As a result, business users are frequently unable to see the functioning system prototype until six months into the project, which is far too long for those to whom time-to-market is of utmost importance.  Lack of timely evaluation and feedback from end users affects the software quality as well. Without the correct understanding of all the system’s viable features and priorities, there is a greater chance that its architecture might not be developed correctly and that any changes will be more difficult and costlier to implement.
 

• Lack of project transparency
  

At first glance, the waterfall methodology reporting structure seems to be a well-defined process.  Numerous status reports and progress metrics promise full transparency and control over the project.  However, this first impression is often a deceiving one.  In most cases, real project status is hindered by unnecessary paperwork and overly formal process-oriented relationships.  Being unable to physically “go see” the system, try its features, and provide feedback regularly may add to the frustration.

• High system complexity and batch delays
  

Long-term, up-front planning and high cost of change create numerous dependencies, making it impossible to test the system quickly. This, in turn, increases the system’s complexity and leads to batch delays.

The holistic approach and systemic thinking that make up the foundation of Agile project management are gaining additional traction as they are quickly becoming main drivers of successful and predictable software development initiatives. More companies are turning to Agile development because it proactively addresses core reasons for project failures, delivering greater transparency and flexibility. More importantly, Agile avoids most of the assumptions of the waterfall model, helping companies better manage risks, increase flexibility, react faster to business opportunities, and stay ahead of the competition by meeting the ever-changing needs of their customers.

The Agile method of development has several winning practices that directly  address flaws of the waterfall methodology:

• Business value driven prioritization
  

In Agile projects, system functionality is prioritized and delivered according to business value, enabling faster realization of desired benefits.

• Paying for “DONE”
  

Developed functionality is demonstrated to the client through regular delivery iterations (every 15 – 30 days) and is accepted only if it meets the definition of “DONE “.
   

• Change without penalty
  

The built-in change management approach of Agile projects provides the framework for fast and objective impact assessment and free-of-charge change of the control process due to the naturally imbedded flexibility of the development process.

• Full control of the project
  

Operational transparency combined with the iteration-based delivery feature guarantee a fully functional system at all times during the Agile engagement. This helps keep the client in full control of the project and enables stakeholders to painlessly change project scope and delivery milestones at any given time.  It even allows for money-saving early project termination, should market conditions warrant.

• Reduced delivery risks
  

Frequent delivery cycles and reduced batch complexity combined with an open atmosphere of trust and collaboration between Agile project teams  enables easier identification and elimination of project risks . According to the Standish Group’s 2011 CHAOS Report, Agile projects are three times more successful then waterfall (42% vs.14% success rate).

The transition to Agile/Lean Software Development is not a trend; it is a direct response to changing business realities. Just as the automobile engine largely displaced horsepower, Agile software development will soon become the industry standard and a software development methodology of choice for the majority of businesses around the world.  Nonetheless, small niches will continue to exist where waterfall development is applicable, just as  horsepower is still preferred over automobiles in certain conditions today.

Agile methodology or Lean methodology is currently undergoing a similar transition to that  from “steam engine” to  “first automobile”:  Although there  may not exist comfortable paved roads for Agile developers right now, in the next five years, this methodology will become the standard “Ford” and waterfall will go the way of the horse and buggy.

Transitioning to Agile « being Lean» involves changing people’s mindset because it’s a complex initiative capable of taking years to implement. That’s the reason many companies prefer to execute Agile projects by partnering  with a specialized third-party service provider in order to speed up time-to-market and to deliver high-quality innovative software that meets customer demand.