Being Agile

Let’s discuss the process of developing software.

People often think that building software is like building anything else, e.g., a car. At first glance is seems reasonable: you determine some requirements for what you want to build, design it, and then build a prototype. You refine your production process, and eventually you can mass-produce as many of these cars as you want.

However, if we attempt to apply these concepts to software, they don’t work. Designing and building software is subtly different than building other products.

Software Development Lifecycle

The Software Development Lifecycle (SDLC) is a simple model that we use to describe the steps of a software project, from inception to a final working product.

The model has evolved over the last few decades, and is similar to other project tracking processes. It usually prescribes a series of activities like these:

• Planning - “What are high level goals?”, “What is our budget?”, “Who is working on the project?”
• Requirements definition - “Who are our users?”, “What problem are we solving?”
• Analysis & design - “What technical constraints exist?”
• Implementation - “How do we build it?”
• Testing - “Does it meet specifications?”
• Deployment - “How do we keep it running properly?”

Formally, we consider the SDLC to be a software process model: a collection of activities, actions and tasks that we perform when creating software.

This particular model is also referred to as a waterfall model, named for the way in which you are supposed to move through each activity in sequence.

---
title: Waterfall Model
config:
    theme: neutral
---
flowchart LR
    A[Planning] --> B[Requirements]
    B[Requirements] --> C[Analysis & Design]
    C[Analysis & Design] --> D[Implementation]
    D[Implementation] --> E[Testing]
    E[Testing] --> F[Deployment]

The implications of using a waterfall model:

• A project starts at the first activity and advances through stages. Each stage must be completed before the next stage begins.
• Stages are typically modelled after organizational units that are responsible for that particular stage (e.g. Product Management owns Requirements, Architects own Analysis & Design, QA owns Testing and so on).
• There are criteria that need to be met before the project can exit one stage and enter the subsequent stage. This can be informal (e.g. an email letting everyone know that the design is “finished”), to a more formal handoff that includes artifacts (e.g. Product Requirements documents, Design documents, Test Plans and so on that must be formally approved).

This linear approach strongly suggests that you can and should define a project up-front (i.e. determine cost, time and so on before starting the project).

What’s wrong with this model?

If you’re a project manager, or a business owner, the Waterfall model is very appealing: it suggests that you can define work up-front, and plan out time, costs and other factors. This model suggests that you can work out the parameters of a project, then execute on that plan. Any failure to meet a deadline would be a failure of people to complete their stages ‘on-time’.

However, this isn’t realistic, at least not for software projects.

1. Requirements can and will change as project development is underway. Often the context in which the product is being developed will change in ways we cannot anticipate (e.g., the market changes, so new features need to be added). Our understanding of the problem that we’re solving will also naturally evolve as we’re building something, which can result in unexpected requirements changes as well.
2. Design and implementation activities are not that predicable! We are often building new features or combinations of features. Managing novelty is always going to be challenging and include some level of uncertainty (i.e., how long it will take, and what the solution will look like).

We knew in the 1970s that the Waterfall model didn’t work, and spent 20 years trying to come up with a better solution. By the mid-1990s, there was a widespread recognition that this way of building software just didn’t work – either for developers or for customers.

• Developers were frustrated by rigid processes and changing requirements.
• Business owners were frustrated by the inability to make changes to projects in-flight.
• Projects were being delivered late and/or over-budget. This frustrated everyone!

There are a large number of software process models that were developed at this time, including Extreme Programming (XP) (Beck 1999), Scrum (Schwaber & Sutherland 1995), Lean ( Poppendieck & Poppendieck 2003). No single model dominated.

Instead, the frustrating state of software development led to a a group of software developers, writers, and consultants meeting and delivering the Manifesto for Agile Software Development in 2001.

Agile was an attempt to find some common-ground principles across competing process models. It isn’t a process model by itself, but a set of principles that influenced the development of other models.

The Agile Manifesto

The Agile Manifesto (agilemanifesto.org)

What does it mean?

1. Individuals and interactions (over processes and tools): Emphasis on communication with the user and other stakeholders.
2. Working software (over comprehensive documentation): Deliver small working iterations of functionality, get feedback and revise based on feedback. You will NOT get it right the first time.
3. Customer collaboration (over contract negotiation): Software is a collaboration between you and your stakeholders. Plan on meeting and reviewing progress frequently. This allows you to be responsive and correct your course early.
4. Responding to change (over following a plan): Software systems live past the point where you think you’re finished. Customer requirements will change as the business changes.

Agile Software Development is used to describe any process that encompasses this philosophy. Agile encourages team structures and attitudes that make communication easier (among team members, business people, and between software engineers and their managers). It emphasizes rapid delivery of operational software, but also recognizes that planning has its limits and that a project plan must be flexible.

Iterative processes

This idea of an iterative, evolutionary development model remains central to all Agile processes (although they may present it differently). Instead of building a “complete” system and then asking for feedback, we instead attempt to deliver features in small increments, in a way that we can solicit feedback continuously though the process. Over time, we will add more features, until we reach a point where we have delivered sufficient functionality and value for the customer.

Agility forces us to continually reassess the state of the project, and supports the idea that decisions can unfold over time. The most significant project benefit of Agile is that it reduces (and tries to eliminate) breaking late-project changes.

Being Agile

Although we often refer to Agile as a singular model, it is actually an umbrella: a philosophy that drives a number of other models, such as Kanban, Lean and Scrum. In practice, we often find that software teams adopt practices from different methodologies, changing them where it makes sense.

We’ll do the same thing here. We’ll reconstruct our SDLC to be a two-stage iterative model.

We have 2 distinct phases:

• An early planning and design phase, where we do basic project planning, define our users, and their requirements. This is also where we propose early solutions and get early feedback to narrow down our decisions. We’ll discuss this further under Design Thinking.
• A second development phase, where we deliver the functionality that we proposed in the first phase, to meet our user goals. We will spend the remainder of the course discussing topics for this phase.