# Process Models

People like to think that building software is like building anything else, e.g., a car. At first glance is seems reasonable: you determine some requirements for what you want to build, design it, and then build a prototype. You refine your production process, and eventually you can mass-produce many of those objects. Profit!

Except it's not quite like that. Software is unique:

• It's intangible and abstract. There is no physical product.
• Software is highly customizable. You can design software to do practically anything (that is possible within it's limited domain).
• Software may be expensive to develop, but it's relatively cheap to distribute since distribution can often be done digitally i.e. with no physical copies required.
• Once "built", we can make as many copies as we wish at little to no additional cost. You can't do this with a tangible product! (The next-closest thing might be a book or music or some expression).
• Software doesn't "wear out" like a physical product but it requires ongoing maintenance as new features are introduced, or the operating environment changes e.g. new OS versions being released, driver updates.

Software is also highly dependent on the environment in which it operates. Yes you can "do anything" in a software system, provided that the underlying hardware/operating system/toolchain supports it.

A software process is a collection of activities, actions, and tasks that we perform when creating software. An activity is used to achieve a broad objective (e.g., communication with stakeholders) and is applied during the process. Actions (e.g., architectural design) encompass a specific set of tasks that may be used for a particular activity (e.g., an architectural model).

# Project activities

We can identify activities that are applicable to all projects, regardless of their size or complexity. A generic process framework for software engineering encompasses five activities:

• Communication: We need to define the problem that we're trying to solve, and discuss the goals and requirements with the customer, potential users, and other stakeholders. This is critical to ensure that we're building the "right" product or solving the main problem that needs to be addressed.

• Planning: This involves defining the tasks that need to be completed, and high-level milestones that you need to reach. It also identifies people and resources, potential risks to the project, and ways of mitigating these risks. These are what we would typically call "project management" activities.

• Modeling: It's always cheaper to think ahead prior to actually building something. This step includes design efforts to create abstract models or representations of the product that you need to build. By iterating on a design, you refine your understanding of the problem, and move towards a more-correct solution. This is crucial before you actually start building it, both to improve the accuracy and usefulness of what you build, but to minimize costs.

• Construction: The process of building your product i.e. realizing your design. This may require successive iterations to complete all the required features, and should also include some level of testing and validation.

• Deployment: Tasks required to produce a working product and deliver it to the customer. This includes collecting their ongoing feedback and making revisions as needed.

The people impacted by your product are called stakeholders. This includes the people who will use your solution, but can also include anyone else that is affected by it. For example, in a software project, stakeholders can include users, the IT department who will install and maintain your software, people who rely on the output of your program and so on.

# Process models

We use the term process model to describe the structure that is given to these project activities.

A software process model is simply a type of process model adapted to describe for software systems. It defines the complete set of activities that are required to specify, design, develop, test and deploy a system, and describes how they fit together.

Software activities tend to be named slightly differently than the activity names that we've been using:

Compared to the standard model, we've split Planning into separate Planning and Requirements definition. These activities are often performed by different departments or individuals, so traditionally they're split apart. Analysis & Design corresponds to Modelling, and Implementation and Testing together correspond to Construction.

This is a simplified understanding of a software development project. People generally accept that we have planning, formal requirements, modelling and design, implementation and testing—but there are lots of disagreements on how these pieces "fit together." Let's continue talking about the structure of software projects.

# Waterfall model

In the 1970s, there was a concerted effort to formalize 'known-good' methods of project management. Software projects were seen as expensive and time-consuming, and there was considerable pressure to improve how they were managed. In a 1970 paper, Winston Royce laid out a mechanism for formalizing the large-scale management of software projects [Royce 1970], dubbed the Waterfall model. This envisions software production as a series of steps, each cascading into the next one, much like a waterfall. In this model, requirements are defined first, a design is created and then implemented, then tested and so on. Software development is treated as a set of linear steps that are followed strictly in-order.

The Waterfall Model, as understood and practiced for a long time, most closely resembles a linear project model, and is similar to how other construction or manufacturing projects are organized.

The Waterfall model the following characteristics:

• A project starts at the top and advances through stages. Each stage must be completed before the next stage begins.
• The stages are modelled after organizational units that are responsible for that particular stage (e.g. Product Management owns Requirements, Architects own Analysis & Design, QA owns Testing and so on).
• There are criteria that need to be met before the project can exit one stage and enter the subsequent stage. This can be informal (e.g. an email letting everyone know that the design is "finished"), to a more formal handoff that includes artifacts (e.g. Product Requirements documents, Design documents, Test Plans and so on).

This linear approach strongly suggests that you can and should define a project up-front (i.e. determine cost, time and so on).

# What's wrong with this model?

If you're a project manager, or a business owner, the Waterfall model is very appealing: it suggests that you can define work up-front, and plan out time, costs and other factors. This models suggests that you can work out the parameters of a project, plan and execute it.

However, this isn't realistic.

1. Requirements can and will change as project development is underway. This is partly because the context in which the product is being developed will also change, often in ways that we cannot anticipate (e.g., the market changes, so new features become important; the customer needs to pivot on early decisions). Our understanding of the problem that we're solving will also change as we're building something, which can result in unexpected requirements changes as well.
2. Implementation and related activities are not that predicable! We are often building new features or combinations of features; building something novel is always going to be challenging and have some levels of uncertainty (in how long it will take, and what the solution will look like).

How do we fix this? We look at non-linear, iterative models to describe these activities. We'll consider requirements and implementations as separate sets of activities, with their own iterative process.

We'll look at User-Centred Design and related requirements and design processes first. Afterwards, we'll consider Agile methods for implementation and project tracking.

# Agile Development

By the mid-90s, there was a widespread recognition that old, manufacturing-based ways of building software just didn't work—either for developers or for customers.

• Developers were frustrated by rigid processes and changing requirements.
• Business owners were frustrated by the inability to make changes to projects in-flight.
• Projects were often late and/or over-budget. This frustrated everyone!

There were a number of new, competing, process models introduced during this time. No one single model dominated. Instead, this movement culminated in 2001, when a group of software developers, writers, and consultants signed and published the Manifesto for Agile Software Development.

Agile was an attempt to find commonality across competing models. It defined a common way of approaching the problem.

What does this mean?

1. Individuals and interactions (over processes and tools): Emphasis on communication with the user and other stakeholders.
2. Working software (over comprehensive documentation): Deliver small working iterations of functionality, get feedback and revise based on feedback. You will NOT get it right the first time.
3. Customer collaboration (over contract negotiation): Software is a collaboration between you and your stakeholders. Plan on meeting and reviewing progress frequently. This allows you to be responsive and correct your course early.
4. Responding to change (over following a plan): Software systems live past the point where you think you’re finished. Customer requirements will change as the business changes.

"Agile Software Development" isn't a single process, but rather an approach to software development that encompasses this philosophy. It encourages team structures and attitudes that make communication easier (among team members, business people, and between software engineers and their managers). It emphasizes rapid delivery of operational software, but also recognizes that planning has its limits and that a project plan must be flexible.

There are a large number of software process models that were developed at this time, including Extreme Programming (XP) [Beck 1999], Scrum [Schwaber & Sutherland 1995], Lean [ Poppendieck & Poppendieck 2003]. Collectively, these are called "Agile Processes."

Much of this section is based on a more thorough discussion of Agility by [Pressman & Maxim 2020].

Agile is also implicitly about shifting power and decision-making from product managers and other business leaders to the development team, the ones actually building software. At least part of the failure of previous models is the failure to understand that development is not usually predictable. We're often building something for the first time, or solving a unique problem, so it's extremely difficult to predict the outcome far in advance.

# What is the benefit?

Agility means recognizing that requirements and plans will change over time.

1. Software is too complex to design and build all at once. It’s more manageable to add features and test incrementally.
2. Software is in a constant state of change, and requirements will change during the development cycle[.

The conventional wisdom in software development is that the cost of change increases as a project progresses. Agility is often characterized as "embracing change" since it expects project and requirements changes, and is constantly reassessing the state of the project. The benefit of Agile is that it reduces (and tries to eliminate) breaking late-project changes.

There’s a common misperception that software can be “finished,” shipped and then ignored. That’s not really true. By the time software reaches the market, the conditions will have changed (e.g. a competitor has introduced a feature, or you need to support a new type of device). Software needs to be constantly updated to keep up with changing environments (e.g., OS versions). Simple programs may last years without requiring updates, but as a program becomes more complex, it's more likely to need ongoing maintenance.

# Agile assumptions

Any agile software process is characterized in a manner that addresses a number of key assumptions about the majority of software projects:

1. It is challenging to predict in advance which software requirements will persist and which will change. It is equally challenging to predict how customer priorities will change as the project proceeds.
2. For many types of software, design and construction are interwoven. That is, both activities should be performed in tandem so that design models are proven as they are created. It is challenging to predict how much design is necessary before construction is used to prove the design.
3. Analysis, design, construction, and testing are not as predictable (from a planning point of view) as we might like.

Given these three assumptions, how do we create a process that can manage unpredictability?

Central to Agile processes is that any process must be adaptable to rapidly changing projects and technical conditions. It must also be incremental and incorporate customer feedback so that the appropriate adaptations can be made.

This idea of an iterative, evolutionary development model remains central to all Agile processes (although they may present it differently). Instead of building a "complete" system and then asking for feedback, we instead attempt to deliver features in small increments, in a way that we can solicit feedback continuously though the process. Over time, we will add more features, until we reach a point where we have delivered sufficient functionality and value for the customer.

We know that we want to iterate over our development activities.

We can probably get more precise by looking at two important Agile models: Scrum and Extreme Programming (XP).

# Scrum & Sprints

Scrum is the standard accepted process model for managing scope during a project iterations, i.e., it's focused on the overall project structure. Scrum breaks down a project into fixed-length iterations called sprints (typically 2-4 weeks in length for each sprint). Sprints are defined so that you iterate on prioritized features in that time, and produce a fully tested and shippable product at the end of each sprint.

Typically, a project will consist of many sprints, and you will iterate until you and the customer decide that you're done (i.e., the only remaining requirements are deemed low enough priority that you decide to defer them or not complete them). Practically, having a buildable and potential "shippable" product at the end of each cycle is incredibly valuable for testing, demonstrating functionality to customers, and it provides flexibility in how you deploy.

In Scrum, everything is structured around sprints:

Key Concepts

• Product Owner: the person responsible for gathering requirements and making them available in the product backlog. They are not considered part of the project team, but represent both the external business and customer. At the start of each sprint, they work with the team to prioritize features and decide what will be assigned to a sprint.
• Product Backlog: a list of all possible features and changes that the Product Owner thinks should be considered. There is no guarantee that these will all be developed! The team must agree to bring features forward into a sprint before they are developed.
• Sprint Backlog is the set of features that are assigned to a specific sprint. This is the "scope" for that sprint.
• The Scrum Master is the person that helps facilitate work during the sprint. They are not in charge (!) but track progress and help identify blocking issues that might prevent the team from meeting their deliverables.
• The Daily Scrum is a standup meeting where you discuss (a) what you've done since the last meeting, (b) what you intend to do today, and (c) any obstacles that might prevent you from accomplishing b. The Scrum Master runs this meeting, and the entire team attends.

Sprint Breakdown

The following steps are followed in each sprint:

1. The project team and Product Owner collectively decide what requirements to address, and they are moved from the Product Backlog to the Sprint backlog. Once features have been decided, you do not allow any further scope changes (i.e. you cannot add anything to the sprint once its started). Work is actually assigned to team members (and you collectively agree that you believe it can be completed in the sprint).
2. During the sprint, you iterate on the features in the Sprint Backlog. This includes design, development, testing until you complete the feature or the sprint ends. The Scrum Master facilitates daily meetings to make sure that nobody is "stuck" on their feature.
3. At the end of the sprint, have a review with the team to see what was accomplished. Demo for the Product Owner (and sometimes the actual customer). Reflect on your progress, and be critical of how you might improve process the next sprint. (e.g. could we have communicated better? should we have done more testing during the sprint? did we take on too many features?)

What do we like about Scrum?

• It provides structure for your iterations.
• It acknowledges that we can and should iterate over most of our activities. In other words, we always start a sprint with requirements (deciding what to include), design (elaborating on a feature), implementation, and testing.
• It explicitly identifies how to handle user feedback. We ask for input at the start of the sprint, and demo for feedback at the end of the sprint.

# Extreme Programming (XP)

Extreme Programming (XP) is an Agile methodology focused on best-practices for programmers. It was based on a large-scale project that Kent Beck managed at Chrysler in the late 90s, and attempted to capture what was working for them at that time. It aims to produce higher-quality software and a higher quality-of-life for the development team. XP aims to help us build high-performing software teams.

Core Values The five core values of XP are: communication, simplicity, feedback, courage, and respect.

• Communication: The key to a successful project. It includes both communication within the team, and with the customer. XP emphasizes face to face discussion with a white board (figuratively).
• Simplicity. Build the "simplest thing that will work". Follow YAGNI ("You Ain’t Gonna Need It") and DRY (Don’t Repeat Yourself).
• Feedback. Team members solicit and react to feedback right away to improve their practices and their product.
• Courage: The courage to insist on doing the "right thing." The course to be honest with yourselves if something isn't working, and fix it.
• Respect: Respect your team members; accept that development is a collaborative exercise.

Best Practices XP launched with 12 best practices of software development [Beck 2004]. Some of these (e.g. The Planning Game, 40-Hour Week, Coding Standard) have fallen out of disuse. Others have been added or changed over time, so it is difficult to find a "definitive" list of commonly used XP practices.

This is ok, and actually aligns with the practice of fix XP when it breaks.

Although some XP practices never really worked very well, many have been adopted as "best practices." We'll revisit these in the next section.

XP is rarely used as-is. It’s common for development teams to adopt one or more of these ideas based on what suits them, and their environment. For example, daily stand-ups are very common, but very few places will implement pair programming.

# SDLC

Let's revisit our lifecycle diagram.

There is no single set of practices that will work in every environment. It's common (and reasonable!) for organizations to adapt practies to suit the pecularities of their environment.

In this section, we'll piece together UCD and Agile practices into a working SDLC that suits this course and our objectives. We will "borrow" practices that make sense in this context, and ignore others that simply won't be as useful in our particular situation.

# SDLC Diagram

Let's add Agile practices to our previous SDLC:

Here's how to read the diagram:

• Each project starts with some basic planning activities. These are typically identifying team members and other resources, critical milestone dates and other variables for your project. (In this course, you are given all of these factors up-front! Your main planning activity is finding team-mates).
• Use the UCD to identify users, a problem to address and potential features. You should also prototype your user interface and get feedback from users.
• Implementation includes multiple iterations, where you choose features; design, code and write unit tests, and evaluate your results with users.

Each development iteration is a fixed-block of time, called a sprint (taken from Scrum). In a typical development project, sprints should be relatively short, from two to four weeks in length, and the team typically works through multiple sprints until the project is completed.

In our course, Sprints are two-weeks long, and we will have four sprints in total (i.e. 4 x 2-week sprints).

# Sprint Activities

Each sprint includes the following activities:

1. Feature Selection: On the first day of the Sprint, the team meets and decides what features to add (and what bugs to fix) during that iteration.
2. Implementation. During most of the sprint, the team iterates on their features. As each feature is completed, it is tested.
3. Evaluation. At the end of the Sprint, the team meets with the Product Owner to demonstrate what they have completed, and get feedback. The team also has a Retrospective, where they reflect on how to improve.

The cycle repeats for however many Sprints the team has available (or until they decide they are "done"). The product should be usable and potentially shippable to a customer at the end of each Sprint. The team may not always complete everything that they planned for a sprint, but the features that are implemented need to be bug-free.

Each sprint is supposed to result in a product release of high enough quality that it could be deployed.

This process model incorporates the XP ideas of iterations, iteration planning and small releases.

# Last Word