Testing & Evaluation

The goal of testing is to ensure that the software that we produce meets our objectives when deployed into the environment in which it will be used, and when faced with real-world contraints.

The goal of testing is to ensure that the software that we produce meets our objectives when deployed into the environment in which it will be used, and when faced with real-world constraints.

Why do we test?

We consider testing to be an integral part of software production. Why do we test?

1. To gain confidence in the correctness of your results.
2. To gain confidence that you are handling edge cases and errors properly, which will result in a better user experience.
3. To produce an improved design, usually as a by-product of having written tests.

As a matter of course, it’s also concerned with identifying deficiencies and flaws in software, and helping to determine how they should be addressed. In an iterative model, testing impacts implementation (by providing iterative feedback during the implementation process), and by providing input into planning phases, where we may need to fix bugs or plan product changes.

How do we test?

We can identify three types of testing:

1. Unit testing: tests operating at the class level (or smallest functional unit), which are meant to check the validity of low-level interfaces and systems.
2. Integration testing: testing across multiple classes or functional units, to check interaction between objects.
3. Functional testing: this is testing functionality from the perspective of the user; end-to-end feature testing. Sometimes called System testing.

Unit tests are particularly important and are meant to exercise the interface of a single class or module.

• Unit tests should be very quick to execute and report results.
• They should be restricted in scope to a class or entity, and dependencies should be eliminated for testing.
• They should be integrated into our development workflow, so that they are routinely executed (e.g. prior to commits).

Unit tests should always include “black box” tests. You should always test the interface by examining expected/actual results from a given input. NEVER write tests that rely on knowledge of the inner workings of a class. You should be able to interchange classes with identical interfaces and expect identical results.

We also need to want to confirm that the software works in the environment where it will be deployed. We may need to have multiple staged configurations where we will perform integration tests, to confirm that various systems work together, and functional tests to check user-level correctness.

When should we test?

Traditional views were that testing should be done after implementation. This is costly. Testing is more useful when done earlier in the process.

Different tests are suitable for different parts of the development process:

• Unit Tests: done during implementation, when you are working on a class.
• Integration Tests: done during implementation, when you want to ensure that classes work together.
• System Tests: done when features are complete and merged, to ensure that the system continues working.

In the previous chapter, we discussed Unit Testing and the value of writing tests as you develop (TDD). We’ll continue talking about testing that is done post-implementation. Specifically, let’s discuss Integration testing.

Resources

Martin Fowler. 2006. TestDouble. https://martinfowler.com/bliki/TestDouble.html

Lisa Crispin, Janet Gregory. 2008. Agile Testing: A Practical Guide for Testers and Agile Teams. Addison-Wesley Professional. ISBN 978-0321534460.

Vladimir Khorikov. 2020. Unit Testing Principles, Practices, and Patterns. Manning Publishing. ISBN 978-1617296227.

Glenford J. Myers, Corey Sandler, Tom Badgett. 2011. The Art of Software Testing. Wiley. ISBN 978-1118031964.