CS 346 Winter 2025

This is a course about designing and building software.

Modern software is often too complex for a single person to develop on their own. By working together, you and your project team will use best-practices to design and build a commercial-quality, robust, full-featured application, using a modern technology stack.

See the academic calendar for the official course description.

Course Sections

Classes are offered at the following times. You must be enrolled in corresponding sections i.e. both morning sections, or both afternoon sections.

Course Links

• Piazza 🔗: Forum software. Used for course annoucements, and you can ask questions.
• Learn 🔗: Used for quizzes, and project submissions. Grades are also recorded here.
• GitLab 🔗: CS 346 public repository, with sample code and templates.

Prerequisites

This course is restricted to Computer Science students.

Additionally, you must have successsfully completed CS 246 prior to taking this course. From that course, you should be able to:

• Design, code and debug small C++ programs using standard tools. e.g. GCC on Windows, macOS or Unix.
• Write effective unit tests for these programs. e.g. basic I/O tests; checking for a range of valid and invalid conditions.
• Demonstrate programming proficiency in C++, which includes:
  • understanding of fundamental OO concepts. e.g. abstraction, encapsulation
  • knowing how to use classes, objects, method overloading, and single inheritance; polymorphism
  • understanding how to use assertions, and
  • knowing how to manage exceptions.

Learning Objectives

This course includes a mix of lectures, demos and project activities. The course project is a significant element of the course.

On successful completion of the course, students will be able to:

• Work effectively as a member of a software development team.
• Use an iterative process to manage the design, development and testing of software projects.
• Design and develop different styles of application software in Kotlin, with appropriate architectural choices.
• Include online and offline capabilities in your application, leveraging both local and remote data storage.
• Design services that can provide remote capabilities to your application.
• Produce automated tests as an essential part of the development process.
• Apply debugging and profiling techniques as required during development.

Required Resources

All required course materials are freely available online. These include:

• Lecture slides for each in-person lecture, typically posted a couple of days ahead of time.
• Sample code to accompany the course notes, in our public GitLab repository.

You will require access to a computer to work on the course project.

• See Development > Getting-Started > Toolchain installation for details.

Finally, you will need access to the following websites.

• Piazza 🔗: Forum software. Used for course annoucements, and you can ask questions.
• Learn 🔗: Used for quizzes, and project submissions. Grades are also recorded here.
• GitLab 🔗: CS 346 public repository, with sample code and templates.

Assessment

This course is designed around your team project, and most activities are tied to the course project in some way. You will be assessed on both your individual work and the team project.

The course schedule lists the due dates for each component.

Individual grades (28%)

Assessments based on individual work.

Quizzes

We have weekly quizzes for the first 10 weeks of the course. Quiz content is based entirely on material presented in lectures that week. You are responsible for all material presented, not just what is in the slides.

Quizzes are hosted in Learn (Submit > Quizzes). Each quiz opens on the Fri indicated, and closes the following Fri (i.e. each one is open for a full week).

• Each quiz is 30 minutes long, and you are only allowed one attempt.
• Quizzes are open-book, meaning that you may refer to your notes while writing the quiz.
• Quizzes must be completed individually, without assistance. You are not permitted to collaborate or communicate with other students about the quiz content.
• We normally do not grant extensions for missed quizzes. See policies.

Participation

Everyone is expected to attend project demos and talk about their contribution. Participation marks are awarded for attending and actively participating in the process. Attendance is mandatory. See policies.

Team grades (72%)

Project milestones earned by the team. Everyone on the team receives the same grade.

Project proposal, Design proposal, Final submission

• See the appropriate page for specifics of each milestone.
• Grading details are available for each item in Learn > Submit > Rubric

Team demos

You and your team are required to demo your project to your TA in-class on the indicated dates. See the teams page for your assigned timeslot, and the project demo page for information on what you should have completed.

Individual attendance will be recorded (see above), and a team grade will also be assigned based on your team’s progress at that time. Code changes made after the demo will not be considered. It is critical that you follow the guidelines on the project demo page.

Policies

The following policies set expectations for how this course will be managed. Your responsibilities are outlined here, so you should take time to review them carefully.

Course Project

Participation

You are expected to participate in the course project, and all related activities, to the best of your ability. This means:

• You must be able to attend class with your team. You are not allowed to take this course remotely, or while on a work term that prevents you from attending in-person. If you attempt to take this course remotely, you may be required to withdraw from the course.
• If you fail to participate in a meaningful way during the term, you may be removed from the course. This may be done at any time prior to the start of the “Drop with WD” period. See the Important dates calendar for specific term dates.
• The instructor reserves the right to adjust grades if it is determined that a team member failed to substantially contribute to the course project.

Team formation

You are expected to form project teams in the first couple of weeks. The following guidelines apply:

• Students are responsible for matching up and forming teams. Teams must be formed by the end of the second week (i.e. the add-course deadline). If you fail to find a team, you must inform the instructor by this deadline.
• We do not guarantee you a team after the end of the second week. If you have not joined a team, and have not made some arrangement with the instructor, you may be required to withdraw from the course.
• Course enrolment will be managed by the instructor to encourage teams of four students. If necessary, the instructor may authorize larger or smaller teams, or modify team membership to accommodate everyone that is enrolled in the course.

Team demos

You are expected to participate in each project iteration. This includes contributing to each sprint, and attending and participating in team demos. If you fail to attend without a valid reason, you will receive a zero for that demo participation grade. At the instructors discretion, you may be exempt if:

• You have a conflicting coop interview and you are unable to reschedule it (per coop policy, rescheduling should be your first attempted course of action).
• You are ill on the day of the demo, and submit a VIF - see below for instructions.
• Other reason approved by the instructor ahead-of-time.

If an exemption is granted, it will take the form of an EXEMPT status for that component.

We do not typically allow absences from demos without a very compelling reason. e.g., “I want to study for another course”, or “I want to go home on the weekend”, or “I missed the bus” are not typically valid reasons for an exemption. Also, failure to contact us before missing the demo is an automatic zero for demo participation.

Absences

If you need to be absent and miss an assessment, please see the specific policies below.

Illness

If you are ill, and want relief from a course deliverable, you should follow the guidelines and steps outlined under Math Accommodations > Submitting a VIF.

You should also inform the instructor via email and also inform your team members so that they can make accommodations in your absence.

Religious accommodations

If you need to be absent for reasons related to a religious observance, please follow the instructions under Math Accommodations > Religious observances.

Please also inform the instructor via email.

Short-term absences

You cannot use a short-term absence (STA) to delay a team deliverable (including team demos). You may use a STA for an exemption from a quiz, if it’s submitted within 24 hours of the quiz deadline.

To use a STA to request an exemption from a quiz, you need to fill out 2 forms:

• Math Accommodations > Short-Term Absence form, and
• CS 346 declaration form to inform the instructor.

Note that you MUST fill in both forms for the absence to be considered valid.

Grading policies

Grade exemptions

If an exemption is granted for any reason (see above), it will normally take the form of an EXEMPT status for that component. The specific exempt component will not be included in your grade calculation, and grade weights will be redistributed across other components.

Regrade requests

Quizzes are marked automatically by Learn, and course grades are released by 10:00 AM on the Monday after a quiz closes. Other materials are manually graded by TAs, and grades are normally returned within 1 week of the due date.

Regrade requests for graded materials will be considered for one week after the grade has been released; we will not entertain last-minute regrade requests. If you wish to dispute a grade, email the instructor (for quizzes) or your TA (for project submissions). Contact information is here.

Incomplete (INC) grades

A grade of INC based on missed work will not normally be granted in this course.

Inclusiveness

It is our intent that students from all diverse backgrounds and perspectives are well-served by this course, and that student’s learning needs be addressed both in and out of class. We recognize the immense value of the diversity in identities, perspectives, and contributions that students bring, and the benefit it has on our educational environment. Your suggestions are encouraged and appreciated. Please let us know ways to improve the effectiveness of the course for you personally or for other students or student groups.

In particular:

• We will gladly honour your request to address you by an alternate/preferred name or pronoun. Please advise us of this preference early in the term, so we may make appropriate changes to our records.
• We will honour your religious holidays and celebrations. Please inform us of these at the start of the course.
• We will follow AccessAbility Services guidelines and protocols on how to best support students with different learning needs. Please submit these requests early in the term so that suitable accommodations can be provided.

Academic integrity

To maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility. Contact the Office of Academic Integrity for more information.

You are expected to follow the policies outlined here for quiz and project submissions.

To ensure academic integrity, MOSS (Measure of Software Similarities) is used in this course as a means of comparing student projects. We will report suspicious activity, and penalties for plagiarism/cheating are severe. Please read the available information about academic integrity very carefully.

Ethical behaviour

Students are expected to act professionally, and engage one another in a respectful manner at all times. This expectation extends to working together in project teams. Harassment or other forms of personal attack will not be tolerated. Course staff will not referee interpersonal disputes on a project team; incidents will be dealt with according to Policy 33.

Plagiarism and third-party code

Students are expected to either work on their own (in the case of quizzes), or work with a project team (for the remaining deliverables in the course). All work submitted should either be their own original work or work created by the team for this course.

The team is also allowed to use third-party source code or libraries for their project under these specific conditions:

• Published third-party libraries in binary form may be used without restriction. This may include libraries for networking, user interfaces and other libraries that are introduced in class.
• Source code from external sources may only be used if the contribution is less than 25 lines of code from a single source. Code copied in this way must be acknowledged with a comment embedded directly in your source code in the appropriate section.
• All external sources (libraries and source code) should be identified in the project’s README file.

Failure to acknowledge a source will result in a significant penalty to your final project grade (ranging from a minor deduction to a grade of zero for the project, depending on the severity of the infraction). Note that MOSS will be used to compare student assignments, and that this rule also applies to copying from other student projects, source code found online, or projects from previous terms of this course.

Reuse of materials

You cannot base your project in part or in whole on coursework that you have completed for a different course. You cannot submit the same project to multiple courses for credit, even if you are taking the courses concurrently (i.e. CS 346 and CS 446 would require different projects to be submitted that do not share source code).

Similarly, you cannot use anything that you created prior to the start of this course without explicit written permission from the instructor. This includes code, documentation, and other materials that you may have created in previous courses or work terms, or materials that you have created on your own time.

Generative AI & LLMs

The use of Generative AI and/or Large Language Models (e.g., ChatGPT, CoPilot and similar systems) is restricted in this course. You are not allowed to use such systems when writing quizzes, or for generating any part of your written submission (e.g., design document, user instructions, final report).

You are allowed to use these tools to generate a limited amount of code towards your project, provided that you adhere to the policies described above under plagiarism. Content generated by these systems must be sourced and cited properly, just like any other third-party source code.

Improper use of these tools will be subject to Policy 71 and an investigation into academic misconduct.

Keep in mind that if you use these types of systems to generate code, you are responsible for what is produced! These models can generate incorrect or illogical code. Non-working generated code will not be an acceptable excuse for failing to meet project requirements or deadlines.

Student discipline

A student is expected to know what constitutes academic integrity to avoid committing an academic offence, and to take responsibility for his/her actions. A student who is unsure whether an action constitutes an offence, or who needs help in learning how to avoid offences (e.g., plagiarism, cheating) should seek guidance from the course instructor, academic advisor, or the undergraduate Associate Dean.

For information on categories of offences and types of penalties, students should refer to Policy 71, Student Discipline. For typical penalties, check Guidelines for the Assessment of Penalties.

Intellectual property

Students should be aware that this course contains the intellectual property of their instructor, TA, and/or the University of Waterloo. Intellectual property includes items such as:

• Lecture content, spoken and written (and any audio/video recording thereof)
• Lecture handouts, presentations, and other materials prepared for the course (e.g., PowerPoint slides)
• Questions or solution sets from various types of assessments (e.g., assignments, quizzes, tests, final exams), and
• Work protected by copyright (e.g., any work authored by the instructor or TA or used by the instructor or TA with permission of the copyright owner).

Course materials and the intellectual property contained therein, are used to enhance a student’s educational experience. However, sharing this intellectual property without the intellectual property owner’s permission is a violation of intellectual property rights. For this reason, it is necessary to ask the instructor, TA and/or the University of Waterloo for permission before uploading and sharing the intellectual property of others online (e.g., to an online repository). Permission from an instructor, TA or the University is also necessary before sharing the intellectual property of others from completed courses with students taking the same/similar courses in subsequent terms. In many cases, instructors might be happy to allow distribution of certain materials. However, doing so without expressed permission is considered a violation of intellectual property rights.

Continuity plan

As part of the University’s Continuity of Education Plan, every course should be designed with a plan that considers alternate arrangements for cancellations of classes and/or exams.

Here is how we will handle cancellations in this course, if they occur.

• In the case of minor disruptions (e.g. one lecture), the lecture content will be reorganized to fit the remaining time. This should not have any impact on demos or deliverables.

• Cancellation of multiple classes may result in a reduction in the number of sprints and associated deliverables to fit the remaining time. If this happens, lecture content will also be pruned to fit available time. Assessment weights will be redistributed evenly over the remaining content if required to align with the material.

• Cancellation of in-person (midterm or final) examinations has no effect on this course, since we do not have scheduled exams and quizzes are online.

Contact us

Who are we? How to contact course staff.

Contact information

You are welcome to contact us directly. Your assigned TA will be listed on the Project teams page after Week 3.

Contact List
Prof. Jeff Avery Course Instructor jeffery.avery@uwaterloo.ca
Caroline Kierstead Instructional Support Coordinator (ISC) ctkierst@uwaterloo.ca
Shaikh Shawon Arefin Shimon Teaching Assistant ssarefin@uwaterloo.ca
Rafael Ferreira Toledo Teaching Assistant rftoledo@uwaterloo.ca
Favour Kio Teaching Assistant gn2kio@uwaterloo.ca
Paul Wooseok Lee Teaching Assistant w69lee@uwaterloo.ca
Aniruddhan Murali Teaching Assistant a25mural@uwaterloo.ca
Amber Wang Teaching Assistant j2369wan@uwaterloo.ca

Getting help

Classes are organized so that you have free time during class to work on your project. The instructor will always be present to help you with any questions you might have!

We also have a Piazza forum where you can post questions, and discuss the course with one another.

We will use Piazza for class annoucements:

• Please sign up with your real name (tip: you can post with an alias, the course staff can see your name).
• Make sure to monitor regularly for course announcements (tip: make sure that notifications are turned on, and you will receive an email when important posts are made).

Course Plan

The planned course structure is shown below. Follow these links for a larger image, or to see the legend.

Weeks 1-6

In weeks 1-4, all class time will be used for lectures. Some time at the end of class will be used for project activities with your team. You will have early project deliverables that you and your team will submit, including a project proposal and design proposal. By week 5-6, you will be implementing features for your project.

Weeks 8-13

After reading week, you will be focused on your project. Lectures will often be shorter, and Fri labs are almost completely dedicated to working on your project. You will need to demo your progress to your TA every second week. At the end of the course, you will produce a final submission with expanded documentation.

Course Schedule

Significant dates and deadlines for the term are listed below.

Lecture Notes

Week-by-week lecture topics, with links to course slides and notes.

Week 01 - Introduction, Design thinking

Welcome to the course! We have a lot of material to cover in the first few weeks so it’s important that you attend class. Please email the instructor or post on Piazza if you have questions.

Wed Jan 8

Introduction (slides pending)

Course outline

• Course plan
• Assesssment
• Course policies

Course Project

• Requirements
• Forming a team
• Picking a project

Development Process

• Teamwork (slides pending)
• Agile (slides pending)

Fri Jan 10

Design thinking

• Design process (slides pending)

Week 02 - GitLab, Kotlin

Wed Jan 15

GitLab Setup

• GitLab overview
• Documentation
• Project proposal

Kotlin

• Kotlin Basics

Fri Jan 17

IntelliJ IDEA

• Install the toolchain

Kotlin, continued

• OO Kotlin
• Functional Kotlin

Project teams

A list of project teams, with links to their project repositories.

Morning schedule

LEC 001 & LAB 101

Afternoon schedule

LEC 002 & LAB 102

Getting started

How do you get started with your project?

Requirements

Guidelines

We are designing and building graphical applications that leverage cloud services and remote databases. The project should be a non-trivial Android or desktop application that demonstrates your ability to design, implement, and deploy a complex, modern application.

You must use the technology stack that we present and discuss in-class:

• GitLab for storing all of your project materials, including documentation, source code and product releases. All assets and source code must be regularly committed to GitLab. See specific requirements as the course progresses.
• Kotlin as the programming language for both front-end and back-end. No other languages are allowed.
• Jetpack Compose or Compose Multiplatform must be used for building graphical user interfaces.
• Ktor must be used for any networking requirements that your application might have.
• The kotlin-test classes and JUnit for writing tests.

You may choose any hosted cloud/db provider. The following have been used successfully in past offerings of this course, and are recommended: Firebase, Supabase, and MongoDB.

Additional libraries may be used, provided that you follow the course policies.

You are also expected to leverage the concepts that we discuss in class:

• You should demonstrate design principles that we discuss in lectures.
• You must build a layered architecture, using MVVM, MVI or another pattern we discuss.
• You are expected to write and include a reasonable number of unit tests for your application.

You will be assessed based on:

• How well you define the problem you are solving, and understand the users you are solving it for.
• How well you design features to address that problem and the quality of your design.
• The clarity, quality and ease-of-use of your application.
• Your ability to work as a team to incrementally deliver features over time.

Requirements

In your Project Proposal, you will define your target users, a problem to address, and a set of custom features that you intend to implement to address that problem. The details will be unique to each team, but should be non-trivial features that demonstrate your ability to design and implement complex functionality to address a problem. You should be able to articulate how these features make your product different or better than existing applications.

Your application should minimially consist of a desktop or Android application with the following requirements.

1. Graphical user interface

Your application must have a graphical user interface, created using either Jetpack Compose (Android) or Compose Multiplatform (Desktop).

Your application should include at least three different screens, and you must support navigating between them as part of your normal application functionality.

You should have a custom theme that you have produced i.e., custom colors, fonts and so on. Marks will be deducated if you simply use the default Material theme that is included in Compose.

For a desktop application:

• Your screens should be windowed.
• Users should be able to minimize, maximize and resize windows as you would expect for a desktop application, and windows should have titles.
• Your application should have a custom icon when installed.

For an Android application:

• Users should be able to use normal gestures to navigate between screens. You must support back-stack navigation.
• When rotated, your screens should rotate to match the orientation, and layouts should modify themselves to fit the screen dimensions in either orientation.

You are expected to produce prototypes for these screens as part of your initial design.

2. Multi-user support

Your application should support multiple users. Each user should be able to login, and see their own specific data and application settings.

• Each user should be able to create an account that represents their identity in the application.
• Accessing user data must be restricted by account i.e. users can only see their own data, unless a feasure specifically reveals shared data e.g., sharing a recipe among friends using the application.
• You should support standard account functionality:
  • Each user should be able to create their own account, and login/logout using that account.
  • You should perform standard validation when creating accounts or entering information e.g., do not allow duplicate username or password; mask out the password when entered; passwords must contain special characters.
• Credentials should persist between sessions i.e., your user should not have to login each time they launch the application.

3. Networking and security

You must use Ktor for any network access.

• Network requests do not need to be encrypted, but you should require authentication for any access (i.e., the user has to login before they can access any service or data).
• You should leverage existing authentication services where possible e.g., Firebase Auth or Supabase Auth. Do not “roll your own” insecure system!

4. Data storage

Your application must manipulate and store data related to your problem e.g., recipe for a recipe tracking application; user profile information needed to login; the user’s choice of theme to use in the UI.

You must implement both local and remote storage.

Local storage

• Local storage should include the last-known-good application data.
• Your application should attempt to connect to a remote data store and fetch that data (and either merge, or replace your local data, depending on the design).
• If remote data is not available (i.e., no internet connection, or unable to login to a remote database) then the local data should be used instead. The application should be completely functional using local data.
• Your local data storage does not need to be any specific format: it can be a file, or a local database e.g., SQLite.

Remote data

• This is your preferred place to store data, either to keep personal data in the cloud, or to share it with other users e.g., sharing recipes with other users of your recipe application.
• If remote data is available, your application should fetch it on launch (see above) and use a local copy. On exit, or when other reasonable conditions are met, your application should push local data to the remote data store to keep it synchronized.
• Your remote data storage should be in a hosted database of your choice. This can be a dedicated instance e.g., one that you have installed and control, or a hosted database through a cloud provider like AWS, Firebase or Supabase. Use of a custom SDK is allowed, as is a database framework like Room or Exposed.

5. Custom features

Using the guidelines above, you are expected to propose, design and implement multiple features that solve a problem for your users.

• Your application must manipulate and store some information related to your problem e.g., recipes for a cooking application, markdown documents for a journalling application.
• Your functionality must be more complex than just simple CRUD operations. e.g., your recipe app should also generate a shopping list, or store user reviews of recipes, or suggest alternate ingredients.

See the picking a project page for suggestions.

Examples

Recipe tracker

Image courtesy of Recipe Keeper

Users

A group of friends that want to cook together e.g., roommates.

Problem

How do you keep track of what recipes you have tried, and which ones each person liked?

Proposal

A recipe planning and tracking application that allows users to:

• Enter recipes (imported from a recipe site or manually entered).
• Browse a collection of recipes, with pictures. Search your recipes and friends recipes.
• Rate recipes, and view what other users have rated.
• Generate a list of “most wanted” recipes, or “most popular” after the group has tried them.
• Extra: track food allergies and support automatic filtering of recipes.

Design

Technical aspects to consider.

• Android application, with screens for all of this functionality
• Multi-user, so each person has a profile (login/logout/edit).
• Local database for caching details (SQLite & file storage)
• Remote database for storing user data and recipes (Firebase)
• OAuth authentication using Google accounts (also Firebase)

Software design tool

Image courtesy of Visual Paradigm

Users

Software developers that are interested in collaborative design.

Problem

Collaborative tools aren’t tailored towards software design, so we end up trying to create UML documents in Google Docs (awkward). UML tools often don’t support collaborative, which would be incredibly valuable!

Our goal is collaborative UML drawings.

Proposal

An online tool that lets multiple people work together to draw UML diagrams in real-time.

• Can have multiple drawing canvases; on launch choose which one to open and work on.
• Canvases should have a name, date-created, date-edited. Anyone can edit anything.
• Drawing tools: draw shapes; draw lines to connect them; move shapes; change properties.
• Export canvas to JPG (PNG, other formats) for use in other applications.
• Extra: Templates so that you can draw a plain box-arrows diagram, or a specific UML diagram e.g., class diagram, component diagram. (Do not support all UML, but a small subset; focus on infrastructure to add more later).
• Extra: Support more diagrams by expanding the templates!

Design

• Desktop tool, since it’s more precise for drawing.
• Multi-user, so each person has a profile (login/logout/edit).
• Investigate canvas classes in Compose for drawing arbitrary shapes.
• SQL database, since we expect a large collection of templates (predefined shapes) and think that’s a better approach.
• Will likely need a way to handle real-time updates for multiple users working on a document e.g., WebSockets.

Forming Teams

The first, and most important thing to do is to find teammates! Ideally, you should form a project team in the first week of the course.

Find other people

How do you find team members, or find an existing team to join?

• Join friends who are also taking the course! If you are in different sections, speak with the instructor to see if you can be moved to the same section.
• Post on Piazza to introduce yourself and find teams that are looking for people.
• If you’re in-class, introduce yourself to people sitting near you! We’ll even give you time at the end of lectures to meet.

Discuss how to work together

Before agreeing to work together, make sure that you agree on some teamwork basics:

• Are you all registered in the same section? This is a course requirement.
• Are you all able to attend class?
• Do you have free time outside of class to meet?

Make sure to review the team contract guidelines. You will be required to complete a team contract as part of your Project Setup milestone.

Register in Learn

Once you have a team formed, register yourselves on Learn:

• Navigate to Connect > Groups and choose an empty team from the list
• Add yourself and your team members.

Picking a topic

We recommend the following steps:

• Have a discussion with your team about any personal goals that you might have. For example, if you’re interested in learning about a particular technology e.g., database or user interfaces, this might be a good opportunity to explore that.
• Brainstorm! Try and identify potential users, and problems that you could solve for them. If you need inspiration, look at existing applications and try and improve on those. This is important: identify the need first, and then second think of how to solve that need.
• Record all of your ideas. You want to capture everything, and as they say, “there is no bad idea” at this point!
• Discuss as a team and narrow down your ideas to a single problem/solution that you would like to address.

Public data/APIs

If you’re stuck for an idea, here are some interesting public APIs that you might be able to leverage. Build an application to track a comic-book collection, or a cool new weather app for your phone!

• Stack Overflow API: https://api.stackexchange.com/
• UWaterloo Open API: https://openapi.data.uwaterloo.ca/api-docs/index.html
• Notion API: https://developers.notion.com/
• Pokemon API: https://pokeapi.co/
• Marvel Developer Portal: https://developer.marvel.com/
• REST Countries: https://restcountries.com/
• NASA Open APIs: https://api.nasa.gov/
• Weather API: https://openweathermap.org/api
• Market Data API: https://polygon.io/
• News API: https://newsapi.org/
• YouTube API: https://developers.google.com/youtube/?ref=apilist.fun
• OMDb (Open Movie Database) API: https://www.omdbapi.com/
• DeckOfCards API: https://deckofcardsapi.com/
• Open Library API: https://openlibrary.org/developers/api

ChatGPT project ideas

More ideas, courtesy of ChatGPT. Choose an idea that aligns with your interests and skills! If you use one of these, you will likely need to add features to make your application stand out.

1. Task Manager: Create a desktop task manager application that allows users to organize their tasks, set reminders, and prioritize tasks based on deadlines or importance. To make it really interesting, integrate with their Google Calendar.
2. Note-taking App: Develop a note-taking application where users can create, organize, and search through their notes. Include features like rich text formatting, image embedding, tagging, and synchronization of notes across devices.
3. Expense Tracker: Build a desktop expense tracker to help users manage their finances. Include features like expense categorization, budget tracking, and generating reports or visualizations. Let them export/import data from their banking application.
4. Password Manager: Design a password manager application that securely stores and manages users’ login credentials for various accounts. Implement encryption and features like password generation and synchronization. Encrypt everything!
5. Weather App: Create a desktop weather application that provides users with real-time weather information, forecasts, and weather alerts for their location or specified locations. Provide alerts based on conditions that they set e.g., “it’s below 10 degrees, so wear a coat”.
6. Fitness Tracker: Build a fitness tracker application that helps users track their physical activity, set fitness goals, and monitor their progress over time. Include features like exercise logging, calorie tracking, and workout planning.
7. Recipe Organizer: Develop a recipe organizer application where users can save, categorize, and search for recipes. Include features like meal planning, ingredient shopping lists, and recipe sharing.
8. Code Snippet Manager: Design a code snippet manager application that allows developers to store, organize, and search through their code snippets. Include features like syntax highlighting, tagging, and integration with code editors.
9. File Encryption Tool: Create a desktop application for encrypting and decrypting files to ensure users’ data privacy and security. Implement strong encryption algorithms and user-friendly interface.
10. Screen Capture Tool: Build a screen capture tool that allows users to capture screenshots or record screen activities. Include features like annotation tools, customizable capture settings, and sharing options.
11. Music Player: Develop a desktop music player application with features like playlist management, equalizer settings, and support for various audio formats. Optionally, include online streaming capabilities.
12. Calendar App: Design a calendar application that helps users manage their schedules, appointments, and events. Include features like multiple calendar views, event reminders, and synchronization with other calendar services.
13. Language Learning Tool: Build a desktop application to help users learn a new language, with features like flashcards, vocabulary exercises, pronunciation practice, and language learning progress tracking.
14. Movie tracker: Build a mobile application to track movies that you’ve seen. Import movie data from IMDB, rate them, add comments and share reviews with your friends!

Deliverables

Items that you are required to submit for grading.

Project setup

This section describes a project milestone. See the schedule for due dates.

Create a GitLab project

Once your team has been formed, you will need to create a project space in the University of Waterloo GitLab. You should create a single GitLab project for your team, and add the information detailed below. When done, you will need to submit your project details in Learn.

Step 1: Use new-project wizard

1. Open the GitLab home page.
2. Select + > New project/repository > Create Blank Project.
3. Fill in the form:

• The project should be placed under the Username of one of your team members (e.g. j2avery in the URL example above).
• Visibility should remain Private.
• Your project should have a descriptive name (see Forming a Team). e.g., UberTweets or Social Calendar.

4. Select Create project to proceed.

Step 2: Set project security

It’s important that you set permissions on your project. This provides access to those who need it, while preventing others from accessing it!

The person who sets up the project should make the following changes:

1. Check project visibility.

• Settings > General > Visibility Level
• Ensure that it’s set to Private.

2. Add your teammates to the project with full access.

• Manage > Members, Invite Members.
• When they accept, change their role to Owner.

3. Add course staff with reduced permissions. Add the instructors and TAs to your project.

• Manage > Members, Invite Members and use each person’s email address.
• When they accept, set their role to Developer.

Step 3: Add project details

Optionally, you can also add a project icon, and other details.

• Settings > General, Project avatar.
• Settings > General, Project description.

Step 4: Create a README.md

You should have a markdown file in the root of your source tree named README.md. This will be displayed by default when users browse your project and serves as the main landing page for your project.

You must have at-least the following details included in your README.md.

# SUPER-COOL-PROJECT-NAME

## Title
A description of your project e.g., "an Uber-eats app for parakeets!". 

## Team Details
Basic team information including:
* Team number
* Team members, listed in in alphabetical order. Include full names and email addresses.
* Link to your Team Contract wiki page.

Step 5: Create a team contract

In your Wiki, create a page for your Team Contract and link it to your README.md under Team Details.

Minimally, your team contract needs to contain:

• Names and contact information of all team members.
• Agreement on how you will meet in-person e.g., how often, and location. You are expected to meet at least twice per week and document it.
• Agreement on how you will communicate e.g., email, Messages, WhatsApp, MS Teams. You need one agreed-up communication channel that everyone will check.
• Agreement on team roles: who is the project lead? Are people taking on specific design responsibilities?
• Agreement on how the team will make decisions. Do you vote? Do you need a majority?

How to submit

Login to Learn, navigate to Submit > Dropbox > Project Setup, and submit a link to your top-level project page. We need this so that we can track your project and assign you a TA for the term.

This must be submitted by the listed time and date.

Project proposal

This section describes a project milestone. See the schedule for due dates.

A project proposal is a high-level description of your project. It should detail what you intend to build. We usually do this up-front to ensure that everyone is on the same page ¹. It’s also a great way to get initial feedback before you have invested too much time in the project.

What to include

This document should be a Wiki page in your project named Project Proposal. You should include the following sections:

1. Purpose

What is the purpose of your project? What problem are you solving, and for what users? Think of this as an “elevator pitch” i.e., a paragraph that defines your project goals.

e.g., “We are building a software application that will allow parakeets to order takeout from their cages.”.

2. Background

Describe your users, what they need, and what problems exist that you are trying to address for them. Provide at least one persona for a typical user of your product².

e.g., “Polly the parakeet is a typical parrot who can get peckish. Parakeets are always hungry, but they are often caged and can’t just go for takeout whenever they want”.

3. Requirements

Identify the target platform and OS version you are targeting with your application i.e., Android, iOS, Windows, macOS.

What are the features that you plan to deliver on that platform, to solve this particular problem? If you identified any User Stories as part of defining your project, you can include them in this section.

Minimally, list 5-10 features that serve as the core of your project, and that you expect to implement. Make sure to consider the core project requirements, since you need to address those as well.

• You should create these features as issues under Issues > List. They should be unassigned, and listed as “No milestone,” since you haven’t scheduled them yet.

e.g., “This needs to be a phone app. We’ll need to support a stylus because she has claws. The parakeet can use it to pull up her favorite recipes on the main menu, and then click a button to order. We’ll send the order through the Uber Eats server. Somehow. We also need to figure out how to get her to enter a password.”

Where to store this document

Your proposal should be contained in a Wiki page titled Project Proposal, and linked from the README.md file of your GitLab project.

How to submit

When completed, submit your Project Proposal for grading.

Login to Learn, navigate to the Submit > Dropbox > Project Proposal page, and submit a link to your top-level project page.

Proposals must be submitted by the listed time and date.

Design proposal

This section describes a project milestone. See the schedule for due dates.

Your design proposal builds on your Project Proposal by considering how you will build the system that you describe in that document.

What to include

You should have the following sections, where you address the questions listed. At this stage, answers should be relatively short; you should expect to do further investigation during the implementation phase.

1. Architecture

Assume that you will implement the requirements listed in your Project Proposal.

1. Provide a high-level component diagram, showing the structure of your system. You should identify your application, including relevant application components, and any external systems e.g., remote database, cloud service.
2. You’ve been provided with a number of architectural styles. Which one do you think suits your application?
3. For components outside of your local application, where do you expect them to be hosted? Will they be hosted by some third party? e.g, a cloud service like AWS? A cloud database like MongoDB?
4. Are there privacy and security implications to your design? Does being online change how you think about creating an application?

2. Application features

1. Are there any features that you do not know how to implement? Identify risks and features that you need to investigate early in your project.
2. Sketch out low-fidelity prototypes of your UI/screens, showing basic functionality of your application. Keep these simple! You need to illustrate basic information and screen flow; you don’t need much more detail.

Where to store this document

Your proposal should be contained in a Wiki page titled Design Proposal, and linked from the README.md file of your GitLab project.

How to submit

When complete, submit your Design Proposal for grading.

Login to Learn, navigate to the Submit > Dropbox > Design Proposal page, and submit a link to your top-level project page.

Proposals must be submitted by listed time and date.

Product demo

This section describes a project milestone. See the schedule for due dates.

You are working through product iterations, where you plan, design, and code in two-week cycles. You always start with planning, and end with a demo of what you accomplished.

First day: planning

The first day of an iteration should include be a planning meeting, where you:

• Review the results from the previous demo.
• Review work that was not completed and decide if you wish to continue working on it.
• Plan out the remaining work, taking into account everyone’s availabilty, priority of issues and so on.

By the end of your meeting on the first day, everyone should have issues and work assigned to them.

Middle days: design, coding

Over the remaining two weeks, you should meet occasionally to coordinate your work.

You and your team should meet at least twice each week to review your progress.

• Record meeting minutes and store in the wiki. Meeting minutes need to include: date, who attended, what you decided.
• See gitlab/templates for a sample of a meeting minute format.

Issues must be maintained and updated as you work. Document your work indicating what was done. Use GitLab issues as the primary place to store information about what you’re working on.

• The issue list in GitLab should always reflect the state of the project.
• Issues always stay assigned to the person responsible for that work.

Last day: demo

On demo days, you will meet your TA in class and present your progress.

• See the Team schedule for your assigned timeslot.
• You have 15 minutes for the demo.

Before the demo:

• You MUST product a software release. Please read and follow those instructions carefully.
  • Your application must be installed from the software release and runnable from a single machine. Do NOT demo from the IDE or an editor.
  • You should expect that your TA will review the code after your demo. You should NOT make code changes afterwards since we will grade what was tagged for the release.

During the demo:

• You must run the demo from a single machine.
• Each person on the team should demonstrate what they completed.
  • Do NOT demonstrate code, we want to see working functionality when possible.
  • You are allowed to show GitLab wiki documents, unit tests results, or other work that demonstrates what you accomplished (since these are things that cannot easily be demonstrated by a feature).

How to submit

You do not need to submit anything. However, you can expect the TA to git clone your project and run your code following the demo to review what you showed them.

Final submission

This section describes a project milestone. See the schedule for due dates.

At the end of the term, we will review your entire project. This section describes what you will be required to submit.

This submission should reflect all the work that you did during the term. You can and should be working towards these items incrementally. In the final weeks of the course, you should focus on making sure that documentation is updated and reflects what you actually accomplished!

Software release

You should produce a software release.

The following guidelines apply:

• Your project should be buildable from main using gradlew.
• Installers should also be buildable from main using gradlew. If there are manual steps required to build your project, they must be documented and made clear to the TA grading it.
• Unit tests must be included for all entities, models and view models.

Documentation

All documentation should be written in Markdown format, using Mermaid for injecting diagrams.

With the exception of the README.md file, each document should be a Wiki page that follows the naming convention here (e.g., create a document named “User documentation”, another named “Design documents” and so on). Consider linking your Wiki page to the top-level README file to make it easier to locate.

Your project should include the following sections.

1. README file

Your top-level document should serve as the landing page for your project, and a table-of-contents for the rest of your content.

It should include:

• Product title
• Product description
• Requirements to install/run your project
• Screenshot and/or introduction video
• Team names and contact info

It should also include links to the following Wiki documents:

• Project proposal (unchanged from initial release)
• Design proposal (unchanged from initial release)
• Meeting minutes
• Software releases, with links to release notes/installers (final release, plus previous submissions).
• User documentation (new, see below)
• Design diagrams (updated, see below)
• Team reflections (new, see below).

2. User documentation

You should expect to document your project! Documentation will include:

• Include brief instructions on how to install your product. Bullet points is fine.
  • Make sure that you should specify what device to use (e.g., Android AVD Medium Phone).
  • Include specific steps on how to install your application. For desktop, this should include steps on how to run the installer. For Android, discuss setting up the AVD and dragging the APK into it the launch it.
  • If there are any other instructions e.g., how to launch your server, include them here.
• Provide guidelines on how to use your main features. If they are complex, you should include diagrams or screenshots.
  • It is not required to describe all of your functionality, just provide a guide to the main features.
  • If you have multiple screens, describe what each ones does and how to navigate between them.

3. Design documents

We want to see both your original designs, and any updated documents. It should be clear when comparing them what, if anything, has changed.

Create new, revised versions of the following:

• Your personas
• Figma UI prototypes.
• A high-level component diagram showing your application structure e.g, layered architecture, with external systems for auth and database.

Create class diagrams for your main classes in your application.

• Focus on your “most important” classes: entities, models and view models. You do not need to create diagrams for external systems like your database or the user interface compose functions (although you do need to diagram view model classes, and it should be clear in your diagram which screen each one is supporting).
• Make sure that you illustrate where the classes “fit” in a Layered Architecture e.g., organise them into layers.

It’s entirely possible that you may not need to update your personas, or prototypes, or component diagram from earlier. That’s ok. In this case, just create copies of the earlier diagrams, and put the copies in the Design diagrams page.

4. Source code repository

Source code and other assets required to build your application should be stored in your project’s source code repository. The structure should look something like this by the end of the term.

.
├── .gitignore
├── README.md
├── application
│   ├── build.gradle
│   └── src
│       ├── main
│       │   └── kotlin
│       └── test
│           └── kotlin
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
├── releases
│   └── v0.1-release-notes.md
│   └── v0.1-installer.dmg
│   └── v0.1-installer.exe
├── server
│   ├── build.gradle
│   ├── gradle.properties
│   └── src
│       ├── main
│       │   ├── kotlin
│       │   └── resources
│       └── test
│           └── kotlin
├── settings.gradle
└── shared
    ├── build.gradle
    └── src
        └── main
            └── kotlin

For information on creating your project structure correctly, see Build configuration and Gradle documentation.

Updated project details

Your GitLab project should reflect the final (current) state of your application.

• Issues should be assigned to the appropriate sprint milestones.
• Issue status should be correct i.e. completed and merged features should be marked as closed in GitLab.

It’s acceptable to have open issues at the end of the project, but they should not be assigned to a milestone (and related code should be on a feature branch, not merged back into main).

Assessment

You will be graded on the documents above, as well as your final product release. When evaluating your product, marks will be assigned based on how well you have met project objectives.

See the Final Release rubric in Learn for grading details.

How to submit

You do not have to explicitly submit your final. We will grade based on what was submitted by the listed time and date using the criteria above.

Introduction

Let’s introduce application development topics.

This is a course about designing and building software. There are many different styles of software:

• Application software: software that is designed to perform a task for users. e.g., your phones calendar application, Minecraft, Visual Studio Code, Google Chrome.
• System software: software that is meant to provide services other other software e.g., Linux or other operating systems, drivers for your NVIDIA graphics card, the printer control panel on your computer.
• Programming languages: languages and tools designed to produce other software.
• Embedded software: meant to control systems; included in the manufacturing of a hardware based system. e.g., firmware in your phone; control systems for a robotic arm in a factory; monitoring systems in a power plant.

In this course, we’re primarily concerned with application software, or software that is used by people to perform useful tasks. Most of the software that you interact with on a daily basis would be considered application software: the weather app on your phone, the spreadsheet that you use for work, the online game that you play with your friends.

Applications aren’t restricted to a single style either; they can include console applications (e.g., vim), desktop software (e.g., MS Word for Windows), mobile software (e.g., Weather on iOS), or even web applications (e.g., Gmail).

The categories are not always this clear-cut. For example, a web browser is an application, but it also provides services to other applications e.g., rendering HTML, executing JavaScript. Similarly, an operating system is system software, but it also provides services to applications e.g., managing memory, scheduling processes. When you install your operating system, it probably comes preinstalled with a bunch of applications! These distinctions are not always obvious (of meaningful).

For us, we just want to think in terms of software that is usable by an end user for a specific task.

History of software

Computer science has a relatively short history, that mostly started in the 1940s. Let’s consider how software development itself changed over time.

1960s: Terminals & text displays

In the early days of computers, mainframe computers dominated. These were large, expensive systems, which were only available to large instututions that could afford to maintain them. This era started with batch processing dominated, where users would submit their jobs (programs), and wait for the results. By the mid 1960s, we saw the introduction of time-sharing systems, where multiple users could interact with the computer at the same time through text-based terminals.

Development snapshot

Hardware was limited to managing text, so software was (mostly) text-based. Systems were powerful enough to support interactive sessions with users, so software was either either command or menu driven (e.g., you were presented with a screen, and navigated through lists of options, and answering prompts.

Software development at this time was interesting. We started with batch processing (punch cards) and ended the 1970s with dedicated terminals connected to a time-sharing mainframe. Programing meant editing your COBOL or Fortran program on your terminal, with programs stored on bulk storage.

1970s: Personal computing

By the mid 1970s, the situation had changed. Small and relatively affordable computers were becoming commonplace. These personal computers were relatively inexpensive, and designed to be used by a single user i.e., hobbyists, or small businesses that couldn’t afford a mainframe or mini computer (the term personal reinforced the idea that these were single-user machines).

While companies were investing in these so-called personal computers, at the end of the 19702, they will still relatively niche. They had limited capabilities, and very limited software available for them.

Development snapshot

Personal computer software was text-based and command or menu driven, with some limited ability to display color and graphics.

1980s: Graphical user interfaces

The early 1980s was a time of incredible innovation in the PC industry: software was sold on cassettes, floppy disks or cartridges, and program listings were printed in books and magazines (you were expected to type in a program before running it). BBSes existed, where you could connect and download software.

The growth of this sector provided a huge opportunity for specialized vendors to emerge and launch a small software industry. Companies like Borland and Lotus emerged specifically to produce software for this growing market. Commercial software was being distributed on cassettes or floppy disks and was more professionally produced (remember, no internet yet, so downloading commercial software wasn’t yet an option).

The early 1980s was a time of rapid innovation. Console applications had advanced significantly.

Released on Jan 26 1983, Lotus 1-2-3 was the world’s first spreadsheet. It was so popular that it is often considered a driving factor in the growth of the personal computer industry. Businesses were buying computers specifically to run Lotus 1-2-3. Early software companies like Microsoft were born in this era (their first products were DOS, a PC-based operating system, and BASIC, a programming language).

Applications from this era were text-based, at least for a few more years. Text-based applications still exist today: bash, git, and other tools are commonly used for system administration tasks. Many software developers also like to use console and text-based tools for software development.

As hardware quickly improved, graphical systems quickly became more popular. Introduced in 1984, the Apple Macintosh was the first successful commercial graphical operating system.

Other vendors e.g. Microsoft, Commodore, Sun quickly followed with their own designs.

The move to GUI interfaces of course meant that software was ported to these environments.

Key features of a graphical user interface include:

• Use of a keyboard and pointing device (mouse) for interaction. Point-and-click to interact with graphical elements.
• Windows are used as logical container for an application’s input/output.
• Cut-Copy-Paste is used as a paradigm for manipulating data (originally test, but we’re accustomed to copy-pasting images and other binary objects as well).
• Undo-Redo as a paradigm for reverting changes.

Modern desktop operating systems, including Linux, macOS and Windows, are remarkable similar to these early designs. Very little has actually changed in terms of how we interact with desktop systems.

How have graphical applications and operating systems stayed so fixed in design over almost 50 years? There are measurable benefits to having the users environment be familiar, and predictable. It turns out that most users just want things to remain the same.

Development snapshot

There was a scramble as vendors delivered tools and development environments that could deliver interactive, highly graphical applications. OO programming with C++ dominated through the early 90s, but when Java was introduced in 1996 it became an instant hit and quickly became a favorite programming language. For a few years at least ¹.

1990s: The Internet

Prior to about 1995, most applications were standalone. Businesses might have their own internal networks, but there was very limited ability to connect to any external computers e.g., you might be able to connect to a mainframe at work, but not to a mainframe across the country. You might have a modem at home, but it was used to connect to a local bulletin board system only. The rise of the Internet in the 1990s changed this. Suddenly, it was possible to connect to any computer in the world, and to share information with anyone, anywhere.

The Internet was originally conceived around a series of protocols e.g., FTP, Gopher. The World Wide Web (WWW) simply reflected one way that we expected to share information and services. However, it was definitely the most approachable way, and public demand led to the WWW dominating user’s experience of what a network could do. This led to the rise of web applications – applications that run in a web browser, connecting to one or more remote systems.

This shift towards the World Wide Web happened at a time when developers were struggling to figure out how to handle software distribution more effectively. It also provided a single platform (the WWW) that could be used to reach all users, regardless of operating system, as long as they had a web browser installed. For better or worse, the popularity of the WWW pushed software development towards using web technologies for application development as well. We’re still dealing with that movement today (see Electron and the rise of JS frameworks for application development).

Development Snapshot

2000s: Mobile-first

Prior to 2007, most home users had one or more home PCs, and possibly a notebook computer for work (and probably a few game consoles or other devices). Phones were just devices for making phone calls.

The iPhone was launched in 2007, and introduced the idea of a phone as a personal computing device. Society quickly adopted them as must-have devices. Today, there are more smartphone users than desktop users, and the majority of Internet traffic comes from mobile devices (99.9% of it split between iPhones and Android phones).

This has led to the notion of mobile-first design, where applications are designed primarily for mobile devices. Depending on the type of software, desktop software may be a secondary concern (and delivered as a web site, or not at all). This is a significant change from the past, where personal computers used to represent the dominant sector.

Mobile devices have changed the way we think about and deliver software. Again.

The business of building software

Computing environments and the context in which we run software has changed radically in the last 15 years. It used to be the case that most software was run on a desktop or notebook computer. Tasks were means to be performed at a specific location, with dedicated hardware. In the present day, most software is run on mobile devices, with desktops being relegated to more specialized functionality e.g., word processing, composing (as opposed to reading) email.

Does this mean that we can just focus on mobile devices? Probably not. Many of your users will have multiple devices: a notebook computer, a smartphone, and possibly a tablet or smartwatch, and they expect your software to run everywhere. Often simultaneously².

Platform challenges

This means that you will produce one or more of these application styles, depending on which hardware you target first.

1. Console application: Applications that are launched from a command-line, in a shell.
2. Desktop application: Graphical windowed applications, launched within a graphical desktop operating system (Windows, macOS, Linux).
3. Mobile application: Graphical application, hosted on a mobile operating system (iOS, Android). Interactions optimized for small form-factor, leveraging multi-touch input.
4. Web applications. Launched from within a web browser. Optimized for reading short-medium block text with some images and embedded media.

Additionally, your application is expected to be online and network connected. Application data should reside in the cloud, and synchronize between devices so that you can can switch devices fluidly.

How we develop applications

So, as developers, how has software development changed? It application development the same now as it was in the 1970s? 1990s? As you might expect, software development as a discipline has had to change drastically over time to keep up with these innovations.

• Console development is relatively simple, and often doesn’t need much more than a capable compiler, and standard libraries that are included with the operating system.
• Desktop development added graphical user interfaces, which in turn, require more sophisticated libraries to be included with the operating system. Eventually we needed to start adding other capabilities e.g., networking support.
• Mobile development expanded this even further, adding support for highly specific hardware capabilities e.g., cellular modem, gyroscope, fingerprint sensors.

Modern applications then, are typically graphical, networked, and capable of leveraging online resources. They must also be fast, visually appealing, and easy to use.

Let’s examine how we achieve this. We can consdier application development as the interplay between the operating system, a programming language, and libraries that bridge these two things and support our required functionality.

The role of the operating system

An operating system is the fundamental piece of software running on your computer that manages hardware and software resources, and provides services to applications which run on it. Essentially, it provides the environment in which your application can execute. The core of an operating system is the kernel, the main process which runs continuously and manages the system.

The role of an operating system includes:

• Allocating and managing resources for all applications, and controlling how they execute. This include allocating CPU time (i.e. which application is running at any given time), and memory (i.e. which application has access to which memory).
• Providing an interface that allows applications to indirectly access the underlying hardware, and does this in such as way that each application is isolated from the others.
• Providing higher-level services that applications can use e.g., file systems, networking, graphics. The OS abstracts away the underlying hardware so that applications don’t need to be concerned with the underlying implementation details i.e. you don’t need to write programs that directly interact with a hard drive, or a particular graphics card.

This illustrates the relationship between the operating system and applications running on it:

Our application stack includes multiple layers. Each layer can only call down into the layer directly below it. e.g., applications do not have direct access to hardware. Instead, they must use system libraries to request hardware capabilities from the underlying operating system (which in turn manages the hardware).

Layers are further subdivided based on their level of privilege/access:

• User mode represents code that runs in the context of an application. This code does not have direct access to the hardware, and must use system libraries and APIs to interact with the hardware. Applications and many libraries only exist and execute at this level.
• Kernel mode represents private, privileged, code that runs in the context of the operating system. This code has direct access to the hardware, and is responsible for managing the hardware resources. The OS kernel is the core of the operating system, and provides the fundamental services that applications rely on. Some system-level software can also execute at this level.

Notice that the kernel of the operating system runs in a special protected mode, different from user applications. Effectively, the core operating system processes are isolated from the user applications. This is a security feature which prevents user applications from directly accessing the hardware, and potentially causing damage to the system.

Programming languages rely heavily on the underlying operating system for most of their capabilities. In many ways, you can think of them as a thin layer of abstraction over the operating system. For example, when you write a file to disk, you are actually making a system call to the operating system to write that file. When you read from the network, you are making a system call to the operating system to read that data.

Why is this layering important?

• Applications have limited ability to affect the underlying OS, which makes the entire system more stable (and secure).
• We can abstract the hardware i.e. decouple our implementation from the exact hardware that we’re using so that our code is more portable.

Programming languages

Your choice of programming language determines the capabilities of your application. You also need to make sure that you pick a programming language that is suitable for the operating system you’re targeting.

We can (simplistically) divide programming languages into two categories: low-level and high-level languages.

Low-level languages are suitable when you are concerned with the performance of your software, and when you need to control memory usage and other low-level resources. They are often most suitable for systems programming tasks, that are concerned with delivering code that runs as fast as possible, and uses as little memory as possible. Examples of systems languages include C, C++, and Rust.

Systems programming is often used for operating systems, device drivers, and game engines. Low-level languages are obviously tightly-coupled to the underlying OS.

High-level languages are suitable when you are concerned with the speed of development, and the robustness of your solution. Performance may still be important, but you are willing to trade off some performance relative to a low-level language, for these other concerns.

High-level languages are well-suited to applications programming , which can often make performance concesssions for increased reliability, more expressive programming models, and other language features. In other words, applications tend to be less concerned with raw performance, and so we can afford to make tradeoffs like this ³.

These languages are also less coupled to the underlying OS (at least in theory).

Examples of application languages include Swift, Kotlin, and Dart. Applications programming languages are often used for web applications, mobile and desktop applications. They may also be used when creating back-end services e.g. web servers.

Note that is is certainly possible to develop applications in low-level languages, and to develop systems software in high-level languages. However, the choice of language is often driven by the requirements of the software you are building.

Language tradeoffs

So what kinds of benefits do we get from high-level languages?

Modern application languages like Swift, Kotlin and Dart share similar features that are helpful when building applications:

Functional programming features. Modern languages feature support for functional features like lambdas, higher-order functions, and immutability. These features can make your code more expressive, and easier to understand. Theses are in addition to the traditional object-oriented features that you would expect from a modern language.

Memory management. Garbage collection is common with application languages, where you want to optimize for development time and application stability over performance. Garbage collection is simply a paradigm where the language itself handles memory allocation and deallocation by periodically freeing unused memory at runtime. GC results in a system that is significantly more stable (since you’re not manually allocating/deallocating with all of the risks that entails!) but incurs a runtime performance cost. The majority of the time, the performance difference is negligible⁴.

NULL safety. Swift, Kotlin and Dart have null safety built in, which can prevent null pointer exceptions. For example, you cannot access uninitialized memory in Kotlin, since the compiler will catch this error at compile time.

Runtime performance. Application languages tend to be slower than systems languages, given the overhead of GC and similar language features. Practically, the difference is often negligible at least when working with application software.

Portability. Systems languages are often “close to the metal”. However, being closely tied to the underlying hardware also make it challenging to move to a different platform. Application languages can be designed in a way that leverages a common runtime library across platforms, making it simpler to achieve OS portability. Note that this is not always the case; Swift, for example, is only available on Apple platforms.

Using libraries

You could (in theory) write software that directly interacts with the operating system using system calls⁵. However, this is not practical for many situations, since you are then tied to that particular operating system (e.g. Windows 10). Instead, we use system libraries, which are a level of abstraction above system calls that provide a higher-level interface, to your programming language. This is the level at which most applications interact with the operating system.

Vendors and OSS communities also usually provide user libraries, which are simply libraries that run in user-space and can be linked into your application to provide high-level functionality. Accessing a database for instance isn’t handled by the operating system directly, but by a user library that abstracts away the details of the database and provides a simple interface for your application to use.

We’ll make extensive use of libraries in this course! We’ll use system libraries to interact with the operating system, and user libraries to provide higher-level abstractions that make it easier to build applications.

Technology stacks

Given what we now know about programming languaged and libraries: which software stack is “the best one” for building applications?

Is there a “best one”?

Let’s consider which technologies are commonly used:

Why are there so many different technology stacks!? Why isn’t there one language (and set of libraries) that works everywhere?!

In a word, competition.

Development tools, including programming languages, are typically produced by operating system vendors (Microsoft for Windows, Apple for macOS and iOS, Google for Android). For each vendor, their focus is on producing development tools to make it easy to develop on their platforms. Apple wants consumers to run macOS and iOS, so they developed Swift and the supporting frameworks to make it easy for you to build applications on their platforms. Microsoft spend billions producing C# to make development easier for Windows easier; it would be against their interests to support it equally well on macOS.

As a result, each platform has it’s own specific application programming languages, libraries, and tools. If you want to build iOS applications, your first choice of technologies should be the Apple-produced toolchain i.e. the Swift programming language, and native iOS libraries. Similarly, for Android, you should be using Kotlin and the native Android libraries.

What do we do?

So, choose technologies is complicated. In our case, we have the luxury of picking something that is interesting (as opposed to “something that the team knows”, or “what your manager tells you to use”).

We’ll start with a platform, and then narrow down from there:

1. Pick a platform: Android (mobile).
2. Pick a programming language: Kotlin (high-level, modern).
3. Sprinkle in libraries to cover expected functionality.

• graphical user interfaces: Compose
• networking: Ktor
• databases, web services: specific to our cloud provider.

We’ll explore all of this as we progress through the course.

Last word

https://imgs.xkcd.com/comics/old_days.png

Project management

Topics related to managing the project itself.

Teamwork

Credit for this section belongs to Michael Liu and members of the Teamwork Skills Project. Their materials have been adapted for this course.

Assembly

In this section, we’ll discuss the importance of teamwork and how to build a successful team. Why are we learning this?

• You will work in teams in this course, and likely in your future career.
• You will experience conflict at some point.
• You will have time management issues.
• You will have team motivation challenges.

We want you to be prepared for these challenges and to have the tools to overcome them.

What are the attributes of a successful team?

Phases of team development

Teams don’t come together and become effective overnight. It takes time, and deliberate effort on the part of team members to build a successful team.

One common model of team development is Scott M. Graffius’ “Five Stages of Team Development”:

1. Forming: The team comes together and gets to know each other. This is a time of excitement and anticipation.
2. Storming: The team starts to work together and differences in opinion may arise. This is a time of conflict and disagreement.
3. Norming: The team starts to resolve their differences and work together more effectively. This is a time of cooperation and collaboration.
4. Performing: The team is now working together effectively and achieving their goals. This is a time of high productivity and success.
5. Adjourning: The team has completed their project and is ready to move on. This is a time of reflection and celebration.

Team meetings and team contracts can help the team move through these stages more quickly.

Step 1: Team contract

A team contract is a document that outlines the expectations and responsibilities of each team member. It is a tool that helps the team establish a shared understanding of the project and the roles of each team member.

The exact content is decided by the team, but the contract typically addresses:

• Team leadership and communication
• Team meeting expectations
• Team & individual expectations
• Managing team challenges and conflict
• Other considerations

It may include contract clauses such as:

• “All members will attend meetings or notify the team by email or phone in advance of anticipated absences.”
• “All members will be fully engaged in team meetings and will not work on other assignments during the meeting.”
• “All members will come to meetings prepared by reading the assigned material and/or watching assigned lecture videos ahead of time.”

See the Public Gitlab Repo > Templates for a sample team contract.

Step 2: Define team roles

It is useful to determine team roles at the start of a project, to help guide the team’s work. Roles can include:

1. Team Lead: This is the project leader, NOT the technical leader for the project. This person is responsible for keeping the team on track and ensuring that the project is completed on time. They do this by ensusing that the requirements are defined accurately, that the team is tracking work properly, and that the team is meeting its goals for each deliverable. They often define the agenda for the team meeting, take minutes and ensure that the team is following the team contract.

2. Technical Lead: This person is responsible for ensuring that the technical aspects of the project are completed correctly. They are responsible for ensuring that the team is following best practices, that the code is complete and follows course guidelines. In the way that the team lead helps manage the project, the techical leader helps manage the code contributions of the team. They often review code, help team members debug, and ensure that code is reviewed and merged properly.

3. Design Lead: Teams (at least in this course) may have a Front-End Design Lead, and a Back-End Design Lead, each of which is responsible for the design and delivery of a core part of the product.

Note that these roles are NOT prescriptive. They are meant to guide the team in their work. The team should be flexible and adapt to the needs of the project. It’s normal for someone to wear multiple hats e.g. design lead for the front-end, but also handling packaging (because they may have done that before).

This means that, for example:

• Team leads still write code. Managing the project is not a full-time job.
• Technical leads can still take meeting minutes and write code. They are not just code reviewers.
• Design leads can still help with other parts of the project outside of their immediate area of responsibility.

No single person is “the boss” of the project, and significant decisions should be made with the input of the entire team.

Step 3: Project planning

A project plan is a document that outlines the tasks, resources, and timeline for a project. It helps the team stay on track and meet deadlines. This can help with time management issues.

Identifying project components

Identifying what needs to be done should be, at least initially, a team effort. The team should brainstorm and list all the tasks that need to be completed.

This is a good time to use a whiteboard or a shared document to list all the tasks. This can be done in a team meeting or asynchronously.

Given an Agile framework, you should expect to review and update this list regularly.

This is an ongoing activity as you learn more about your project, and make design decisions.

Creating a Gantt chart

A Gantt chart should be included in your project proposal to summarize your project components.

This type of chart lists high-level objectives down the left side and dates across the top. Each task is represented by a bar that spans the dates when the task is planned to be completed.

https://www.productplan.com/glossary/gantt-chart/

See Mermaid for an example of how to create a Gantt chart in Markdown.

Communication

Communication is the key to success in any project! This includes communication between the customer, team members, and stakeholders.

Source: Unknown

In this course, you won’t be speaking directly with the customer, but you will be communicating with your team members and other stakeholders (like your instructor and TA). It’s important to communicate effectively and efficiently to ensure that everyone is on the same page and working towards the same goals.

Team meetings

Team meetings are an essential part of any project. They provide an opportunity for team members to discuss progress, share ideas, and make decisions. Communication is most effective when it’s both written and oral, so it’s important to have regular team meetings.

However, meetings are expensive in terms of time and resources. They are most effective when you have a clear goal for the meeting. For each meeting:

• You should have an agenda: what are you going to discuss? This can be as simple as a set of bullet points for discussion, or as detailed as a design for a feature where you require input. A team lead will typically prepare and circulate the agenda ahead of time.
• Everyone should come to the meeting prepared to discuss topics on the agenda. This means reading any required materials, watching lecture videos etc.
• The project lead should keep the team organized around the agenda and ensure that the meeting stays on track. Make sure you accomplish what you intended to accomplish.
• Someone should be designated to take notes during the meeting. These notes should be circulated to the team after the meeting.

Here’s a sample of an agenda for a team meeting¹:

Informal communication

It’s often helpful to have other ways to communicate with your team members. This can include:

• Messaging applications e.g., Slack, Discord, Microsoft Teams.
• Shared online spaces e.g., Google Docs, MS Teams channels.
• Email. e.g., your UW email account.

These tools can be used to share information, ask questions, and discuss ideas. They can also be used to share files and other resources.

These tools are extremely helpful for day-to-day communication, but they should not replace regular team meetings. They are most effective when used in conjunction with regular meetings.

Conflict management

Conflict is a natural part of work on a team. It can arise from personality differences, differences in opinion, or differences in work style.

There are seven main sources of potential conflict on a team: t

1. Work Scope: Differences of opinion on how the work should be done, how much should be done, or the appropriate level of quality.
2. Resource Assignments: The particular individuals assigned to work on certain tasks, or the quantity of resources assigned to certain tasks.
3. Schedule: The sequence in which the work should be completed, or about how long the work should take.
4. Cost: How much the work should cost (n/a for this course).
5. Priorities: People being assigned multiple conflicting tasks, or when various people need to use a limited resource at the same time.
6. Organizational: Disagreement over the need for certain procedures, or ambiguous communication, or failure to make timely decisions.
7. Stakeholder: Issues with certain stakeholders e.g. disagrement with the instructor.
8. Personal: Differences in individual values and attitudes on the team.

Conflict can be beneficial to a team, as it can lead to better decision-making and more creative solutions. However, if not managed properly, it can also lead to stress and demotivation.

Handling conflict

There are five main approaches to handling conflict:

1. Avoiding or Withdrawing. Individuals in conflict retreat from the situation in order to avoid an actual or potential disagreement. This approach can cause the conflict to fester and then escalate at a later time.
2. Competing or Forcing. In this approach, conflict is viewed as a win–lose situation in which the value placed on winning the conflict is higher than the value placed on the relationship between the individuals. This approach to handling conflict can result in resentment and deterioration of the work climate.
3. Accommodating or Smoothing. This approach emphasizes finding areas of agreement within the conflict and minimizes addressing differences. Topics that may cause hurt feelings are not discussed. Although this approach may make a conflict situation livable, it does not resolve the issue.
4. Compromising. Team members search for an intermediate position. The solution may not be the optimal one.
5. Collaborating, Confronting, or Problem Solving. Team members confront the issue directly, with a constructive attitude, and look for a win–win outcome. They place high value on both the outcome and the relationship between the individuals. For this approach to work, it is necessary to have a healthy project environment.

In a particular conflict situation, you will tend to have a preference for one style over the others depending on the dynamic of the team and your personality.

By understanding the consequences and impact of each of the styles, we can make more informed and productive decisions about how to approach a conflict situation.

These strategies differ along two dimensions (how much it values the individual need vs. how much it values collective need):

• Cooperation scale: value placed on maintaining relationships
• Assertiveness scale: value placed on addressing individual goals or concerns

In a team environment, we recommend team-friendly stragies: accommodating, compromising or collaborating. This is in-part why we enforce regular team meetings, and decision-making by consensus; it helps ensure that everyone’s opinion is considered and that decisions are made in a way that is fair to everyone.

The end goal of a conflict resolution process is to explore different options and comes up with the best (or mutually acceptable) idea.

Being Agile

Let’s discuss the process of developing software.

People often think that building software is like building anything else, e.g., a car. At first glance is seems reasonable: you determine some requirements for what you want to build, design it, and then build a prototype. You refine your production process, and eventually you can mass-produce as many of these cars as you want.

However, if we attempt to apply these concepts to software, they don’t work. Designing and building software is subtly different than building other products.

Software Development Lifecycle

The Software Development Lifecycle (SDLC) is a simple model that we use to describe the steps of a software project, from inception to a final working product.

The model has evolved over the last few decades, and is similar to other project tracking processes. It usually prescribes a series of activities like these:

• Planning - “What are high level goals?”, “What is our budget?”, “Who is working on the project?”
• Requirements definition - “Who are our users?”, “What problem are we solving?”
• Analysis & design - “What technical constraints exist?”
• Implementation - “How do we build it?”
• Testing - “Does it meet specifications?”
• Deployment - “How do we keep it running properly?”

Formally, we consider the SDLC to be a software process model: a collection of activities, actions and tasks that we perform when creating software.

This particular model is also referred to as a waterfall model, named for the way in which you are supposed to move through each activity in sequence.

---
title: Waterfall Model
config:
    theme: neutral
---
flowchart LR
    A[Planning] --> B[Requirements]
    B[Requirements] --> C[Analysis & Design]
    C[Analysis & Design] --> D[Implementation]
    D[Implementation] --> E[Testing]
    E[Testing] --> F[Deployment]

The implications of using a waterfall model:

• A project starts at the first activity and advances through stages. Each stage must be completed before the next stage begins.
• Stages are typically modelled after organizational units that are responsible for that particular stage (e.g. Product Management owns Requirements, Architects own Analysis & Design, QA owns Testing and so on).
• There are criteria that need to be met before the project can exit one stage and enter the subsequent stage. This can be informal (e.g. an email letting everyone know that the design is “finished”), to a more formal handoff that includes artifacts (e.g. Product Requirements documents, Design documents, Test Plans and so on that must be formally approved).

This linear approach strongly suggests that you can and should define a project up-front (i.e. determine cost, time and so on before starting the project).

What’s wrong with this model?

If you’re a project manager, or a business owner, the Waterfall model is very appealing: it suggests that you can define work up-front, and plan out time, costs and other factors. This model suggests that you can work out the parameters of a project, then execute on that plan. Any failure to meet a deadline would be a failure of people to complete their stages ‘on-time’.

However, this isn’t realistic, at least not for software projects.

1. Requirements can and will change as project development is underway. Often the context in which the product is being developed will change in ways we cannot anticipate (e.g., the market changes, so new features need to be added). Our understanding of the problem that we’re solving will also naturally evolve as we’re building something, which can result in unexpected requirements changes as well.
2. Design and implementation activities are not that predicable! We are often building new features or combinations of features. Managing novelty is always going to be challenging and include some level of uncertainty (i.e., how long it will take, and what the solution will look like).

We knew in the 1970s that the Waterfall model didn’t work, and spent 20 years trying to come up with a better solution. By the mid-1990s, there was a widespread recognition that this way of building software just didn’t work – either for developers or for customers.

• Developers were frustrated by rigid processes and changing requirements.
• Business owners were frustrated by the inability to make changes to projects in-flight.
• Projects were being delivered late and/or over-budget. This frustrated everyone!

There are a large number of software process models that were developed at this time, including Extreme Programming (XP) (Beck 1999), Scrum (Schwaber & Sutherland 1995), Lean ( Poppendieck & Poppendieck 2003). No single model dominated.

Instead, the frustrating state of software development led to a a group of software developers, writers, and consultants meeting and delivering the Manifesto for Agile Software Development in 2001.

Agile was an attempt to find some common-ground principles across competing process models. It isn’t a process model by itself, but a set of principles that influenced the development of other models.

The Agile Manifesto

The Agile Manifesto (agilemanifesto.org)

What does it mean?

1. Individuals and interactions (over processes and tools): Emphasis on communication with the user and other stakeholders.
2. Working software (over comprehensive documentation): Deliver small working iterations of functionality, get feedback and revise based on feedback. You will NOT get it right the first time.
3. Customer collaboration (over contract negotiation): Software is a collaboration between you and your stakeholders. Plan on meeting and reviewing progress frequently. This allows you to be responsive and correct your course early.
4. Responding to change (over following a plan): Software systems live past the point where you think you’re finished. Customer requirements will change as the business changes.

Agile Software Development is used to describe any process that encompasses this philosophy. Agile encourages team structures and attitudes that make communication easier (among team members, business people, and between software engineers and their managers). It emphasizes rapid delivery of operational software, but also recognizes that planning has its limits and that a project plan must be flexible.

Iterative processes

This idea of an iterative, evolutionary development model remains central to all Agile processes (although they may present it differently). Instead of building a “complete” system and then asking for feedback, we instead attempt to deliver features in small increments, in a way that we can solicit feedback continuously though the process. Over time, we will add more features, until we reach a point where we have delivered sufficient functionality and value for the customer.

Agility forces us to continually reassess the state of the project, and supports the idea that decisions can unfold over time. The most significant project benefit of Agile is that it reduces (and tries to eliminate) breaking late-project changes.

Being Agile

Although we often refer to Agile as a singular model, it is actually an umbrella: a philosophy that drives a number of other models, such as Kanban, Lean and Scrum. In practice, we often find that software teams adopt practices from different methodologies, changing them where it makes sense.

We’ll do the same thing here. We’ll reconstruct our SDLC to be a two-stage iterative model.

We have 2 distinct phases:

• An early planning and design phase, where we do basic project planning, define our users, and their requirements. This is also where we propose early solutions and get early feedback to narrow down our decisions. We’ll discuss this further under Design Thinking.
• A second development phase, where we deliver the functionality that we proposed in the first phase, to meet our user goals. We will spend the remainder of the course discussing topics for this phase.

Project repository

We want our software to be effective, useful, and long-lived. This means tracking all of the details of the project for both our own use, and for people that will maintain the project after us.

It’s normal to use Project Management Software to keep track of this information.

Typical responsibilities of this software include:

• Tracking progress: tracking the project details e.g., items in-progress, items that are complete.
• Resource management: who is on the project, what are they working on.
• Scheduling: when will items be completed.
• Information: project documents, user documentation.

GitLab

In this course, we’ll use GitLab. GitLab is a very “developer-focused” tool, which also provides support for:

• Tracking your source code in an associated Git repository.
• Storing project documents e.g., requirements, design documents, work in-progress, test results.
• Storing user documents e.g,. how to install and use it, features that are implemented.

Project tracking

We track the work to be done as issues, and assign groups of issues to project milestones (or deadlines).

Establish milestones

A milestone is a deadline that you are working towards. This could be a customer demo, or a product release, or any other deadline. In our course, sprints represent two-weeks worth of work, and each sprint ends with a demo. Each demo is considered a milestone in your project.

To setup your project to track milestones:

• Plan > Milestones, create milestones Sprint 1 through Sprint 4. Make sure to assign starting and ending dates to each of them - see the schedule for dates.

Here’s an example of milestones that have been created for a course project. Milestones are important because they allow us to track work against a particular goal. This reflects a similar structure to what you should use in this course.

Create issues

To manage work in a project, we first define a list of activities that we need to perform (i.e. the actual “work” that needs to be done). We refer to items of work as issues in GitLab.

An issue:

• Represents one meaningful “unit of work” that needs to be performed for our project.
• Can be anything that needs to be done e.g., programming, writing unit tests.
• Should represent a complete feature (or if the requirement is complex, one of a set of related features).
• Can be completed in less than one day.

To create an issue in GitLab:

• Plan > Issues, New Issue

An issue will typically have the following fields. You should make a best-effort attempt to fill these in when you create the issue:

• Title: A descriptive title that descibes the work, e.g., “Move model into shared project”.
• Comment: When you create a new issue, you are prompted to type a lengthier description. This will show as a comment in your issue at the bottom of the Activity pane. You can add additional comments as the issue unfolds, e.g., notes to yourself, or from teammates.
• Assignee: The person responsible for the work. Typically unassigned for new issues.
• Milestone: The milestone that this issue is assigned to. Typically blank for new issues.
• Due date: Can be left blank, since we’ll rely on milestone dates.

Additionally, you can link this issue to other related issues, e.g., multiple changes towards the same requirement. You can also choose to attach external files, e.g., log files for debugging, config files required to test the issue and so on.

Your project will have many issues associated with it! Typically you assign issues to a milestone, and then update them as you work on them.

Updating issues

To review and update issues together, the team can use a scrum board to visualize all the work being towards a milestone. Scrum boards can have multiple steps visible in the workflow, like To Do, In Progress, and Done. This can be very helpful in planning.

Tracking progress

Finally, you might want to use a Gantt chart to visually show your project’s progress.

GitLab does not support generating Gantt charts, but you can manually create them in Mermaid.js and add to your documentation.

gantt
    title A Gantt Diagram
    dateFormat  YYYY-MM-DD
    section Section
    A task           :a1, 2014-01-01, 30d
    Another task     :after a1  , 20d
    section Another
    Task in sec      :2014-01-12  , 12d
    another task      : 24d

Documentation

Technical documentation is an important part of your project! Although you aren’t expected to write exhaustive documentation most of the time, we often need to record important ideas, or communicate with other team members. Usable documentation really helps.

Examples of useful documents that you will likely encounter in a software project include:

• Project documents, detailing project constraints, timelines, and so on.
• Requirements documents, detailing the requirements that you are expected to deliver.
• Design documents that explain how some critical system works. You should be writing these when planning out features: both as a way to share your design with your team-mates, and as a way to record the details for reference later (remember: we’re always plan to maintain systems long-term).
• User documents, explaining how to use your software.
• Product release documents, including Release Notes, README files.

We’ll discuss using Markdown for technical documents, and Mermaid.js for embedding diagrams.

Markdown

Markdown is a markup language that allows you to add formatting elements to a text file. The elements that you can add are reminiscent to HTML elements, which makes sense - Markdown was designed to be easily convertable to HTML (which is itself also a markup language). The basic syntax is here.

Syntax

Markdown supports the following tags:

When Markdown is used in GitLab, either in a file or a Wiki page, the renderer is able to display formatted text! This makes it beneficial when writing technical documentation.

Example

For example, in my editor I can add a paragraph with a bunch of styling:

#### Sub-sub-sub-sub-heading
I can write some:
* **bold** text
* _italicized_ text

I can even add an image.
![image caption](architecture-needs.png)

This gets displayed in my editor’s preview window like this:

Authoring a Markdown document can be done in any text editor, but you might appreciate an editor that has syntax highlighting and a live preview. These all work well:

• IntelliJ IDEA with a plugin
• VS Code
• Typora.

Diagrams (Mermaid.js)

We often add diagrams to documents to clarify and communicate additional information. These can include Gantt charts or burn-down charts for project tracking; illustrative graphs when determining requirements, and a variety of diagrams during the design and implementation phases.

Mermaid is a Javascript-based charting tool that can convert Markdown diagramming notation into diagrams.

The advantage to using Mermaid.js is that your diagrams are completely described in Markdown. GitLab also renders Mermaid.js diagrams easily, so you can use them directly in your project README file, or any of your Wiki pages to render inline diagrams.

The editors described above all support Mermaid.js rendering. You might also want to try the Mermaid live editor. It generates raw mermaid syntax, which you can then add to an existing Markdown document.

To render diagrams in Markdown, they need to be embedded as a code block element. Markdown uses three backticks to start and end a code-block. After the first three backticks, you enter the language name, in this case mermaid, followed by the diagram syntax, and then three final backticks to close it.

The diagram syntax is a diagram type, followed by a flow indicator. e.g. flowchart LR for a flowchart flowing left-to-right (indicators include LR, RL, TB, BT). For example, here’s a simple flowchart, placed in the middle of some text in a Markdown document.

This would produce output like this:

Flowchart

Here’s a flowchart diagram showing program execution.

flowchart LR
    A[Start] --> B{Is it?}
    B -->|Yes| C[OK]
    C --> D[Rethink]
    D --> B
    B ---->|No| E[End]

flowchart LR
    A[Start] --> B{Is it?}
    B -->|Yes| C[OK]
    C --> D[Rethink]
    D --> B
    B ---->|No| E[End]

flowchart TD
    Auth{"User Authenticated?"} -- No --> LoginScreen["Show Login Screen"]
    LoginScreen -.-> Auth
    Auth -- Yes --> MainMenu["Add"]
    MainMenu --> Logout["Details"]
    Logout --> Exit(["End"])
    Start(["Start"]) --> Auth
     Auth:::Sky
     Exit:::Rose
     Start:::Rose
     Start:::Aqua
    classDef Sky stroke-width:1px, stroke-dasharray:none, stroke:#374D7C, fill:#E2EBFF, color:#374D7C
    classDef Aqua stroke-width:1px, stroke-dasharray:none, stroke:#46EDC8, fill:#DEFFF8, color:#378E7A
    classDef Rose stroke-width:1px, stroke-dasharray:none, stroke:#FF5978, fill:#FFDFE5, color:#8E2236

User Journey Diagram

User Journey Diagrams are a useful way to show a series of steps that a user might take to perform an operation in your software. These could be used to model User Stories.

journey
    title My working day
    section Go to work
      Make tea: 5: Me
      Go upstairs: 3: Me
      Do work: 1: Me, Cat
    section Go home
      Go downstairs: 5: Me
      Sit down: 5: Me

journey
    title My working day
    section Go to work
      Make tea: 5: Me
      Go upstairs: 3: Me
      Do work: 1: Me, Cat
    section Go home
      Go downstairs: 5: Me
      Sit down: 5: Me

Gantt Chart

Here’s the Gantt chart syntax. Gantt charts are the most common project management charts, great for showing milestones and critical project dependencies.

gantt
    title A Gantt Diagram
    dateFormat  YYYY-MM-DD
    section Section
    A task           :a1, 2014-01-01, 30d
    Another task     :after a1  , 20d
    section Another
    Task in sec      :2014-01-12  , 12d
    another task      : 24d

gantt
    title A Gantt Diagram
    dateFormat  YYYY-MM-DD
    section Section
    A task           :a1, 2014-01-01, 30d
    Another task     :after a1  , 20d
    section Another
    Task in sec      :2014-01-12  , 12d
    another task      : 24d

Timeline diagram

Here’s a timeline diagram, which is also useful for project tracking.

timeline
    title History of Social Media Platform
    2002 : LinkedIn
    2004 : Facebook
         : Google
    2005 : Youtube
    2006 : Twitter

timeline
    title History of Social Media Platform
    2002 : LinkedIn
    2004 : Facebook
         : Google
    2005 : Youtube
    2006 : Twitter

Mindmap

Here’s a mindmap of the course!

mindmap
  root((Topics))
    (Design)
        Requirements
        Design thinking
        Personas
        User stories
        Prototyping
    (Practices)
        Agile
        Branching
        Build configuration
        Pair programming
        Unit testing
        Documentation
    (Technologies)
        Gradle
        JUnit
        Docker
        Markdown
        Mermaid
    (Development)
        Kotlin programming
        OO programming
        Functional programming
        Databases
        Concurrency
        Web services
        Cloud hosting

mindmap
  root((Topics))
    (Design)
        Requirements
        Design thinking
        Personas
        User stories
        Prototyping
    (Practices)
        Agile
        Branching
        Build configuration
        Pair programming
        Unit testing
        Documentation
    (Technologies)
        Gradle
        JUnit
        Docker
        Markdown
        Mermaid
    (Development)
        Kotlin programming
        OO programming
        Functional programming
        Databases
        Concurrency
        Web services
        Cloud hosting

Technical Diagrams (UML)

The Unified Modelling Language (aka UML) is a modelling language consisting of an integrated set of diagrams, useful for designers and developers to specify, visualize, construct and communicate a design. UML is a notation that resulted from the unification of three competing modelling techniques:

1. Object Modeling Technique OMT (James Rumbaugh 1991) - was best for analysis and data-intensive information systems.
2. Booch (Grady Booch 1994) - was excellent for design and implementation. Grady Booch had worked extensively with the Ada language, and had been a major player in the development of Object-Oriented techniques for the language. Although the Booch method was strong, the notation was less popular.
3. OOSE Object-Oriented Software Engineering (Ivar Jacobson 1992) - featured a model known as Use Cases.

All three designers joined Rational Software in the mid 90s, with the goal of standardizing this new method. Along with many industry partners, they drafted an initial proposal which was submitted to the Object Management Group in 1997. This led to a series of UML standards driven through this standards body, with UML 2.5 being the current version.

The primary goals of UML include:

• providing users with a common, expressive language that they can use to share models.
• provide mechanism to extend the language if needed.
• remain independent of any particular programming language or development process
• support higher-level organizational concepts like frameworks, patterns.

UML contains a large number of diagrams, intended to address the needs to a wide range of stakeholders. e.g. analysis, designers, coders, testers, customers.

UML contains both structure and behaviour diagrams.

• Structure diagrams show the structure of the system and its parts at different level of abstraction, and shows how they are related to one another.
• Behaviour diagrams show the changes in the system over time.

Below, we’ll highlight the most commonly used UML diagrams. For more comprehensive coverage, see Visual Paradigm or Martin Fowler’s UML Distilled (Fowler 2004).

Class Diagram

The class diagram is a central modeling technique that runs through nearly all object-oriented methods. This diagram describes the types of objects in the system and various kinds of static relationships which exist between them.

There are three principal kinds of relationships which are important to model:

1. Association: Represent relationships between instances of types (a person works for a company, a company has a number of offices).
2. Inheritance: The most obvious addition to ER diagrams for use in OO. It has an immediate correspondence to inheritance in OO design.
3. Aggregation: Aggregation, a form of object composition in object-oriented design.

Mermaid also supports class diagrams.

classDiagram
    Animal <|-- Duck
    Animal <|-- Fish
    Animal <|-- Zebra
    Animal : +int age
    Animal : +String gender
    Animal: +isMammal()
    Animal: +mate()
    class Duck{
        +String beakColor
        +swim()
        +quack()
    }
    class Fish{
        -int sizeInFeet
        -canEat()
    }
    class Zebra{
        +bool is_wild
        +run()
    }

classDiagram
    Animal <|-- Duck
    Animal <|-- Fish
    Animal <|-- Zebra
    Animal : +int age
    Animal : +String gender
    Animal: +isMammal()
    Animal: +mate()
    class Duck{
        +String beakColor
        +swim()
        +quack()
    }
    class Fish{
        -int sizeInFeet
        -canEat()
    }
    class Zebra{
        +bool is_wild
        +run()
    }

Component Diagram

A component diagram depicts how components are wired together to form larger components or software systems. It illustrates the architectures of the software components and the dependencies between them. Those software components include run-time components, executable components, also the source code components.

Deployment Diagram

The Deployment Diagram helps to model the physical aspect of an Object-Oriented software system. It is a structure diagram which shows the architecture of the system as deployment (distribution) of software artifacts to deployment targets. Artifacts represent concrete elements in the physical world that are the result of a development process. It models the run-time configuration in a static view and visualizes the distribution of artifacts in an application. In most cases, it involves modelling the hardware configurations together with the software components that lived on.

Use Case Diagram

A use-case model describes a system’s functional requirements in terms of use cases. It is a model of the system’s intended functionality (use cases) and its environment (actors). Use cases enable you to relate what you need from a system to how the system delivers on those needs. The use-case model is generally used in all phases of the development cycle by all team members and is extremely popular for that reason.

Keep in mind that UML was created before UX. Use Cases are very much like User Stories, but with more detail. Think of Use Cases == User Stories, and Actors == Personas (except that they can also model non-human actors like other systems).

Activity Diagram

Activity diagrams are graphical representations of workflows of stepwise activities and actions with support for choice, iteration and concurrency. It describes the flow of control of the target system. Activity diagrams are intended to model both computational and organizational processes (i.e. workflows).

I use these a lot when designing interactive systems, and modelling transitions between screens or areas of functionality.

State Machine Diagram

A state diagram is a type of diagram used in UML to describe the behavior of systems. State diagrams depict the permitted states and transitions as well as the events that effect these transitions. It helps to visualize the entire lifecycle of objects.

Mermaid also supports state diagrams.

stateDiagram-v2
    [*] --> Still
    Still --> [*]

    Still --> Moving
    Moving --> Still
    Moving --> Crash
    Crash --> [*]

stateDiagram-v2
    [*] --> Still
    Still --> [*]

    Still --> Moving
    Moving --> Still
    Moving --> Crash
    Crash --> [*]

Sequence Diagram

A Sequence Diagram models the collaboration of objects based on a time sequence. They show how objects interact over time and are great for showing ordering effects.

When do you use a Sequence Diagram over a more simple Activity Diagram? Use it when you have a more complex interaction between components. I’ve used these to model client-server authentication, for instance, or passing data between systems (where it might need to be encrypted/decrypted on each side).

Mermaid does a great job of sequence diagrams as well.

sequenceDiagram
    Alice->>Bob: Hello Bob, how are you?
    Bob-->>Alice: Great!
    Alice-)Bob: See you later!

sequenceDiagram
    Alice->>Bob: Hello Bob, how are you?
    Bob-->>Alice: Great!
    Alice-)Bob: See you later!

There are various software packages on the market for creating UML diagrams.

• Figma: Figma Jam Boards have UML templates. They are very simple, but work for basic diagrams. Using Figma for this has the advantage of a single tool for all project diagrams.
• Mermaid.js: A library that can be used to draw inline diagrams in Markdown.
• Draw.io: A free web-based tool for drawing rich UML diagrams.

Design thinking

How do we work through the initial design decisions for our project?

We all want to produce exceptional products. What does that mean?

Of course, our product should be useful: it should satisfy a need or desire for our users. We should be able to articulate what that need is, and specifically how our product will address it.

However, that is not enough for it to be successfully adopted. We don’t want an adequate user experience, we want an outstanding user experience, both as designers and consumers.

Our product needs to be approachable, easy to learn, simple to use, and ultimately satisfying to our users. In a world of competing solutions, it’s not enough for a product to just be “good enough”, it has to be outstanding and that means designing for all of our users wants and needs.

Design thinking provides a framework for doing exactly this.

Design thinking is a way of thinking about how to approach the problem of building something suitable for people. It’s both an ideology (designs should be human-focused and consider human-factors), and a process (how to do this efficiently and effectively by iterating over potential solutions).

Design ideology

Design thinking is a way to think about designing for people. It can be used to describe a process for building anything, from a clock-radio, to an electric vehicle, to an software application. It is commonly used when designing complex, human-centred software systems and is particularly effective when you want to solve a problem in a new and interesting way.

Design process

The design thinking process involves these steps, all of which are done before you start implementation. As a team, you work through these steps together.

This is an iterative process; teams will often revisit earlier decisions as they progress through this process.

UNDERSTAND phase: We take steps to understand the user, their context and potential problems that they might be experiencing.

1. Empathize: Develop knowledge about your users, and their context. You goal is to gather enough information that you understand your users, how they work, and what motivates them.

2. Define. Look at the information that you’ve gathered, and observe where your user problems exist. Look for opportunities.

EXPLORE phase: Brainstorm ways to solve their problems. Consider alternative solutions.

3. Ideate. Brainstorm ideas to solve the problem! You want lots of ideas, from any source (including competing products, things that “sound like a good idea”, or direct suggestions from users). Discuss them all and narrow down to the ones that work best.

4. Prototype. Work together to build mockups of your “best attempt” at a solution. Use the process to determine which components of your solution work and which ones need further refinement. Some ideas might not work together, and need to be rejected - that’s ok too.

MATERIALIZE phase. Iterate over potential solutions with your users, and collect their feedback.

5. Test. Test with users by showing them the prototype and collect their feedback. Identify problems and areas to improve. Iterate on your designs!
6. Implement. Implement your final design solution.

We’ll explore each of these steps in further detail below.

Phase I: Understand

1. Empathize

The first step to building a solution for someone is to understand who they are, what challenges and constraints they face. You want to be able to clearly state:

• “Who are my users?”
• “What are they trying to accomplish?”
• “What motivates them?”
• “What discourages them?”

Your goal is to understand and empathize with your users and their perspectives.

Action: Conducting interviews

The best way to understand a set of users is to talk to them! You need to interview people to understand them, their perspective (how they work, what they like, what they dislike and so on).

Interview guidelines

• Interview 5–6 users (Nielsen 2000)
• Ask open-ended questions to understand what they do, how they work.
• Continue asking questions to identify issues or challenges that they might have.
• Use followup questions to make sure that you understand their goals and motivations.

The outcome from interviews should be detailed interview notes! It is critical that you write down everything that the user says, so that you can analyze it in the next step.

It can also be helpful at this stage to characterize your users by creating personas.

Action: Generating personas

A persona is an archetypal description of a user of your product. Personas are often based on “real people” but are meant to be generalized descriptions that represent a category of people that share similar characteristics. We use personas a stand-ins when discussing product goals and features.

You may have multiple personas, each “type” representing users with different roles, or who you think are distinct. It’s not uncommon to have two or three personas even for a small solution.

There are some basic elements that your user persona should include:

• Personal info: Present personal information first, including (fictional) name, job title, company, job description, family status, and more. Include any details that can help your team understand the persona better.
• Demographic info: Other demographic information, such as age, gender, income, education, location, and other background information can help you create a more authentic character.
• Image: A vivid image, sketch, or cartoon can help your team picture your users clearly, and help you establish a consistent understanding of the target users. Use a photo that reminds you of this user.
• Goals: You should also make it clear what your audience wants to get or to do with your product. List all possible goals and motivations that you’ve found from your user research.

Here’s some examples:

2. Define

This step involves identifying problems that your users may have disclosed in the interview.

In reviewing your interview notes, you should be able to identify the needs that people are trying to meet with their activities.

• One way to do that is to underline verbs in the interview notes. These represent activities.
• A second way is to extract detailed stories that describe a software feature i.e. something that a user was trying to accomplush.

A user story is an informal, general explanation of a software feature written from the perspective of the end user. Its purpose is to articulate how a software feature will provide value to the customer.

User stories are a few sentences in simple language that outline the desired outcome. They don’t go into detail. Requirements are added later, once agreed upon by the team.

We will often pair user stories with personas to talk about “hypotheticals” when discussing potential solutions. e.g., “Do you think Alice would find this screening quiz adequate?”

• Stories, also called “user stories,” are short requirements or requests written from the perspective of an end user.
• Epics are large bodies of work that can be broken down into a number of smaller tasks (called stories).
• Initiatives are collections of epics that drive toward a common goal.

atlassian.com

Action: Define a problem statement

You should be able to produce a detailed problem description, which matches your user with a particular problem that they need addressed. This should follow directly from your interviews, and is the logical output from the define stage.

This represents the problem that you are going to address. Include as much detail as you can.

Phase II: Explore

3. Ideate

This is all about brainstorming solutions! Bring your team together and brainstorm all of the crazy ideas you have that could contribute towards a solution.

No idea is too far-feteched! Your (initial) goal is quantity over quality, and to uncover some unique and interesting ideas. Share ideas with one another, mixing and remixing, building on others’ ideas.

You will want to document these ideas. One option is to create an affinity diagram to collect everything.

The output from this step should be a set of features that represent your “best solution at this point”.

Action: Affinity diagrams

One popular method to collect ideas when brainstorming is to build a large affinity diagram.

1. Write down each idea that the group comes up with on a sticky note.
2. Put them on a whiteboard or large wall.
3. Organize them into clusters of related ideas. This can help you create hierarchies, or figure out dependencies e.g., “this feature would have to be implemented first before all of these others”, or “these two features work really well together!”.

This doesn’t need to be a physical board. There are many online software platforms that let you create collaborative diagrams together. e.g., invision Freehand.

4. Prototype

Next, you will build a prototype of your solution. A prototype is essentially a mockup of your solution, that is built to demonstrate functionality and elicit feedback from your users.

The goal of this phase is to understand what components of your ideas work, and which do not! In an ideal world, everything would work perfectly, but you probably will need to iterate a few times.

Note that a prototype is NOT an “early version” of release software, but an early demonstration of functionality. Compared to the full implementation, a prototype should require significantly less time to build. This makes it much easier to discard or modify as you work through design iterations.

Suggestions for building prototypes:

• Create static mockups of all of your screens. Make them low-fidelity to start, but include sufficient information to identify their purpose.
• Define the relationship between screens, so that it’s clear to a user how navigation through screens will work.
• Include enough visual clues to suggest how functionality is activated. The user should be able to tell how to perform major tasks.

Action: Prototyping in Figma

Figma is a collaborative design tool that allows designers to build a range of interactive prototypes. We’ll use it to build UI prototypes, and then we can show these to potential users and stakeholders (including our team members) for feedback before we implement features.

A design in Figma is meant to encompass all interactions for a product. It consists of:

• a frame, representing a single UI window (e.g. Android Pixel C screen, or 14“ MacBook Pro Desktop).
• UI elements, represented by customizable shapes.

Figma presents itself like a vector drawing application, letting you “build” a user interface much like you would in any drawing application. However, unlike drawing applications, it also supports adding interaction e.g. clicking buttons.

You can build your UI screen by

1. Adding a frame,
2. Adding primitive shapes (rectangles, circles) to represent UI elements. Use the property sheet on the right-hand side to align, change properties, and so on.

This is slow; you can also import and use Component Libraries that the Figma community has made available. In a browser, navigate to the Figma Community page. From this page, you can search and import libraries directly into your Figma account.

A library is just a set of predefined components. Using a library instead of designing your own has some advantages:

• You can save a lot of time!
• The components in a library will have a similar look-and-feel.
• You can use libraries to mimic the appearance of standard platforms e.g., iOS.

For example, here’s a sheet from the Mobile Wireframe library, which includes iOS, Android components.

You can create a new diagram, and copy-paste these components into your screens. Minimally, your initial prototypes should include labelled wireframe diagrams for every screen (displaying the structure of your application and each of the screens).

Phase III: Materialize

5. Test

This step involves taking your prototype back to your users, and allowing them to walk through it. Your goal is to collect their feedback! You want to address questions like:

• Does this solve your problem?
• Is it easy to understand? To use?
• Does (this functionality) make sense to you?
• What would you change?

Action: Collect feedback

Write down feedback and take it back to your team. Iterate!

6. Implement

At some point, you will have enough feedback to be able to proceed with the actual implementation. You should still demo occasionally to users, but you have enough information to proceed.

We’ll cover implementation in the development section.

Last Word

https://imgs.xkcd.com/comics/new_products.png

Software design

How should you structure software? What do we mean by “good code”?

Design principles

Before proceeding further, we should discuss how to structure an application properly. This requires a broader discussion of software architecture, and software qualities.

What is architecture?

Architecture is the holistic undertanding of how your software is structured, and the effect that structure has on it’s characteristics and qualities. Architecture suggests that structuring software should be a deliberate action.

Why is architecture important?

Decisions like “how to divide a system into components” have a huge impact on the characteristics of the software that you produce.

• Some architectural decisions are necessary for your software to work properly.
• The structure of your software will determine how well it runs, how quickly it performs essential operations, how well it handles errors.
• Well-designed software can be a joy to work with; poorly-designed software is frustrating to work with, difficult to evolve and change.

I like this quote by Bob Martin:

In Martin’s view, software should be enduring. Software that you produce should be reliable, and continue to function for a long period of time. You should expect to make adjustments over time as defects are found and fixed, and new features are introduced, but these changes should be relatively easy to make.

Building software to this level of quality does not happen accidentally, but requires deliberate action on our part.

https://www.atr.org/40-years-of-failure-irs-unable-to-fix-computer-system/

Software qualities

Let’s think about the qualities that we want in the software that we build. What would we expect from any piece of software that we produce?

Usability

A system must be usable for its intended purpose, and meet its design objectives. This includes both functional and non-functional requirements.

• Functional: features that are required to address the problem that the software is intended to solve; desirable features for our users.
• Non-Functional: qualities or characteristics of software that emerge from its structure. e.g., performance, reliability and other quality metrics.

Usability requires us to carefully ensure that we are meeting all reasonable requirements up-front, typically by collaborating with our users to define problems and solutions. See Process Models.

Extensibility

You should expect to adapt and modify your software over a lifetime of use:

• You will find errors in your code (aka bugs) that will need to be addressed.
• You might find that requirements were incomplete, so existing features may need to be modified.
• You might uncover new features that need to be implemented.
• Your operating environment might change (e.g. OS update) which necessitates updates.

We need to design systems such that you can respond and adapt your software to these changes, effectively, and with the least amount of work.

Your design goal is not to deliver software once, it’s to design in a way that supports delivering frequently and reliably over the life of your solution.

Scalability

Extensibility refers to handling new or changing requirements. Scalability refers to the ability to handle increased data, number of users, or features. e.g., an online system might initially need to handle hundreds of concurrent users, but could be expected to scale to tends of thousands over time. Scalability is challenging because you don’t want to incur the deployment costs up-front, but instead you want to design in a way that lets you expand your system over time. This can include replacing modules with more capable ones at a later time, e.g., swapping out a low-performance database for something more performant.

Robustness

The system should be able to handle error conditions gracefully, without compromising data. This includes user input errors, errors processing data from external sources, and recovering from error conditions, e.g., power outages.

Reusability

Software is expensive and time-consuming to produce, so anything that reduces the required cost or time is welcome. Reusability or code reuse is often positioned as the easiest way to accomplish this. Reusing code also reduces defects, since you’re presumably reusing code that is tested and known-good.

SOLID principles

SOLID was introduced by Robert (“Uncle Bob”) Martin around 2002. Some ideas were inspired by other practitioners, but he was the first to codify them.

The SOLID Principles tell us how to arrange our functions and data structures into classes, and how those classes should be interconnected. The goal of the principles is the creation of mid-level software structures that:

• Tolerate change (extensibility),
• Are easy to understand (readability), and
• Are the basis of components that can be used in many software systems (reusability).

The SOLID principles are as follows. The diagrams and examples are from Ugonna Thelma’s Medium post “The S.O.L.I.D. Principles in Pictures.”

1. Single Responsibility

The Single Responsibility Principle (SRP) claims to result in code that is easier to maintain. The principle states that we want classes to do a single thing. This is meant to ensure that are classes are focused, but also to reduce pressure to expand or change that class. In other words:

• A class has responsibility over a single functionality.
• There is only one single reason for a class to change.

• There should only be one “driver” of change for a module.

2. Open-Closed Principle

This principle champions subclassing as the primary form of code reuse.

• A particular module (or class) should be reusable without needing to change its implementation.
• Often used to justify class hierarchies (i.e. derive to specialize).

This principle is also warning against changing classes in a hierarchy, since any behaviour changes will also be inherited by that class’s children! In other words, if you need different behaviour, create a new subclass and leave the existing classes alone.

The Design Pattern principle of Composition over inheritance runs counter to this, suggesting instead that code reuse through composition is often more suitable.

3. Liskov-Substitution Principle

It should be possible to substitute a derived class for a base class, since the derived class should still be capable of all the base class functionality. In other words, a child should always be able to substitute for its parent.

In the example below, if Sam can make coffee but Eden cannot, then you’ve modelled the wrong relationship between them.

4. Interface Substitution

It should be possible to change classes independently of the classes on which they depend.

Also described as “program to an interface, not an implementation.” This means focusing your design on what the code is doing, not how it does it. Never make assumptions about what the underlying code is doing—if you code to the interface, it allows flexibility, and the ability to substitute other valid implementations that meet the functional needs.

5. Dependency Inversion Principle

The most flexible systems are those in which source code dependencies refer to abstractions (interfaces) rather than concretions (implementations). This reduces the dependency between these two classes.

• High-level modules should not import from low-level modules. Both should depend on abstractions (e.g., interfaces).
• Abstractions should not depend on details. Details (concrete implementations) should depend on abstractions.

This is valuable because it reduces coupling between two classes, and allows for more effective reuse.

In the example below, the PizzaCutterBot should be able to cut with any tool (class) provided to it that meets the requirements of able-to-cut-pizza. This might include a pizza cutter, knife or any other sharp implement. PizzaCutterBot should be designed to work with the able-to-cut interface, not a specific tool.

Generalized principles

Can we generalize from these? What if we’re not building a pure OO system; are they still applicable?

Enforce separation of concerns

“A module should be responsible to one, and only one, user or stakeholder.“ - Single Responsibility Principle

It follows from the SOLID principles that our software should be written as a set of components, where each one has specific responsibilities. By component, we can mean a single class, or a set of classes that work closely together, or even a function that delivers functionality.

Modularity refers to the logical grouping of source code into these areas of responsibility. Modularity can be implemented through namespaces (C++), packages (Java or Kotlin). When discussing modularity, we often use two related and probably familiar concepts: cohesion, and coupling.

Cohesion is a measure of how related the parts of a module are to one another. A good indication that the classes or other components belong in the same module is that there are few, if any, calls to source code outside the module (and removing anything from the module would necessitate calls to it outside the module).

Coupling refers to the calls that are made between components; they are said to be tightly coupled based on their dependency on one another for their functionality. Loosely coupled means that there is little coupling, or it could be avoided in practice; tight coupling suggests that the components probably belong in the same module, or perhaps even as part of a larger component.

When designing modules, we want high cohesion (where components in a module belong together) and low coupling between modules (meaning fewer dependencies). This increases the flexibility of our software, and makes it easier to achieve desirable characteristics, e.g. scalability.

Interface vs. implementation

“Program to an interface, not an implementation. Depend on abstractions, not on concretions.” - Interface Substitution

When classes rely on one another, you want to minimize the dependency—we say that you want loose coupling between the classes. This allows for maximum flexibility. To accomplish this, you extract an abstract interface, and use that interface to describe the desired behaviour between the classes.

For example, in the diagram below, our cat on the left can eat sausage, but only sausage. The cat on the right can eat anything that provides nutrition, including sausage. The introduction of the food interface complicates the model, but provides much more flexibility to our classes.

Avoid unnecessary inheritance

Multiple inheritence is terrrible. Who thought of this? - Prof. Avery

Inheritance is a useful tool for reusing code. In principle, it sounds great: derive from a base class, and you get all of its behaviour for free! Unfortunately, it’s rarely that simple. There are sometimes negative side effects of inheritance.

1. A subclass cannot reduce the interface of the base class. You have to implement all abstract methods, even if you don’t need them.
2. When overriding methods, you need to make sure that your new behaviour is compatible with the old behaviour. In other words, the derived class needs to act like the base class.
3. Inheritance breaks encapsulation because the details of the parent class are potentially exposed to the derived class.
4. Subclasses are tightly coupled to superclasses. A change in the superclass can break subclasses.
5. Reusing code through inheritance can lead to parallel inheritance hierarchies, and an explosion of classes. See below for an example.

A useful alternative to inheritance is composition.

Where inheritance represents an is-a relationship (a car is a vehicle), composition represents a has-a relationship (a car has an engine).

Imagine a catalog application for cars and trucks:

Here’s the same application using composition. We can create unique classes representing each vehicle which just implement the appropriate interfaces, without the intervening abstract classes.

Avoid side effects

At a high-level, functional programming emphasizes a number of principles:

• Functions as first-class citizens
• Pure functions, with no side effects
• Favouring immutable data

Although Kotlin is not a “pure functional language” (however, you care to define that), we can certainly adopt some of these principles.

The easiest, and perhaps the most important to program stability is the idea of avoiding side effects in your code. What is a side effect? It is an unintended change to your program’s state that is not expected when you call a function. In other words, it’s the characteristic of a function that does more than it should.

An obvious example of this is the use of global variables, when are then mutated in a function. As your program grows, you may find more and more functions that update these variables, to the point where it is often challenging to determine what caused them to change! Side effects make your code brittle, more difficult to debug and much more difficult to scale.

The opposite are functions that do just what they are designed to do. A function should:

• Use parameters as its only inputs (i.e. don’t look to any external state when determining a function’s behaviour)
• Mutate that data and return consistent results based solely on the input.

We’ll discuss functional approaches in Kotlin a little later in these notes.

Find the “right” abstractions

Part of the value of an object-oriented approach is that you can model “real-world objects”. We often decompose our context and problem space to identify entities that we wish to model in our software, and build classes reflecting those objects. Then, we can add logical behaviours to our objects to reflect how these objects would react in the real-world.

It’s important to keep in mind a few things when taking this approach:

1. Not all software objects need to have corresponding real-world objects (e.g., a software doesn’t need to operate under the same rules as a real factory).
2. Conversely, you don’t need to convert every real-world object into a software entity. There is nothing wrong with your models differing from the real-world.

What’s important when modeling behaviours in software is that you get the correct behaviours being captured and reflected in your system. (More on this soon).

Last Word

https://xkcd.com/974

Architectural styles

Let’s think about the features that every software application needs to support. Are there specific structural requirements that applications need to support?

Requirements

User interaction

Applications are interactive. Fundamentally, they revolve around an interaction cycle between the user and the system:

1. The system launches and waits for user input.
2. The user provides some input e.g., presses keys on a keyboard, moves a mouse, swipes on a screen.
3. The system processes that input.
4. The results e.g., output are displayed for the user.

The cycle looks something like this:

This loop is effectively the same across all interactive applications, including desktop, mobile and even console applications.

• Users perform actions with the user interface e.g., enter a command, tap/swipe on a phone, point/click with a mouse.
• The application responds to those user actions, by performing some computation, or fetching data or whatever else needs to be done.
• The results need to be sent back to the user interface to be displayed as output. This can be graphical, text, audio or any other suitable response that communications the result to the user.

We could model this simple interaction cycle as two components: one representing the user’s input-output and the second as the system that is interacting with them. Communication between them is done using events: messages between these componets to represent either the user’s input, or the system’s output to display. This works fine for simple systems.

Multiple sources

However, systems are rarely this simple. We often have multiple sources of data, and multiple sources of input that we need to manage.

In addition to the user interaction cycle (above), we can also have other events:

• The operating system may send events to your application that are not triggered by a user action. e.g., a timer ticking to indicate that time has elapsed, or a notification being sent to your application from some other service.
• Your application might request data (from a database of web service) which arrives after some delay. e.g., scrolling through a list of images that are on a remote site, while the list continues to populate in the background.
• Interruptions to your applications workflow based on some other high priority event. e.g., receiving a phone call while watching a video, and the phone call “forces itself” to the foreground.

A simple interaction cycle doesn’t handle these scenarios very well. We need to consider a more robust model that can address data coming in from other sources.

Architectural styles

How can you structure code to support these types of requirements?

In the same way that software developers use design patterns to describe useful and recurring software patterns, we use architectural styles to describe standard ways of structuring your code. There are a large number of architectural styles. In this course, we will focus on the subset of patterns that are most commonly used in building applications¹.

Model-View-Controller (MVC)

Model-View-Controller (MVC) was created by Trygve Reenskaug for Smalltalk-79 in the late 1970s as a method of structuring interactive applications. It suggests that an application should consist of the following components:

• Model: the information or program state that you are working with,
• View: the visual representation of the model, and
• Controller: which lays out and coordinates multiple views on-screen, and handles routing user-input.

In a “standard MVC” implementation, input is accepted and interpreted by the Controller class, and routed to the Model, where it changes the program state (in some meaningful way). These changes are published to the View through a notification system so that the changes can be reflected to the user.

MVC is probably the most commonly used (and most heavily reworked) architectural pattern, having been used in web development since the 1990s. Many application frameworks e.g., Java Swing have adapted it as their underlying UI model.

Here’s a simple MVC implementation, using the Observer pattern for Model/View notifications. The model implements the Publisher interface, and each View is a Subscriber. There can be more than one View (just as there can be more than one UI screen) and each one can receive a notification when the state changes. The View ultimately determines how to output the results.

classDiagram
    View "1" --> "1" Controller
    Controller "1" --> "1" Model

    Subscriber "1" <|.. "1" View
    Publisher <|.. Model
    Subscriber "*" <-- "*" Publisher

    class View {
        data: Data
        +display(data)
    }

    class Controller {
        view: View
        model: Model
        +invoke(event)
    }

    class Model {
        data: Data
        +sort(data)
    }

    class Subscriber {
        <<Interface>>
        +update()
    }

    class Publisher {
        <<Interface>>
        List~Subscriber~ subscribers
        -notify()
    }

However, there are a few challenges when implementing this version of MVC.

• Graphical user interfaces bundle the input and output together into graphical “widgets” on-screen (which we will explore further in the user interfaces section). This makes input and output behaviours difficult to separate, so in-practice, the controller class is rarely implemented.
• Modern applications tend to have multiple screens (either multiple windows open, or multiple screens in -memory that the user switches between). This model does not handle screen coordination terribly well.
• A single monolithic model should usually be split into multiple models, to reflect specialized data needs of each screen.

There have been a number of variant versions of MVC (MVP, MVVM, MVI). Let’s discuss a very popular specialization: MVVM.

Model-View-ViewModel (MVVM)

Model-View-ViewModel was invented by Ken Cooper and Ted Peters in 2005. Based on Martin Fowler’s Presentation Model, it was intended to simplify event-driven programming and user interfaces in C#/.NET.

MVVM suggests two major changes from MVC:

• MVVM removes the Controller class, and
• MVVM adds a data container class named the ViewModel, that sits between the View and Model.

This reduces our application to the following components:

• Model: As with MVC, the Model is the primary Domain object, holding the application state.
• View: The structure, layout and presentation of what is on-screen. With modern toolkits, the View handles both input and output i.e. the complete user experience.
• ViewModel: A component that stores data that is relevant to the View to which it is associated. This may be a simple subset of Model data, but is more often a reinterpretation of that data in a way that makes sense to the View e.g., dollar amounts in USD in the Model may be reflected in a local currently in the ViewModel.

One interesting trend that works in favor of MVVM is the idea of reactive programming, where changes in one component are automatically published to other interested components. MVVM is often implemented in a way where we can use a binding mechanism to map variables in the ViewModel directly to widgets in the View, so that updating one directly updates the other.

What is the benefit of a ViewModel?

• We will often want to pull “raw” data from the Model and modify it before displaying it in a View e.g., currency that is stored in USD but should be displayed using a local currency.
• We sometimes want to make local changes to data, but not push them automatically to the Model e.g., undo-redo where you don’t persist the changes until the user clicks a Save button.

MVVM recommends that you have one ViewModel for each View, and that ViewModel manages all data for that View. It looks something like this:

classDiagram
    ISubscriber "1" <|.. "1" ViewModel
    IPublisher <|.. Model
    ISubscriber "*" <.. "*" IPublisher

    View "1" --> "1" ViewModel
    View "1" <-- "1" ViewModel
    ViewModel "*" <.. "*" Model

    class View {
        -Model model
        -ViewModel viewModel
    }

    class ISubscriber {
        <<Interface>>
        +update()
    }

    class ViewModel {
        -View view
        -Model model
        +update()
    }

    class IPublisher {
        <<Interface>>
        -List~Subscriber~ subscribers
        +notify()
    }

    class Model {
        +var data
        +subscribe(ISubscriber)
        +unsubscribe(ISubscriber)
    }

MVVM is common in modern languages and toolkits, but it’s just one variant of MVC. There are many other variants (e.g. Model-View-ViewController) which deviate from “standard” MVC, usually in an attempt to solve a particular problem in a more elegant fashion. They all build on the same observer foundation.

Layered architecture

The second main pattern that we will see if the layered architecture. A layered architecture divides your application into slices, each with its own area of responsibilty.

Like MVC and MVVM, it’s primary concern is separation of concerns between the User-interface and the Model layers.

There are many variations to a layered architecture, with different names for each layer. The critical part of this pattern isn’t the number of layers, or even the names of the layers; it’s the way that the layers communicate, how control and data is passed between layers.

We’ll use the following four layers:

• User Interface: This is the same layer we introduced in the MVVM section above. (In fact, we will implement it using View and ViewModel classes, since that was such a useful idea).
• Domain: Classes that model your particular problem/user stories. These are often your data classes or representational classes that are specific to your application. e.g., if you are building a Recipe tracking application, your Domain layer would include classes like Recipe, RecipeList, RecipeFolder and so on. Domain-specific logic typically goes here.
• Model: This is the same layer that we introduced in MVVM, that stores the primary application state. It’s the “source of truth” for your application.
• Service: This is an API or interface to a system that will retrieve data for our application. This could be called a Database layer or Repository layer in other layered architectures. I prefer the term Service layer because in modern applications, we’re as likely to be pulling in data from a remote API as we are from a remote database. Fundamentally, there are data providers for the Model.

The characteristics of this approach:

• Each layer contains one or more classes with similar functionality. Classes can freely communicate within their layer.
• Dependencies between layers are top-down. i.e., the UI has a reference to the Domain layer, and can pass requests down to it; the Domain layer in turn can only communicate down to the Model layer. Layers call “down” to perform some service.
• As a rule, requests (messages) flow down, and notifications (data) flow back up.

For example, if the user interacts with the User Interface layer to direct it to perform some action, a message is generated that represents that action. This message flows down through each successive layer, triggering behaviour in each layers. In response, data flows back up. e.g., imagine clicking on a button to request details on a customer record; a message would be sent to the Domain layer, which would work a query directing the Model to get data, and return it to the User Interface layer.

Let’s discuss the layers and how they relate to one another.

Layers

User Interface (UI) layer

The User Interface, or Presentation layer, is the part of your application that the user interacts with (i.e. the “person” side of the interaction diagram). This is responsible for handling user input, and expressing output. This layer can be quite complex since it handles IO for all devices (keyboards, mice, touchpads) as well as the UI that the user interacts with (console, graphical).

Graphical interfaces typically consist of multiple screens, each with their own interaction support and visual representation. The layer itself will commonly include a large number of specialized classes (which we will discuss further below).

Domain layer

The Domain layer describes the application logic for our program. e.g., rules for managing customer data, or bank transactions, or how to combine ingredients in a recipe, or whatever else is needed. It serves as an intermediate layer between the raw data (e.g., records from the Model) and how that data is presented (e.g., screens in the UI layer).

Model layer

The actual application data being stored in memory, is often pulled from different sources e.g., bank transactions fetched from a database, user profile information stored in a preferences file. The model is meant to be the primary representation of the data in our application (the “single source of truth” in our application). The model is ultimately responsible for consolidation and presenting data to the rest of the application.

Service layer

This is a representation of the storage mechanism that actually retrieves or persists data. This can take many forms, from files on a file-system, to records in a relational database, or a bytestream from a DVD or some form of storage. The model interacts directly with service layer to fetch and manage this data and present it to other layers.

Data is passed back up the layers through dependency inversion: the use of interfaces keeps layers isolated.

Dependencies (down)

What do we mean by dependencies being directed down?

Imagine that we have simple classes for a CustomerView (e.g., a customer record screen in the user interface layer) and a CustomerModel (e.g., a class in the model layer that stores customer information). Dependencies being directed down means that the View classes can reference the Model classes directly, but not the other way around.

In the example below, we show part of an application (simplified, using the layers as class names for illustration). The UI layer can use (depend) on the Domain layer and any classes it contains. This is a one-way dependency; the Domain classes are not allowed to have any reference back to the UI layer.

classDiagram
    UI "*" --> "*" Domain
    Domain "*" --> "1" Model
    Model "1" --> "*" Service

    class UI {
        data: Data
        +display(data)
    }

    class Domain {
        data: Data
        +sort(data)
    }

    class Model {
        data: Data
        +add(data)
        +del(data)
    }

    class Service {
        data: Data
        +load() data
        +save(data)
    }

Notifications (up)

What do we mean by data being directed up?

The Model needs to have some way of notifying the View that the data has changed, without a direct reference to that view (“loose coupling”, remember?).

The Model doesn’t have a directly reference to the View, so what do we do? We use a loose coupling mechanism to send messages to the View. This is effectively the Observer design pattern, where the View (ISubscriber interface) is registers itself with the Model (IPublisher). The Model publishes changes to all registered Views when they occur.

This mechanism works for any data change in the model e.g., a system event causes the data to change, or data changes in the database, or the user changes data in one window causing a second window to update).

Here’s a simple example with notifications using the Publisher/Subscriber interfaces.

classDiagram

    Subscriber "1" <|.. "1" UI
    Publisher <|.. Model
    Subscriber "*" <-- "*" Publisher

    class UI {
        data: Data
        +display(data)
    }

    class Subscriber {
        <<Interface>>
        +update()
    }

    class Publisher {
        <<Interface>>
        List~Subscriber~ subscribers
        -notify()
    }

    class Model {
        data: Data
        +add(data)
        +del(data)
        +subscribe(Subscriber)
        +unsubscribe(Subscriber)
    }

Abstraction

Finally, one important piece of this approach is the use of Interfaces to promote loose coupling. By describing component relationships in terms of behaviours we have the flexibility to swap in new implementations at any time (abstractions not concretions).

This is especially important for

• Subscribers: we want any form of user interface to be able to receive notifications, not just GUI screens. e.g., we might send output to a voice dictation system, or a printer. I once worked on a project where the build output would trigger a lava lamp to light up (green for a passed build, red for a failed one!).
• Publishers: we want the flexibility of multiple models. We may not do this in production, but we can certainly do it when testing.
• Services: finally, we want to be able to request data and save data to a variety of services without knowing the implementation details. e.g., your model shouldn’t know the details of how to save to a SQL database, it should rely on an abstraction that exposes save behaviour. This let us swap databases, or even just save data to a file for testing instead of our remote DB.

Implementation

Full diagram

Here’s the full diagram, with all of the required classes for a layered architecture (including View/ViewModels and Services).

classDiagram

    UI "1" --> "1" ViewModel
    ViewModel "1" --> "*" Domain
    Domain "*" --> "1" Model
    Model "1" --> "*" Servicer

    Subscriber "1" <|.. "1" ViewModel
    Publisher <|.. Model
    Subscriber "*" <-- "*" Publisher
    Servicer <|.. Service

    class UI {
        data: Data
        +display(data)
    }

    class ViewModel {
        model: Model
        +update()
    }

    class Domain {
        data: Data
        +sort(data)
    }

    class Subscriber {
        <<Interface>>
        +update()
    }

    class Publisher {
        <<Interface>>
        List~Subscriber~ subscribers
        -notify()
    }

    class Servicer {
        <<Interface>>
        data: Data
        +load()
        +save()
    }

    class Model {
        data: Data
        +add(data)
        +del(data)
        +subscribe(Subscriber)
        +unsubscribe(Subscriber)
    }

    class Service {
        data: Data
        +load()
        +save()
    }

Sample code

Let’s imagine a simple application with a single View, ViewModel and Model. Here’s what the code would look like.

In our Main class, we instantiate classes and use dependency injection to connect our class instances. These should mirror the relationships on our diagram above.

// main class
class Main {
    val model = Model()
    val viewModel = ViewModel(model)
    val view = View(viewModel)
    model.add(viewModel)
}

The Subscriber interface is an abstraction for any class that wants to be notified of model changes. Any class can act like a view as long as it supports the appropriate method to allow notifications from the model. In this case, our ViewModel is the Subscriber. It will be subscribed for Model updates, and will take care of notifying its associated View as needed.

// UI classes
interface Subscriber {
  fun update()
}

class View(val viewModel: ViewModel) {
    // some user interface class that relies in the viewModel for its state
    // assume it can pull state from the viewModel and display it
}

class ViewModel(val model: Model) : Subscriber {
    override fun update() {
        // this method is called by the Publisher (aka Model) when data updates
        // fetch data from model when the model updates
    }
}

The Publisher (aka Model) maintains a list of all Subscribers (aka ViewModels), and notifies each one of them when its state changes. The Publisher has no control over how they react; each Subscriber must decide what to do with the update basd on whether the data is relevant to them. i.e., they might ignore the notification if it’s data they don’t use, or they might choose to fetch updated data from the Model.

In the code below, the Publisher is an abstract class (vs. an interface) so that we can add default implementation code for managing the Subscriber list.

abstract class Publisher {
    val views: List<Subscriber> = emptyList()

    fun addView(view: Subscriber) {
        views.add(view)
    }

    fun update() {
        for (view : views) {
            view.update()
        }
    }
}

class Model {
    fun fetchData() {
        // do something that causes data to change
        update() // notify subscribers
    }
}

Benefits

Layering our architecture really helps to address our earlier goals (reducing coupling, setting the right level of abstraction). Additionally, it provides these other specific benefits:

• Independent of frameworks. The architecture does not depend on a particular set of libraries for its functionality. This allows you to use such frameworks as tools, rather than forcing you to cram your system into their limited constraints.
• Testable. The business rules can be tested without the UI, database, web server, or any other external element.
• Independent of the UI. The UI can change easily, without changing the rest of the system. A web UI could be replaced with a console UI, for example, without changing the business rules.
• Independent of the database. You can swap out Oracle or SQL Server for Mongo, BigTable, CouchDB, or something else. Your business rules are not bound to the database.

Software modelling

Imagine that you’ve defined your requirements. You know who your users are, what they need (which may be different from what they want). You’ve identified a problem and the components of a solution. What comes next?

You need to start modelling your application, determining how to structure your application to meet these requirements.

There are three overarching principles that guide software modeling activities:

1. Model the essentials: Good models do not represent every minor feature for all scenarios. Modeling involves the most salient aspects needed to address the specific issues or questions. Trivial details are abstracted away to keep the model understandable and wieldy. This focuses on key characteristics for informed decision-making.

2. Provide perspective: Modeling constructs different views using defined rules to express each within the chosen modeling language and tools. This multifaceted, perspective-driven approach brings dimensionality. For example, models may provide structural, behavioral, temporal or organizational views. Organizing into separate views focuses on modeling for each view on specific concerns.

3. Enable effective communication: Diligent, uniform modeling facilitates straightforwardly conveying software information to stakeholders. Careful use of domain terminology and modeling language semantics enables articulation in a familiar vocabulary. Strict syntactic and semantic adherence also improves expression. Explicit notation contributes to communicative models, while modeling pragmatics helps share unambiguous meaning.

What does each model typically consist of?

A typical software model consists of an aggregation of multiple submodels. Each submodel provides a partial, simplified description and representation of some aspect or perspective of the software under consideration.

What is behavioral modeling?

Behavioral modeling is a model type that focuses on identifying and defining the dynamic behavioral aspects of software components. The goal is to represent how software functions, features, and system elements behave when in operation.

Behavioral models generally take one of three basic forms:

• State machine representations – These models depict software behavior in terms of defined states that the software or components can be in, along with events that can trigger transitions between those states. State machine modeling is valuable for elucidating complex workflows, protocols, lifecycles, and user interaction flows.
• Control flow models – These behavioral models illustrate software behavior through sequences of events, triggers, and messages that cause software processes to be dynamically activated or deactivated over time. Control flows are useful for expressing orchestration and coordination logic.
• Data flow models – This form represents software behavior in terms of how data moves through various processes, transformations, and mappings in route to its ultimate destination in data stores or data sinks. Data flow modeling helps clarify throughput and signal processing behavior.

Domain-Driven Design

Domain-Driven Design (DDD) is all about building a model that reflects the real-world domain (aka “problem area” in which your working). As a software developer, your goal is to build an understanding of the problem domain, and then building classes and functionality that reflect the real-world domain. You may even go so far as to define a Domain Specific Language to reflect the problem domain using domain-specific terminology.

For example, in a DDD model of a banking system:

• You might have classes like Customer, Account, Transaction.
• Your class methods would be actions in the business domain, like withdraw, deposit, authorize.

DDD excels in building up a system that needs to accurately reflect a real-world system or process, along with all of it’s constraints and rules. e.g., a medical imaging system that stores and manipulates medical images is very likely going to be build to accommodate an existing process within a hospital. Getting the domain correct is critical for this type of software!

Goals of DDD

Domain-driven design is predicated on the following goals:

• Placing the project’s primary focus on the core domain and domain logic layer;
• Basing complex designs on a model of the domain;
• Initiating a creative collaboration between technical and domain experts to iteratively refine a conceptual model that addresses particular domain problems.

When to use it?

When should you use this?

• Your problem domain is extremely complex and you need to ensure that you are “following the rules” of the domain.
• You need to engage domain experts, and require their input in your design to encode these rules. DDD provides you with a common set of terminology that you can use e.g., you can discuss the behaviour of your design in terms of the problem domain.
• Your business domain is growing and evolving. DDD is suitable since you can grow the system as your understanding of the domain grows.
• You need to split development across a large number of teams or developers. With DDD it may be easier to isolate parts of the system to be developed independently.

When should you not use this?

• It doesn’t suit more abstract problems.
• Your problem isn’t modelling a real-world system.

Last Word

https://xkcd.com/1739

Code structure

Where do I start?

Now you know what your goals should be, you’ve thought about some design principles to guide your implementation, and you know what the architecture should look like.

What’s next? How do you actually get started?

You know that:

• You need layers for your user-interface, domain and model.
• You have features that are orthogonal to these layers e.g., your recipe application will probably need to fetch data (model and persistance), modify and repackage it into a useful structure (domain) and display it (view).

Do you write all of your views first? Do you build models first? What to do?

Vertical slicing

In software development, vertical slicing is an approach where a feature or functionality is developed and delivered end-to-end in a single iteration. Unlike traditional methods that focus on building software layer by layer (horizontal slicing), vertical slicing carves out a fully functional piece across all layers or aspects of the application – right from the user interface down to the data access layer.

We recommend vertical slicing.

Why do this?

• It aligns your design and code with your user stories and features. You can build related features together, which will result in a more cohesive design.
• You are delivering working functionality each iteration. You can demonstrate (and get feedback) on a specific feature set.
• You have working code faster, which reduces risk in your project (compare this to horizontial scaling, where you might build an entire layer, but can’t actually demonstrate features until you integrate all of the changes later in the project).

Horizontal slicing

The counter to veritical slicing is horizontal slicing where you build entire layers at a time. This approach would suggest that you build the entire model + persistance layer, separate from the user-interface layer. You would leave integrating these layers (and adding user features) later in the project.

There is a time and place for horizontal slicing, typically when you’re building a library or framework to address specific technical needs. For example, maybe your data is so complex that you need a robust model in place to do anything. That might warrant building out most of the Model ASAP. However, even in this scenario, I would recommend dividing your labour: have some people continue building features (vertically) and they can provide API feedback while the Model is being built.

In most cases, vertical slicing will give you better results since it focuses on delivering testable features.

How do I structure my source code?

This is a vertical structure.

Each feature is a top-level package, and within that package, you have sub-packages for the different architectural layers.

e.g. a recipe application with both a card (detail) view and a list (summary) view of recipe data. The lowest level packages would contain the relevant classes for that package.

.
├── main
│   └── kotlin
│       ├── recipe-card
│       │   ├── domain
│       │   ├── model
│       │   └── userinterface
│       └── recipe-list
│           ├── domain
│           ├── model
│           └── userinterface

This has a major advantage in that each feature is isolated and testable within its own namespace. i.e., you can write unit tests that check the model, domain and UI classes for a particular feature and ensure that they work well together.

Best practices

Useful software engineering practices.

Version control

A Version Control System (VCS) is a software system designed to track changes to source code over time. This allows you to carefully track what changes have been made, merge changes from diferent people on a software team, and make sure that only desired changes make it to production.

Common VCS systems include Mercurial (hg), Subversion (SVN), and Perforce. We’ll be using Git, a very popular VCS, in this course. A VCS provides some significant benefits:

History A VCS provides a long-term history of every file. This includes tracking when files were added, or deleted, and every modification that has been made. Did you break something? You can always unwind back to the “last good” change that was saved, or ever compare your current code with the previously working version to identify an issue.

Versioning Software evolves over time, and we often have multiple relevant versions of code e.g., multiple releases that we have provided to customers. Versioning refers to our ability to track sets of changes over time, and assign them semantically meaningful labels or version numbers.

Collaboration A VCS provides the necessary capabilities for multiple people to work on the same code simultaneously. Changes need to be isolated while features are being developed, and then merged together carefully. We’ll discuss the use of branches, a Git feature, to support this.

Distributed Early VCS systems required near-constant coordination between your working environment and a remote server that hosted the code. Git is different; it was designed as a distributed VCS. This means that you can work on changes to source code without any need to access a central server. Decoupling local and remote changes was one of the major drivers to Git’s early success.

Installation

Git can be installed from the Git home page or through a package manager (e.g. Homebrew on Mac). Although there are graphical clients that you can install, Git is primarily a command-line tool.

Once installed, you will want to make sure that the git executable (git or git.exe) is in your path. You can check this by typing git --version on the command-line.

$ git --version
git version 2.39.3 (Apple Git-146)

Concepts

Version control is designed around the concept of a changeset: a grouping of files that together represent a change to the software that you’re tracking (e.g. a feature that you’ve implemented may impact multiple source files).

Git is designed around these core concepts:

Repository The location of your source code. This can be local (where you are only storing source code on your own machine) or a remote server (e.g., GitHub, where source code is stored in a central location). Git operates perfectly well in either of theses situations. A remote server adds some additional functionality: the ability to perform backups and other maintainance, and more importantly, it simplifies sharing your repository with other people.

Working Directory A copy of the repository, on your local system, where you will make your changes before saving them in the repository. Your working directory contains all of the source code; this is required so that you can build, run your code locally.

Staging Area A collection of changes that you wish to track in Git as a changeset. This should be a set of related changed, reflecting a single feature, bug fix or other logical grouping.

Git works by operating on a set of files (aka changeset): we git add files in the working directory to add them to the change set; we git commit to save the changeset to the local repository. We use git push and git pull to keep the local and remote repositories synchronized.

Git Reference Sheet

Git basics

Standard Git functionality is accessed through the command-line. You can use a graphical client, but you will need to understand the command-line concepts to work with Git effectively.

Git commands

Git commands are of the form git <command>. Use git help to get a full list of commands.

$ git --help
usage: git [-v | --version] [-h | --help] [-C <path>] [-c <name>=<value>]
           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]
           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
           [--super-prefix=<path>] [--config-env=<name>=<envvar>]
           <command> [<args>]

These are common Git commands used in various situations:

start a working area (see also: git help tutorial)
   clone     Clone a repository into a new directory
   init      Create an empty Git repository or reinitialize an existing one

work on the current change (see also: git help everyday)
   add       Add file contents to the index
   mv        Move or rename a file, a directory, or a symlink
   restore   Restore working tree files
   rm        Remove files from the working tree and from the index

examine the history and state (see also: git help revisions)
   bisect    Use binary search to find the commit that introduced a bug
   diff      Show changes between commits, commit and working tree, etc
   grep      Print lines matching a pattern
   log       Show commit logs
   show      Show various types of objects
   status    Show the working tree status

grow, mark and tweak your common history
   branch    List, create, or delete branches
   commit    Record changes to the repository
   merge     Join two or more development histories together
   rebase    Reapply commits on top of another base tip
   reset     Reset current HEAD to the specified state
   switch    Switch branches
   tag       Create, list, delete or verify a tag object signed with GPG

collaborate (see also: git help workflows)
   fetch     Download objects and refs from another repository
   pull      Fetch from and integrate with another repository or a local branch
   push      Update remote refs along with associated objects

You can also type git help on each sub-command to get detailed assistance.

$ git help commit
GIT-COMMIT(1)                   Git Manual                  GIT-COMMIT(1)

NAME
       git-commit - Record changes to the repository

SYNOPSIS
       git commit [-a | --interactive | --patch] [-s] [-v] [-u<mode>] [--amend]
                  [--dry-run] [(-c | -C | --squash) <commit> | --fixup [(amend|reword):]<commit>)]
                  [-F <file> | -m <msg>] [--reset-author] [--allow-empty]
                  [--allow-empty-message] [--no-verify] [-e] [--author=<author>]
                  [--date=<date>] [--cleanup=<mode>] [--[no-]status]
                  [-i | -o] [--pathspec-from-file=<file> [--pathspec-file-nul]]
                  [(--trailer <token>[(=|:)<value>])...] [-S[<keyid>]]
                  [--] [<pathspec>...]

Setup a local repository

To create a local repository that will not need to be shared:

1. Create a repository. Create a directory, and then use the git init command to initialize it. This will create a hidden .git directory (where Git stores information about the repository).

$ mkdir repo
$ cd repo
$ git init
Initialized empty Git repository in ./repo/.git/
$ ls -a
.    ..   .git

2. Make any changes that you want to your repository. You can add or remove files, or make change to existing files.

$ vim file1.txt
ls -a
.         ..        .git      file1.txt

3. Stage the changes that you want to keep. Use the git add command to indicate which files or changes you wish to keep. This adds them to the “staging area”. git status will show you what changes you have pending.

$ git add file1.txt
$ git status
On branch master

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
new file:   file1.txt

2. Commit your staging area. git commit assigns a version number to these changes, and stores them in your local repository as a single changeset. The -m argument lets you specify a commit message. If you don’t provide one here, your editor will open so that you can type in a commit message. Commit messages are mandatory, and should describe the purpose of this change.

$ git commit -m "Added a new file"

Setup a remote repository

A remote workflow is almost the same, except that you start by making a local copy of a repository from a remote system.

1. Clone a remote repository. This creates a new local repository which is a copy of a remote repository. It also establishes a link between them so that you can manually push new changes to the remote repo, or pull new changes that someone else has placed there.

# create a copy of the CS 346 public repository
$ git clone https://git.uwaterloo.ca/j2avery/cs346.git ./cs346

Making changes and saving/committing them is the same as the local workflow (above).

2. Push to a remote repository to save any local changes to the remote system.

$ git push

3. Pull from remote repository to get a copy of any changes that someone else may have saved remotely since you last checked.

$ git pull

4. Status will show you the status of your repository; log will show you a history of changes.

# status when local and remote repositories are in sync
$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

# condensed history of a sample repository
$ git log --oneline
b750c10 (HEAD -> master, origin/master, origin/HEAD) Update readme.md
fcc065c Deleted unused jar file
d12a838 Added readme
5106558 Added gitignore

Tags

A tag is a label that you can apply to a particular point in a repositories history. They are useful to track specific important points in time e.g., a software-release point.

Git supports lightweight and annotated tags:

• A lightweight tag is just a pointer to a specific commit.
• Annotated tags are stored as full objects. They’re checksummed; contain the tagger name, email, and date and have a tagging message.

Useful commands:

• git tag to display the tags that have been applied.
• git tag -a X to add annotated tag X to the current branch/head.
• git checkout X to checkout a branch based on a tag. This is useful with a versioned commit history!

Branching

The biggest challenge when working with multiple people on the same code is that you all may want to make changes to the code at the same time. Git is designed to simplify this process.

Git uses branches to isolate changes from one another. You think of your source code as a tree, with one main trunk. By default, everyone in git is working from the “trunk”, typically named master or main (you can see this when we used git status above).

Git Branches

A branch is a fork in the tree, where we “split off” work and diverge from one of the commits (typically from a point where everything is working as expected)! Once we have our feature implemented and tested, we can merge our changes back into the main branch.

Notice that there is nothing preventing multiple users from doing this. Because we only merge changes back into main, when they’re tested, the trunk should be relatively stable code.

We have a lot of branching commands:

$ git status	// see the current branch
On branch master

$ git branch test // create a branch named test
Created branch test

$ git checkout test  // switch to it
Switched to a new branch 'test'

$ git checkout master //switch back to master
Switched to branch 'master'

$ git branch -d test // delete branch
Deleted branch test (was 09e1947).

When you branch, you inherit changes from your starting branch. Any change that you make on that branch are isolated until you choose to merge them.

Advanced Git

Commit guidelines

1. Include just what you need in a commit.

   Make sure that you are only including related changes, for a single issue. You should aim to have a single commit for every significant feature or bug fix.

   If you have multiple, unrelated changes to a file, you can choose which changes to include:
   git add -p <file> will prompt you (y,n) for each change to include.

2. Write a detailed commit message. You should include:

• Title: The first line of the commit. This should be a concise summary.
• Body: Leave a blank line after the title, and then a block of test. You should include more details, including reasons for the change or other information that will be useful. You should also include an issue number if the commit is related to an issue in GitLab (which it should be).

Branching strategies

Option 1: Mainline development

This is a strategy where the most up-to-date changes are kept on main.

• Relatively few branches from main; only created as needed.
• Merge and integrate features to main as they are completed.
• Requires diligent testing and care!
• Releases done from main branch.

Option 2: Release and feature branches

More complex strategies will use separate branches for individual work. Merges are done back to an intermediate release branch, where releases are staged and performed. Main is where all release branches are ultimately integrated.

• Multiple branches, at different levels.
• Merge back to release branches as needed.
• Provides the ability to have multiple, independent releases.
• More resilient to changes, at the cost of complexity.

This model also illustrates the idea of short-lived vs. long-lived branches. Feature branches are created as needed, but are relatively short-lived: once the feature is integrated/merged back into the long-lived release branch, the feature branch can be deleted.

Feature branches

We’ll standardize on a simpler model, closer to Option 1, where we create feature branches for individual work and integrate changes back to main.

Use this workflow for adding a feature:

1. Create a feature branch for that feature.
2. Make changes on that branch only. Test everything.
3. Code review it with the team.
4. Switch back to main and git merge from your feature branch to the master branch. If there are no conflicts with other change on the main branch, your changes will be automatically merged by git. If your changes conflict (e.g. multiple people changed the same file) then git may ask you to manually merge them.

$ git checkout -b test // create branch
Switched to a new branch 'test'

$ vim file1.md // make some changes
$ git add file1.md
$ git commit -m "Committing changed to file1.md"

$ git checkout main // switch to main
$ git merge test // merge changes from test
Updating 09e1947..ebb5838
Fast-forward
 file1.md                   | 136 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 118 insertions(+), 18 deletions(-)

$  git branch -d test // remove branch (optional)
Deleted branch test (was ebb5838).

When you merge, Git examines your copy of each file, and attempts to apply any other changes that may have been committed to main since you created the branch. In many cases, as long as there are no conflicts, Git will merge the changes together. However, if Git is unable to do so, then you will be prompted to manually merge the changes together.

Pull requests (PRs)

One way to avoid merge issues is to review changes before they are merged into main (this also lets you review the code, manually run tests etc). The standard mechanism for this is a Pull Request (PR). A PR is simply a request to another developer (possibly the person responsible for maintaining the main branch) to git pull your feature branch and review it before merging.

We will not force PRs in this course, but you might find them useful within your team.

GitLab also calls these Merge Requests.

Last Word

https://xkcd.com/1597

Build configuration

Build requirements

When developing applications, there is a large list of steps that need to be completed before we can release our software to users. We might need to:

• Make sure that we have the correct version of source code and libraries.
• Setup our build environment and compile everything.
• Setup our test environment, and run tests.
• Build installers to distribute to users.

Performing these steps manually is error-prone and time-consuming. Instead of doing this by-hand, we tend to rely on build systems: software that is used to build other software. There are a number of build systems on the market that attempt to simplify and propvide consistency in these tasks. These tools often language or toolchain dependent due to the specifics of each programming language. e.g. make or scons for C++; cargo for Rust, maven for Java.

Builds systems automate what would otherwise be a manual process. They provide consistency and help address issues like:

• How do I make sure that all of the steps are being handled properly?
• How do I ensure that everyone is building software the same way i.e. using the same compiler options, and libraries?
• How do I ensure that tests are being run before changes are committed?

Gradle setup

We’re going to use Gradle, a modern build system that provides more functionality than make and is more suitable for complex projects. Why Gradle and not some other build system like Maven?

• It’s popular in the Kotlin and Java ecosystems.
• It’s the official Google-endorsed build tool for Android projects.
• It’s cross-platform and programming language agnostic.
• It’s open source and has a large community of users.

Gradle has three main pillars of functionality that we will explore:

1. Managing build tasks: Built-in support for discrete tasks that you will need to perform. e.g., downloading libraries; compiling code; running unit tests and so on.
2. Build configuration: A way to define how these tasks are executed.
3. Dependency management: A way to manage external libraries and dependencies.

We’ll discuss these in more detail later in this section. First, let’s setup Gradle.

Project setup

Gradle is always setup in the context of a project: a specific directory structure, and a set of configuration files that define how your source code will be built. You create the Gradle project, and then your source code (and other assets) are added to that directory structure.

Project creation can be done in IntelliJ IDEA or Android Studio, or by using the gradle command-line tool.

Create a project

We’ll create a project using IntelliJ IDEA.

Use the File > New Project wizard in IntelliJ IDEA to create a new empty project. This will give you a top-level project with starting configuration files.

Projects can be created with a variety of configurations, but for this course, you should choose Kotlin as your programming language, Gradle for your build system, and Kotlin for your DSL language.

The result of this process is a new project with a directory structure and configuration files that are ready to use with Gradle. Note that the project will be specific to the type of project that you chose e.g., Android, Desktop, Web Service.

Using the gradle wrapper

At the top-level of your project’s directory structure will be a script named gradlew (or gradlew.bat for Windows users). This is the Gradle wrapper: a script that you can use to run Gradle tasks without having to install Gradle on your machine. You can run it and pass it the same command-line arguments that you would normally pass to Gradle.

This allows you to run Gradle tasks as-if you had Gradle installed locally (but without needing to install it).

For example, we can invoke the Gradle wrapper with the help task to get information about how Gradle is configured and how to use it. This is the same as running gradle help if you had Gradle installed on your machine.

$ ./gradlew help
Starting a Gradle Daemon (subsequent builds will be faster)

> Task :help

Welcome to Gradle 7.5.1.

To run a build, run gradlew <task> ...
To see a list of available tasks, run gradlew tasks
To see more detail about a task, run gradlew help --task <task>
To see a list of command-line options, run gradlew --help
For more detail on using Gradle, see https://docs.gradle.org/7.5.1/userguide/command_line_interface.html
For troubleshooting, visit https://help.gradle.org

Wrapper configuration

The Gradle project configuration (gradle/gradle-wrapper.properties) lists the version of Gradle to be used for your project.

To change the version of Gradle being used, update the gradle-wrapper.properties file, and change the distributionURL line to the correct version e.g., Gradle 8.0.2 below.

distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.0.2-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

The gradle-wrapper.properties file should be stored in Git, as part of your build configuration scripts.

Gradle tasks

Projects often have complex build requirements that include a series of steps that need to be performed. For example, you might need to:

1. Compile your source code,
2. Run tests to make sure it built and works properly,
3. Build a distributable package.

You might also have additional steps: generate documentation, run static analysis, or deploy to a server.

Any build system needs to support a wide range of steps like this, and it should allow you to define how these tasks will be performed. It should also run them in the correct order.

Gradle uses the term task to describe a set of related functions that can be applied to a particular type of project. For example, you might have a clean task to remove temporary build files, and a build task to build your Kotlin console project. Complex projects will add additional tasks that are required for that style of project e.g., build-android-application.

Often complex projects will require many different tasks to be executed in a particular order to achieve your goal. For instance, there are likely a dozen or so tasks required to be executed in order to build a working Android application. Gradle is able to manage these tasks and execute them in the correct order for you.

In this, we way that Gradle is declarative, in that you describe what you want to do, rather than how you want to do it. This is incredible useful, as it allows Gradle to manage the details of how to perform more complex tasks for you. We will discuss how to describe your project using build configuration files later in this section.

Running tasks

To run Gradle tasks from the command line, use the Gradle wrapper with the appropriate the task name. We can run ./gradlew tasks to see a list of tasks that are supported in your project.

$ ./gradlew tasks

> Task :tasks

------------------------------------------------------------
Tasks runnable from root project 'gradle'
------------------------------------------------------------

Application tasks
-----------------
run - Runs this project as a JVM application

Build tasks
-----------
assemble - Assembles the outputs of this project.
build - Assembles and tests this project.
buildDependents - Assembles and tests this project and all projects that depend on it.
buildNeeded - Assembles and tests this project and all projects it depends on.
classes - Assembles main classes.
clean - Deletes the build directory.
jar - Assembles a jar archive containing the classes of the 'main' feature.
testClasses - Assembles test classes.

# many more that are not shown

You can execute any of these tasks through the Gradle wrapper. e.g. ./gradlew build or ./gradlew clean.

Commonly used tasks include:

• ./gradlew help will provide online help.
• ./gradlew tasks will list all of the tasks that are available.
• ./gradlew clean will remove temporary build files.
• ./gradlew build will build your project.
• ./gradlew run will usually run your project (depending on the platform).

You can also run tasks from within IntelliJ. Both IntelliJ IDEA and Android Studio include a Gradle IDE plugin, which allows you to run Gradle tasks.

View > Tool Windows > Gradle will open the Gradle window. Tasks are grouped by category. You can run individual tasks by double-clicking on them.

Using plugins

Gradle comes with a small number of predefined tasks. You will usually need to add additional tasks that are specific to your type of project. We do this via plugins.

A plugin is a collection of tasks that have been bundled together to perform a specific function. For example, the java plugin adds tasks for compiling Java code, running tests, and building a distributable package. Plugins can also add additional configuration options, or set defaults for existing options.

There are different types of plugins.

1. Core plugins: These are the plugins that are included with Gradle by default. They provide basic functionality that is required by many projects. Core plugins include java (which adds support for compiling Java code), and application (which adds support for running code, and building a distributable package).

Core plugins are listed in the Gradle documentation, and you can use them in your project by adding a line to the plugins section of your build.gradle.kts file.

plugins {
    java
}

2. Community Plugins: These are plugins that are created by the community and are not included with Gradle by default. They provide additional functionality that is not available in the core plugins. An example would be the ‘io.github.gmazzo.codeowners.jvm’ plugin, which provides some very specific functionality.

Community plugins can be found in the Gradle Plugin Portal. This is a repository of plugins that have been created by the community. You can search for plugins by name, or by category.

Gradle plugin search (gradle.org)

Under the plugin page on the Gradle Plugin Portal, you will find instructions on how to add the plugin to your project. Typically, you will need to add a line to the plugins section of your build.gradle.kts file.

Build configuration

Once you have tasks defined, you need some way to configure and control how they are executed.

It’s certainly possible to write custom scripts e.g, bash shell scripts to execute these tasks, but they are challenging to maintain, and need to be written for each project.

Instead, Gradle provides a way to define tasks in build configuration files, and then run them with a single command. This makes it easy to build complex projects, and ensures that the build process is consistent across all of your projects.

Build configuration (gradle.org)

Unlike other configuration-based build systems, Gradle uses a Domain Specific Language (DSL) to define build scripts; you actually write your scripts in Groovy or Kotlin. This makes Gradle extremely configurable and extensible to meet complex project requirements.

Directory structure

When you create a new project with Gradle, it will create a directory structure for you.

This directory structure is opinionated: Gradle requires a specific directory structure to work correctly, so you should work within the structure that it gives you. Most of these files are configuration file, or scripts that help execute gradle. The actual source code will be placed in the app/src directory.

A standard Gradle project has the following directory structure:

.
├── build.gradle.kts
├── gradle
│   └── wrapper
├── gradle.properties
├── gradlew
├── gradlew.bat
├── settings.gradle.kts
└── src
    ├── main
    └── test

Configuration files

build.gradle.kts and settings.kts are the configuration files for your project, describing logical structure, dependencies, and so on.

!!! warning We strongly recommend that you use the New Project wizard to create your project. This will ensure that you have the correct directory structure and configuration files. !!!

settings.gradle.kts

This is the top-level configuration file. You likely don’t need to modify this for single-target projects. Later, when we discuss multi-project builds, we will modify this file to add additional modules.

plugins {
    id("org.gradle.toolchains.foojay-resolver-convention") version "0.5.0"
}

// top-level descriptive name
rootProject.name = "project-name"

build.gradle.kts

This is the detailed build configuration. You might need to modify this file to:

• Add a new dependency (i.e. library)
• Add a new plugin (i.e. set of custom tasks)
• Update the version number of a product release (version below).

Briefly, here’s what each section represents:

• plugins are used to add project-specfic tasks and functionality. In this case, the jvm plugin is added to support Kotlin desktop code.
• repositories are locations where libraries are stored and made available. mavenCentral() is the standard online repository for Java and Kotlin libraries.
• dependencies are external libraries that your project needs. In this case, the kotlin-test library is added.
• the kotlin section is used to specify properties for the corresponding plugins. In this case, “Java 17” is specified for your project config.

// includes jvm tasks
plugins {
    kotlin("jvm") version "1.9.21"
}

// product release info
group = "org.example"
version = "1.0-SNAPSHOT"

// location to find libraries
repositories {
    mavenCentral()
}

// add libraries here
dependencies {
    testImplementation(`org.jetbrains.kotlin:kotlin-test`)
}

tasks.test {
    useJUnitPlatform()
}

// java version
kotlin {
    jvmToolchain(17)
}

Multi-project builds

A simple build is suitable for most standalone, independent projects. Keep in mind that it is restricted to the specific project type that you chose with the New Project wizard e.g., Android, Desktop, Web Service.

What do you do if you want to build a more complex project e.g., a combination of desktop, Android, web service and so on? You could do this with multiple separate projects, but it’s often beneficial to have them in the same project.

In Gradle, this is known as a multi-project build, and it allows you to build multiple projects from a single top-level project. This is useful when you have multiple projects that depend on each other, or when you want to share code between projects.

There is a specific structure that you need to follow to create a multi-project build. In the example below, we have subprojects application, models and server which represent different projects.

.
├── application
│   ├── bin
│   ├── build.gradle.kts
│   └── src
├── gradle
│   └── wrapper
├── gradle.properties
├── gradlew
├── gradlew.bat
├── local.properties
├── models
│   ├── bin
│   ├── build.gradle.kts
│   └── src
├── server
│   ├── bin
│   ├── build.gradle.kts
│   └── src
└── settings.gradle.kts

To create a multi-project build, you need to create subprojects for each project that you want to manage. Each subproject should have its own build.gradle.kts file, and a src directory containing the source code for that project. The top-level settings.gradle.kts describes common dependencies and the overall project structure.

Let’s walk through an example of how to create a multi-project build.

Step 1: Move the application code into an application project:

To move the code:

1. Create a directory named application.
2. Move the build.gradle.kts and src folder from the root into the application directory.
3. Edit the top-level settings.gradle.kts and add the following line to the bottom of the file:

include("application")

Step 2: Add a server project

To add a server project:

• Create a type of project using the IntelliJ IDEA Project Wizard e.g., Ktor server.
• Place it in a subdirectory of your main project. Each subdirectory should just contain the generated src directory, and the build.gradle.kts file. You don’t want anything else in the project subdirectory.
• Finally, update the settings.gradle.kts to add each project subdirectory in the include statement.

include("application", "server")

Step 3: Add a models project

Repeat Step 2 with an empty project to hold shared models.

Dependency management

The final core function is managing libraries and dependencies.

When we write software, we often rely on external libraries to provide functionality that we don’t want to write ourselves. For example, we might use some library to handle networking, or to provide a user interface. These libraries are known as dependencies, and they are a critical part of modern software development.

A large challenge of any build system is managing these dependencies properly. For example, you need to make sure that you have the correct version of a library, and that any dependencies that it might need are also installed (called transitive dependencies). You also need to make sure that the library is compatible with the rest of your software, and that it doesn’t introduce any security vulnerabilities.

Gradle is designed so that you can specify your dependencies in your build scripts. Gradle will download them from an online repository as part of your build process, and manage them for you.

mavenCentral is the standard online repository for Java and Kotlin libraries.

Dependencies (gradle.org)

Setting up repositories

We’ll use a public online repository for this course. The most popular Java/Kotlin repository is mavenCentral, and we’ll use it with Gradle to import any external dependencies that we might require.

You can control the repository that Gradle uses by specifying its location in the build.gradle.kts file. By default, Maven Central should already be included.

repositories {
     mavenCentral()
}

Browsing repositories

You can browse repository websites to find libraries that you might want to use. Additionally, you can use an online package directory.

Package search (jetbrains.com)

Each package information page will include the details of how to import the package into your Gradle project. e.g.

Add dependencies

You add a specific module or dependency by adding it into the dependencies section of the build.gradle.kts file. Dependencies need to be specified using a “group name: module name: version number” (with a colon separating each one).

From the previous example, we can copy and paste the dependency line from the package information page directly into our build.gradle.kts

dependencies {
    implementation("io.coil-kt.coil3:coil-jvm:3.0.0-alpha06")
}

You generally want the most recent version that is available, but you can specify a specific version if you need to. Gradle will download the library and any dependencies that it needs, and make them available to your project.

Version catalogs

One challenge is keeping track of the versions of libraries that you are using. It’s important to keep your libraries up-to-date to ensure that you have the latest features and bug fixes. However, updating libraries can be a time-consuming process, as you need to check the library’s website for new versions, update your build file, and test your application to make sure that everything still works.

Gradle has a feature called version catalogs, which is a centralized file that contains a list of libraries and their versions. When you run a Gradle build, Gradle will check the version catalog to see if there are any updates available for the libraries that you are using. If there are updates available, Gradle will automatically update the libraries in your project to the latest versions.

In Gradle 7.6 or later, the version catalog is contained in a file named libs.versions.toml in your gradle project directory.

A version catalog file contains sections for versions, libraries (dependencies) and plugins.

[versions]
kotlin-version = "2.0.20-RC"
serialization-version = "1.9.0"
clikt-version = "4.4.0"
ktor-version = "2.3.4"
ktor-api-version = "2.2.4"
json-version = "1.7.1"
slf4j-version = "2.0.7"
versions-version = "0.51.0"

[libraries]
clikt = { module = "com.github.ajalt.clikt:clikt", version.ref = "clikt-version" }
ktor-core = { module = "io.ktor:ktor-client-core", version.ref = "ktor-version"}
ktor-cio = { module = "io.ktor:ktor-client-cio", version.ref = "ktor-version"}
ktor-server = { module = "io.ktor:ktor-server-default-headers", version.ref = "ktor-version"}
ktor-api = { module = "dev.forst:ktor-api-key", version.ref = "ktor-api-version" }
json = { module = "org.jetbrains.kotlinx:kotlinx-serialization-json", version.ref = "json-version" }
slf4j = { module = "org.slf4j:slf4j-simple", version.ref = "slf4j-version"}

[plugins]
ktor = { id = "io.ktor.plugin", version.ref = "ktor-version" }
kotlin-jvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin-version" }
kotlin-serialization = { id = "org.jetbrains.kotlin.plugin.serialization", version.ref = "serialization-version" }
versions = {id = "com.github.ben-manes.versions", version.ref = "versions-version" }

In build.gradle.kts, we can use these library names to refer to these specific libraries and versions.

dependencies {
plugins {
    // different forms of the same plugin declaration

    // kotlin("jvm") version "1.9.10"
    // id("org.jetbrains.kotlin.jvm") version "1.9.10"
    alias(libs.plugins.kotlin.jvm) // core kotlin

    // kotlin("plugin.serialization") version "1.9.0"
    // id("org.jetbrains.kotlin.plugin.serialization") version "1.9.0"
    alias(libs.plugins.kotlin.serialization) // json-serialization

    alias(libs.plugins.ktor) // provides networking support
    alias(libs.plugins.versions) // checks versions, see help > dependencyUpdates
}

group = "ca.uwaterloo"
version = "1.2"

application {
    mainClass.set("ca.uwaterloo.ApplicationKt")
}

repositories {
    mavenCentral()
}

dependencies {
    implementation(libs.json)
    implementation(libs.clikt)
    implementation(libs.slf4j)

    implementation(libs.ktor.cio)
    implementation(libs.ktor.core)
    implementation(libs.ktor.server)
    implementation(libs.ktor.api)
}

Last word

It’s staggering how much software is available through package repositories…

https://xkcd.com/2347/

Pair programming

Pair programming (‘pairing’) means that two people design and implement code together, on a single machine.

Pairing is a very collaborative way of working that involves a lot of communication and collaboration between team members. While a pair of developers work on tasks together, they do not only write code, they also plan and discuss their work. They clarify ideas, discuss approaches and come to better solutions.

Originally an Extreme Programming practice, it is considered a best-practice today, and used successfully by many programming teams.

Benefits

There are tangible benefits to pair programming—both to the organization and project team, and to the quality of the work produced.

For the organization:

• remove knowledge silos to increase team resiliency
• build collective code ownership
• help with team building
• accelerate on-boarding
• serve as a short feedback loop, similar to a code review happening live

For the team, and the developers:

• improve learning
• increase efficiency
• improve software design
• improve software quality
• reduce the incidence of bugs
• increase satisfaction
• increase the safety and trust of developers pairing
• increase developer confidence

Research supports the idea that a pair of programmers working together will produce higher quality software, in the same or less time as the developers working independently (Williams et al. 2000).

Surprisingly, research also suggests that there is no loss in productivity when pair programming. In other words, they produce at least the same amount of code as if they were working independently, but it tends to be higher quality than if they worked alone.

Styles of Pairing

Styles adapted from Martin Fowler’s article on How to Pair.

Driver and Navigator

This is the classic form of pair programming.

The Driver is the person at the wheel, i.e. the keyboard. She is focussed on completing the tiny goal at hand, ignoring larger issues for the moment. A driver should always talk through what she is doing while doing it.

The Navigator is in the observer position, while the driver is typing. She reviews the code on-the-go, gives directions and shares thoughts. The navigator also has an eye on the larger issues, bugs, and makes notes of potential next steps or obstacles.

Pair Programming (unruly.co)

Remember that pair programming is a collaboration. A common flow goes like this:

• Start with a reasonably well-defined task. Pick a requirement or user story, for example.
• Agree on one tiny goal at a time. This can be defined by a unit test, or by a commit message, or some goal that you’ve written down.
• Switch keyboard and roles regularly. Alternating roles keeps things exciting and fresh.
• As a navigator, leave the details of the coding to the driver—your job is to take a step back and complement your pair’s more tactical mode with medium-term thinking. Park next steps, potential obstacles and ideas on sticky notes and discuss them after the tiny goal is done, so as not to interrupt the driver’s flow.

Ping-Pong

This technique is ideal when you have a clearly defined task that can be implemented in a test-driven way.

• “Ping”: Developer A writes a failing test
• “Pong”: Developer B writes the implementation to make it pass.
• Developer B then starts the next “Ping”, i.e. the next failing test.
• Each “Pong” can also be followed by refactoring the code together, before you move on to the next failing test.

Strong-Style Pairing

This is a technique particularly useful for knowledge transfer. In this style, the navigator is usually the person much more experienced with the setup or task at hand, while the driver is a novice (with the language, the tool, the codebase, …). The experienced person mostly stays in the navigator role and guides the novice.

An important aspect of this is the idea that the driver totally trusts the navigator and should be “comfortable with incomplete understanding.” Questions of “why,” and challenges to the solution should be discussed after the implementation session.

Switching Roles

Here are some suggestions on role-switching from Shopify Engineering.

Switching roles while pairing is essential to the process—it’s also one of the trickiest things to do correctly. The navigator and driver have very different frames of reference.

The Wrong Way

Pairing is about working together. Anything that impedes one of the people from contributing or breaks their flow is bad. Two of the more obvious wrong ways are to “grab the keyboard” or “push the keyboard”.

Grabbing the keyboard: Sometimes when working as the navigator it’s tempting to take the keyboard control away to quickly do something. This puts the current driver in a bad position. Not only are they now not contributing, but such a forceful role change is likely to lead to conflict.

Pushing the keyboard: Other times, the driver feels a strong need to direct the strategy. It’s very tempting to just “push” the keyboard to the navigator, forcing them to take the driver’s seat, and start telling them what to do. This sudden context switch can be jarring and confusing to the unsuspecting navigator. It can lead to resentment and conflict as the navigator feels invalidated or ignored.

Finally, even a consensual role switch can be jarring and confusing if done too quickly and without structure.

The Right Way

The first step to switching roles is always to ask. The navigator needs to ask if they can grab the keyboard before doing so. The driver needs to ask if the navigator is willing to drive before starting to direct them. Sometimes, switching without asking works out, but these situations are the exception.

It’s important to take some time when switching as well. Both pairers need to time to acclimatize to their new roles. This time can be reduced somewhat by having a structure around switching (e.g. Ping-pong pairing) which allows the pairers to be mentally prepared for the switch to happen.

Pair Rotation

If a feature takes a long time, you might consider rotating people into and out of the pair over time (e.g. one person swaps out, and a new person codes). You don’t want to do this frequently, no more than once per day over a full working day. It’s helpful to prevent the duo from becoming stale or frustrated with one another, and may help with knowledge transfer.

In a small team, with brief cycles, this may not be practical or necessary.

Setup for Pairing

Physical Setup

Ideally, you would work together in the same space. It is worth spending some time figuring out a comfortable setup for both of you.

• Make sure both of you have enough space, and that both of you can sit facing the computer.
• Agree on the computer setup (key bindings, IDE etc). Check if your partner has any particular preferences or needs (e.g. larger font size, higher contrast, …)
• It’s common to have a single keyboard, mouse for the driver, but some pairs setup with two keyboards.

Remote Setup

If you’re working remotely, you may not be able to physically sit together. Luckily, there are practical solutions to pairing remotely. For remote pairing, you need a screen-sharing solution that allows you to not only see, but also control the other person’s machine, so that you are able to switch the keyboard.

Challenges

The biggest challenges with pair programming are not code related, but issues with communication and understanding one another. Don’t be afraid to switch roles or take breaks when required, and be patient with one another!

Post-Mortem

If possible, spend some time reflecting on the pairing session when its over. Ask yourself what went well, and what you can improve. This can help clear the air of any tension or issues that came up during the session, and help you improve as a team.

Unit testing

Introduction

We can characterize tests in terms of the type of testing, types of tests and phase where they are executed (see Types of Automation Testing: A Beginner’s Guide).

Forms of Testing

There are two distinct types of requirements that we can identify when specifying a project: functional requirements, related to features or product requirements, and non-functional requirements or qualities of the system that we can measure.

• Functional: test the features and functonality; determine usability, fit-for-purpose.
• Non-functional: examine qualities of a system: performance, scalability, robustness.

When testing applications, both of these are valid. You should be measuring against the initial requirements that you defined for this product, to ensure that requirements are being met.

Types of Tests

There are different purposes to run tests.

• Smoke tests: functional tests only cover crucial functionality; a quick “sanity check” to ensure that the build ran, for instance.
• Integration tests: tests interrelated functionalities to ensure that they work together.
• Regression tests: tests that ensure that software hasn’t gotten “worse” over time (degraded performance, or functionality no longer working).
• Security tests: test the system for any vulnerabilities.
• Performance tests: non-functional tests that evaluate measurable criteria like performance, stability.
• Acceptance tests: functional tests that must be met for software to meet the minimum acceptable bar for users.

Phases of Testing

Finally, there are different areas of testing.

• Unit test: small functional tests, designed to check behaviour at the class-level.
• API test: test the “outer edges” of your architecture; check that APIs work as expected.
• UI test: check that interaction with the user interface functions properly. May be done by-hand, or automated (see mocking).

Are we expected to write all of these tests?!? No, at least not in this course.

You are expected to write a reasonable subset: unit tests and API tests for your architecture. You should be running these tests every time you build, so they can be considered both smoke tests and integration tests (if you run after integrating branches).

Setup JUnit

We’re going to use JUnit, a popular testing framework to create and execute tests. It can be installed in number of ways: directly from the JUnit home page, or one of the many package managers for your platform.

We will rely on Gradle to install it for us as a project dependency. Include this line in the dependencies of your build.gradle.kts configuration file.

dependencies {
    testImplementation('org.jetbrains.kotlin:kotlin-test')
}

Unit Testing

Unit testing involves writing low-level, class-specific tests.

A unit test is a test that meets the following three requirements (Khorikov 2020):

• Verifies a single unit of behavior,
• Does it quickly, and
• Does it in isolation from other tests.

Unit tests should target classes or other low-level components in your program. They should be small, very focused, and quick to execute and return results.

Testing in isolation means removing the effects of any dependencies on other code, libraries or systems. This means that when you test the behaviour of your class, you are assured that nothing else is contributing to the results that you are observing.

This is also why designing cohesive, loosely coupled classes is critical: it makes testing them so much easier if they work independently!

Unit tests are just Kotlin functions that execute and check the results returned from other functions. Ideally, you would produce one unit or more unit tests for each function. You would then have a set of unit tests to check all of the methods in a class, and multiple sets of units tests to cover all of your implementation classes.

Your goal should be to have unit tests for every critical class and function that you produce.

You will NOT likely get 100% code coverage. You should focus on testing entities/data models, and related features. You do NOT have to unit test user interfaces directly (there are ways to do it, but they are outside-scope of this course).

Why (not) TDD?

Test-Driven Development (TDD) is a strategy introduced by Kent Beck, which suggests writing tests first, before you start coding. You write tests against expected behaviour, and then write code which works without breaking the tests. TDD suggests this process:

1. Think about the function (or class) that you need to create.
2. Write tests that describe the behaviour of that function or class. As above, start with valid and invalid input.
3. Your test will fail (since the implementation code doesn’t exist yet). Write just enough code to make the tests pass.
4. Repeat until you can’t think of any more tests to write.

Running out of tests means that there is no more behaviour to implement… and you’re done, with the benefit of fully testable code.

https://medium.com/@tunkhine126/red-green-refactor-42b5b643b506

There are some clear benefits to testing early:

• Early bug detection. You are building up a set of tests that can verify that your code works as expected.
• Better designs. Making your code testable often means improving your interfaces, having clean separation of concerns, and cohesive classes. Testable code is by necessity better code.
• Confidence to refactor. You can make changes to your code and be confident that the tests will tell you if you have made a mistake.
• Simplicity. Code that is built up over time this way tends to be simpler to maintain and modify.

Structure

A typical unit test uses a very particular pattern, known as the arrange, act, assert pattern. This suggests that each unit test should consist of these three parts:

1. Arrange: bring the system under test (SUT) to the starting state.
2. Act: call the method or methods that you want to test.
3. Assert: verify the outcome of the above action. This can be based on return values, or some other conditions that you can check.

Note that your Act section should have a single action, reflecting that it’s testing a single behaviour. If you have multiple actions taking place, that’s a sign that this is probably an integration test (see below). As much as possible, you want to ensure that you are writing minimal-scope unit tests.

Another anti-pattern is an if statement in a test. If you are branching, it means that you are testing multiple things, and you should really consider breaking that one test into multiple tests instead.

Writing tests

A unit test is just a Kotlin class, with annotated methods that tell the compiler to treat the code as a test. It’s best practice to have one test class for each implementation class that you want to test e.g., class Main has a test class MainTest. This test class would contain multiple methods, each representing a single unit test.

Tests should be placed in a special test folder in the Gradle directory structure. When building, Gradle will automatically execute any tests that are placed in this directory structure.

Your unit tests should be structured to create instances of the classes that they want to test, and check the results of each function to confirm that they meet expected behaviour.

For example, here’s a unit test that checks that the Model class can add an Observer properly. The addObserver() method is the test (you can tell because it’s annotated with @Test). The @Before method runs before the test, to setup the environment. We could also have an @After method if needed to tear down any test structures.

class ObserverTests() {
    lateinit var model: Model
    lateinit var view: IView

    class MockView() : IView {
        override fun update() {
        }
    }

    @Before
    fun setup() {
        model = Model()
        view = MockView()
    }

    @Test
    fun addObserver() {
        val old = model.observers.count()
        model.addObserver(view)
        assertEquals(old+1, model.observers.count())
    }

Gradle will automatically execute and display the test results when you build your project.

Lets walkthrough creating a test.

To do this in IntelliJ, select a class in the editor, press Alt-Enter, and select “Create Test” from the popup menu. This will create a unit test using JUnit, the default test framwork. There is a detailed walkthrough on the IntelliJ support site.

The convention is to name unit tests after their class they’re testing, with “Test” added as a suffix. In this example, we’re creating a test for a Model class so the test is automatically named ModelTest. This is just a convention—you can name it anything that you want.

We’ve manually added the addition() function. We can add as many functions as we want within this class. By convention, they should do something useful with that particular class.

Below, class ModelTest serves as a container for our test functions. In it, we have two unit tests that will be automatically executed when we built. NOTE that you would need more than these two tests to adequately test this class—this is just an example.

import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Before
import org.junit.Test

class ModelTests {
    lateinit var model: Model

    @Before
    fun setup() {
        model = Model()
        model.counter = 10
    }

    @Test
    fun checkAddition() {
        val original = model.counter
        model.counter++
        assertEquals(original+1, model.counter)
    }

    @Test
    fun checkSubtraction() {
        val original = model.counter
        model.counter--
        assertEquals(original-1, model.counter)
    }

    @After
    fun teardown() {
    }
}

The kotlin.test package provides annotations to mark test functions, and denote how they are managed:

In our test, we call utility functions to perform assertions of how the function should successfully perform.

Running tests

Tests will be run automatically with gradle build or we can execute gradle test to just execute the tests.

$ gradle test
BUILD SUCCESSFUL in 760ms
3 actionable tasks: 3 up-to-date

Gradle will report the test results, including which tests - if any - have failed.

You can also click on the arrow beside the test class or name in the editor. For example, clicking on the arrow in the gutter would run this addObserver() test.

Integration Testing

A unit test is a test that meets these criteria (from the previous chapter):

• Verifies a single unit of behaviour (typically a class),
• Does it quickly, and
• Does this in isolation from other dependencies and other tests.

An integration test is a test that fails to meet one or more of these criteria. In other words, if you determine that you need to test something outside the scope of a unit test, it’s considered an integration test (typically because it’s integrating behaviours from multiple components). Performance tests, system tests, etc. are all kinds of integration tests.

There are a lot of different nuanced tests that are used in computer science, but we’ll focus on building generic integration tests.

Typically, an integration test is one where you leave in selected dependencies so that you can test the combination of classes together. Integration tests are also suitable in cases where it is difficult to completely remove a dependency. This can happen with some critical, external dependencies like an external database.

This diagram demonstrates how unit tests primarily test the domain model, or business logic classes. Integration tests focus on the point where these business logic classes interact with external systems or dependencies.

Note that in this diagram, we’re also identifying code that we should not bother testing. Trivial code is low complexity, and typically has no dependencies or external impact, so it doesn’t require extensive testing. Overcomplicated code likely has so many dependencies that it’s nearly impossible to test - and it should likely be refactored into something similar and more manageable before you attempt to add tests to it.

Guidelines

Here’s some guidelines for creating integration tests.

1. Make domain model boundaries explicit.

Try to always have an explicit, well-known place for the domain model in your code base. The domain model is the collection of domain knowledge about the problem your project is meant to solve. This can be a separate class, or even a package reserved for the model.

2. Reduce the number of layers of abstraction

Try to have as few layers of indirection as possible. In most backend systems, you can get away with just three: the domain model, application services layer (controllers), and infrastructure layer. Having excessive layers of abstraction can lead to bloated and confusing code.

3. Eliminate circular dependencies

Try to keep dependencies flowing in a single direction. i.e. don’t have class A call into class B, and class B call back into class A with results. Circular dependencies like this create a huge cognitive load for the person reading the code, and make testing much more difficult.

Dependencies

The idea of a dependency is central to our understanding of how to isolate a particular class or set of classes for testing.

When you are examining a software component, we say that your component may be dependent on one or more other software entities to be able to run successfully. For example, you may need to link in another library, or import other classes, or connect to another software system (like a database). Each of these represents code that affects how the code being tested will execute.

A key strategy when testing is to figure out how to control these dependencies, so that you’re exercising your class independently of the influence of other components.

Concepts

We can further differentiate these types of dependencies:

Managed vs. Unmanaged dependencies. There is a difference between those that we control directly (managed), vs. those that may be shared with other software. A managed dependency suggests that we control the state of that system.

• Examples: a library that is statically linked is managed; a dynamically linked library is not. A database could be single-file and used only for your application (managed) or shared among different applications (unmanaged).

Internal vs. External dependencies. Running in the context of our process (internal) or out-of-process (external). A library is internal, a database is typically external.

• Examples: A library, regardless of whether it is managed, is internal. A database, as a separate process is always external.

It is important to think about these distinctions for our dependencies because they affect what we can and cannot control during testing. For example, an unmanaged dependency means that we do not control it’s state and we may not be able to test that specific component or dependency. The best we could do it test our interface against it, since we may not be able to trust the results of any actions that we take against it.

Generally speaking, we cannot test unmanaged dependencies since we cannot control them. We also tend to be limited in our ability to test external systems, since we do not manage their state.

Test doubles

To achieve isolation in testing, we often create test doubles — classes that are meant to look like a dependent class, but that don’t actually implement all of the underlying behaviour. This lets us swap in these “fake” classes for testing.

A mock is a fake object that holds the expected behaviour of a real object but without any genuine implementation. For example, we can have a mocked File System that would report a file as saved, but would not actually modify the underlying file system.

You can fairly easily create these mock classes yourself for code domain objects.

Several libraries have also been established to help create mocks of objects. Mockito is one of the most famous, and it can be complemented with Mockito-Kotlin. Here’s an example of a mock File that reports a path, but doesn’t actually do anything else.

private val mockedFile: File {
	return mock { on { absolutePath} doReturn "/random"}
}

The value in mocks is that they break the dependency between your method-under-test (MUT) and any external classes, by replacing the external dependency with a “fake class” with predetermined behaviour that helps you test.

The value of interfaces

One important recommendation is that you introduce interfaces for out-of-class dependencies. For instance, you will often see code like this:

public interface IUserRepository
public class UserRepository : IUserRepository

This is common when testing, even in cases when that class may represent the only realization of an interface. This allows you to easily write mocks against the interface, where it’s relatively easy to determine what expected behaviour should be.

Dependency injection

The second half of this is dependency injection. This is the practice of supplying dependencies to an object in it’s argument list instead of allowing the object to create them itself. If you design your classes this way, then it’s easy to create an instance of a mock and provide that to your function, instead of allowing the function to instantiate the object itself.

Dependency injection is basically providing the objects that an object needs (its dependencies) instead of having it construct them itself. It’s a very useful technique for testing, since it allows dependencies to be mocked or stubbed out.

https://stackoverflow.com/questions/130794/what-is-dependency-injection

For instance, we might have a class that uses a database for persistence:

class Persistence {
  val repo = UserRepository() // dependency

  fun saveUserProfile(val user: User) {
    repo.save(user)
  }
}

val persist = Persistence()
persist.saveUserProfile(user) // save using the real database

Imagine that we want to test our saveUserProfile method. This would be extremely difficult to do cleanly, since we have a dependency on the repo that our Persistance class creates.

We could instead change our Persistence class so that we pass in the dependency. This allows us to control how it is created, and even replace the UserRepository() with a mock.

class Persistence(val repo: IUserRepository) {
    fun saveUserProfile(val user: User) {
    repo.save(user)
  }
}

class MockRepo : IUserRepository {
	// body with functions that mirror how the repo would work
  // but no real implementation
}
val mock = MockRepo()
val persist = Persistance(mock)
persist.saveUserProfile(user) // save using the mock database

Obviously you don’t want to mock everything - that would turn this into a unit test! However, you should use this technique to isolate the classes that you are using, and test just the intended path.

How many tests?

When discussing unit tests, we suggested that you should focus on core classes and their behaviours. This is reasonable for unit tests.

We can expand this to suggest that you should “check as many of the business scenario’s edge cases as possible with unit tests; use integration tests to cover one happy path, as well as any edge cases that can’t be covered by unit tests.”

A “happy path” in testing is a successful execution of some functionality. In other words, once your unit tests are done, you should write an integration tests that exercises the functionality that a customer would likely exercise if they were using your software with common features and a common workflow. Focus on that first and only add more integration tests once you have the main execution path identified and tested.

The primary purpose of integration tests is to exercise dependencies, so that should be your main goal. Your main integration test (the “happy path test”) should exercise all external dependencies (libraries, database etc). If it cannot satisfy this requirement, add more integration tests to satisfy this constraint.

Code coverage

Tests shouldn’t verify units of code. Rather, they should verify units of behaviour: something that is meaningful for the problem domain and, ideally, something that a business person can recognize as useful. The number of classes it takes to implement such a unit of behaviour is irrelevant. — Khorikov (2020)

Code coverage is a metric comparing the number of lines of code with the number of lines of code that have unit tests covering their execution. In other words, what “percentage” of your code is tested?

This is a misleading statistic at the best of times (we can easily contrive cases where code will never be executed or tested).

TDD would suggest that you should have 100% unit test coverage but this is impractical and not that valuable. You should focus instead on covering key functionality. e.g. domain objects, critical paths of your source code.

One recommendation is to look at the coverage tools in IntelliJ, which will tell you how your code is being executed, as well as what parts of your code are covered by unit tests. Use this to determine which parts of the code should be tested more completely.

See also

• https://www.jetbrains.com/help/idea/code-coverage.html
• https://www.jetbrains.com/help/idea/running-test-with-coverage.html

Last Word

https://xkcd.com/2200

Software release

At the end of each iteration, you should be producing a software release, and publishing it on your project page.

What is a software release?

A software release is process where you halt development, and generate an installable version of your product that contains specific, known and tested functionality. Typically this is done so that you can provide them with a working copy of your software either for further testing, or actual use/deployment.

Before producing a software release:

• You have completed all of the features that you intended to complete OR you are at the end of your interation.
• Your code is fully committed, and merged on a release branch.
• All tests should be passing correctly on that release branch.

A software release should include:

• An installable version of your software.
• Release notes detailing what has changed in this release.

A software release should ALWAYS be well-tested and high-quality, and something that you could give to a user to install and test.

Why do we produce these releases?

There’s multiple reasons why we do this.

Iterative development

We also produce software releases as a way of being honest with ourselves. It forces us to do mundane but important tasks like:

• Keeping issues in GitLab up-to-date.
• Merging features and code changes more frequently than we might otherwise do. Frequent changes help us avoid code drift (and costly/complex merges later).
• Ensuring that our product can actually be installed in a production environment (vs. a “it works in the IDE!” mentality).
• Checking in with our customers! When used alongside regular demos, they are an important part of our feedback cycle.

Without software releasess (i.e. “lines in the sand”) it’s easy to let these things slide until the project becomes impossible to control.

Tracking progress towards a final release

In Agile development, your goal is to get to a point where every release is a potential candidate release for a customer to deploy. In other words, it should be high enough quality to actually be used in production, with “real” data. Each software release should produce a product that is closer to a “finished” state.

Having an actual installable release is great for our customers since they can install and test it themselves. It gives them confidence that we’re making progress.

Providing project checkpoints

Once we have more broadly released our product to customers, and people are actually using it, we need to be able to “go back” to prior releases and know definitively what was released.

• e.g., imagine that you release version 1.1 to a customer, and they use it for 3 months before reporting a bug. How do you fix it? The first step is knowing exactly what source code was used, and what the state of your codebase was at that point in time (i.e., v 1.1). Without easily-identifiable releases, you are guessing at what went wrong.

Types of releases

You will often hear about different kinds of releases: alpha, beta and so on. What do we mean by these terms?

From launchdarkly.com:

• Pre-alpha: This stage encompasses all activities that lead up to a major release, including gathering requirements, deciding on the pricing for new features, design, and development.
• Alpha: This is usually where testing and validation of the new code’s behavior begins. Alpha releases are typically tested within the company.
• Beta: Once software passes alpha, it enters the beta stage. Some companies share the beta with prospects and customers as a demo, or make it available for beta testing by a group of users who opt in. This can be a good way to engage your power users or people interested in early access to new features.
• Release candidates: Once any bugs surfaced in beta testing have been addressed, a release candidate is ready to undergo evaluation for stability and suitability as the final version of the software to be released.
• Final release: If no significant bugs emerge when testing the release candidate, you’re ready to release the final version. When and how you do that will depend on your methodology and deployment and release strategies (we’ll dig into those below).

In this course, our development cycles are quite short, so we won’t formally use this terminology. You could consider that everything that we produce is going to be a beta, until the final release.

Producing a release

Here’s what you should do each release.

1. Consolidate code changes.

   Commit all completed code changed and merge back to your release branch (main unless you are told otherwise). Make sure that all automated (unit) tests pass.

2. Assign a version number.

   In your build.gradle.kts, you should change the VERSION to the next valid product version number.

   • We recommend incrementing the second digit for a beta release; your final release would be version 1.0.
   • e.g., Demo 1 is 0.10, Demo 2 is 0.20, …, final release is 1.0. See semantic versioning.

3. Generate a release in GitLab (docs).

   You can start this through Deploy > Release. This will (a) tag your code at this point in time, (b) create an archive of your source code from that tag (note: with proper setup, it’s possible to have GitLab build your project installers, but we’ll do this step manually below).

4. Write release notes.

   Each software release should be documented in a Wiki page, named according to the release number e.g., Version 1.0.0 Release. You should have a section titled Releases in your README.md with a link to this page (and all previous releases).

   The Wiki page for a release needs to include:

   • The release date, which should be the date that the installer was generated. e.g., 12-May-2023
   • The version of this release (which you increment for each release. e.g., 1.1.0 for your first sprint.
   • A bullet list of the major changes that were included in the release, and a link to the issues list in GitLab.

5. Add installer images

   The installer should be a packaged version of your application that a user could use to install your software. If your application consists of multiple components that need to be installed separately (e.g. Android AND desktop) then you need to provide installers for each one.

   Acceptable forms of packaging include:

   • Android: an apk file generated from IntelliJ IDEA or Android Studio that can be side-loaded into a virtual device.
   • Desktop: an exe, dmg or pkg file that installs the application.
   • Service: a Docker image that contains your service, with instructions on how to run it (if you are hosting it in the cloud, then you don’t need a Docker image).

   The last step is to attach these images to the wiki page, so that user can locate them in the wiki and install them directly.

   Important

   When you demo to your TA, you must demo from a software release! You will lose significant marks if you run your code from the IDE or an editor.

Software development

Topics related to application development.

Getting started

Technology choices

We need to consider the environments in which we’re operating! This includes both our development and deployment tech stacks (operating system, programming language, libraries).

What’s Next?

We have some additional considerations, since this is an academic environment (and not a commercial product for example).

• We want you to have the option of mobile or desktop development, so we need a tech stack that supports both.
• We also need to make sure that everyone in the course can run the development tools. This means that we cannot use Apple-specific toolchain (which are only available to people using macOS).

Given all of above, we’re going to use Kotlin and build Android applications and desktop applications.

Kotlin is a modern, application focused programming language with a lot of amazing features. We’ll learn how to use Kotlin to build well-designed applications that meet the goals we’ve outlined. This includes discussions of architectural styles, useful design principles, design patterns and best practices like pair programming, unit testing and code reviews.

We’ll also discuss how to implement key features like user-interfaces, database connectivity and web services using industry-standard toolkits e.g. Compose for UI.

Finally, we will do all of this in the context of best team practices. We will take steps to ensure effective team communication and collaboration, and identify things that you can do to work effectively together. These are extremely important skills to develop as we start building larger and more complex systems.

Final Word

https://imgs.xkcd.com/comics/xkcd_stack.png

Toolchain installation

This section describes the software that you will to install in order to work on your project.

Minimum Requirements

To participate in coding and other activities, each person on your team requires access to a modern computer running Windows, Linux or macOS. 8GB of RAM or more is recommended, and you must have administrative rights on this machine (which disqualifies the use of lab computers). A notebook computer is preferred, since you will be working on your project in-class.

Note that a physical phone is NOT required for Android development, since you can use a virtual device (AVD). This will be demonstrated in-class.

What to Install?

The following software is used in this course and represents the minimum tech stack that you must use.

• Install manually indicates that you should install from the link provided.
• IntelliJ plugin indicates that this is bundled with the IntelliJ IDEA IDE. Once the IDE is installed, you can check that this plugin is installed from Settings > Plugins. You should always use the most recent version of any plugin.
• Installed by Gradle means that we will configure our build system to automatically download and manage this dependency. You don’t need to install it manually.

Desktop Development

Android Development

Installing the newest version of any library is usually the best strategy, but be careful about compiler/library compatibility. The Kotlin and JDK versions in particular need to be compatible versions.

See our Libraries reference for more information on libraries that you can import.

We recommend IntelliJ IDEA for desktop development ,and Android Studio for Android development (one is a fork of the other, but Android Studio also includes the Android SDK which you need for Android development).

Configuration

There are two types of settings in your IDE.

IDE product settings

• Changes you make in your editor/IDE preferences, typically in the Settings dialog. Examples: font size, code style, key bindings, etc.
• Saved in your .idea. directory in your project folder.
• Normally should NOT be saved to Git, since these represent your personal preferences.

Project settings

• Settings that affect how your project builds and executes. Examples: JDK version, Gradle settings.
• Saved in your build.gradle.kts file or other project files.
• These absolutely SHOULD be saved to Git, since they affect how your project is built.

My Settings

I always recommend setting up a few things in IntelliJ IDEA to make it easier to use.

The first thing I tend to do is to specify how much memory the IDE can use. I have 16 GB of RAM, so I can dedicate 4 GB to IntelliJ IDEA. This drastically increases performance.

• Help > Change Memory Settings > 4096 MB

Next, I setup the environment to maximize my editing window (and I use hotkeys to toggle windows and navigate).

• View > Appearance > Compact Mode
• View > Appearance > Toolbar ON
• View > Appearance > Tool Window Bars OFF
• View > Appearance > Navigation Bar > In Status Bar

Finally, I install a few plugins that I find useful. You can install plugins from the JetBrains Plugin Repository, under Settings > Plugins.

Required for desktop:

• Compose Multiplatform IDE support

Required for Android:

• Android // android only
• Android Design Tools // android only
• Jetpack Compose

Recommended:

• Mermaid.js
• Rainbow Brackets
• Database Tools and SQL
• Docker

Last Word

We recommend an IDE like IntelliJ IDEA or Android Studio, but hey, you do you.

https://xkcd.com/378

Source code setup

Setup your Source Code

Step 1. Getting a working copy of your Git repository

First, you need to git clone your GitLab repository to your local machine so that you have a working copy.

Open the web page for your GitLab project. Click on the Code button, and copy the URL from Clone with HTTPS.

In a terminal, on your computer, cd to the location where you want to keep your source code. git clone the URL.

$ git clone https://git.uwaterloo.ca/cs346/public/mm.git

This would produce a folder named mm that contains the contents of my Git repository.

Step 2. Create an empty project.

You will want to create a project directly in your Git working copy from the previous step.

IntelliJ IDEA and Android Studio both fully support Gradle, and we can create a new project directly in the IDE.

IntelliJ IDEA (Desktop)

From the Splash screen, select New Project.

In the project wizard, choose New Project and supply the following project parameters.

• Kotlin as your language (from the list on the left).
• Name is the name of the folder that will contain your source code. e.g., source.
• Location is the location of your working copy (from Step 1 above).
• Gradle for your build system
• JDK should point to your JDK installation (see toolchain).
• Gradle DSL should be Kotlin as well.

Click Create to proceed. If successful, IntelliJ IDEA will open into the main window to an empty project.

You should be able to click the Run button in the toolbar to execute it.

Android Studio (Android)

From the Splash screen, select New Project.

Select Empty Activity and Next. A second screen will prompt you for project parameters.

• Name: Your application name
• Package name: A unique package name.
• Save location: Your working copy location.
• Minimum SDK: API 26 or later.
• Build configuration language: Kotlin DSL.

Click Finish and your project should be created.

You will need to add an Android Virtual Device for testing:

• Tools > Device Manager
• +, Create Virtual Deviceand walk through the wizard to add an emulated device for testing.

Step 3. Add a gitignore file

You should probably add a .gitignore file to the top level of your project, specifying which files to NOT include in your repository.

Typically this would be a list of temporary build directories of any local configuration files. For example.

$ cat .gitignore
build/
out/
*.class
*.tmp
.DS_Store
.idea

Step 4. Push changes to the repository

Once you have confirmed that your project is working, you can commit and push the changes.

$ git add *
$ git commit -m "Initial commit"
$ git push

Your teammates should now be able to git pull to fetch your changes!

Navigating Projects

IntelliJ has a number of windows that it will display by default. You can toggle project windows.

• Project: a list of all files (Cmd+1).
• Structure: methods and properties of the current open class/source file (Cmd+7).
• Source: the current source files (no hotkey).
• Git: Git status and log (Cmd+9) - not shown.
• Gradle: tasks that are available to run (no hotkey) - not shown.

Running Sample Code

We maintain a public Git repository of the source code shown in lectures. To get a copy, git clone the repository URL. This command, for instance, would create a copy of the course materials in a directory named cs346.

$ git clone https://git.uwaterloo.ca/cs346/public/ cs346

The slides folder contains sample code to accompany some of the slide decks. You can build and execute these projects directly in IntelliJ:

• File -> Open and navigate to the top-level directory containing the build.gradle.kts file. Do NOT open a specific file, just the directory. Click Ok.
• Click on the Run button in either IntelliJ or Android Studio to run the project.

Further Help

There are many excellent online resources for using IntelliJ. I’d suggest starting with the JetBrains IntelliJ Documentation and online help.

• Getting Started
• List of Official Resources
• IntelliJ IDEA YouTube Channel

Kotlin programming

Kotlin basics

Why Kotlin?

Kotlin is a modern, general-purpose programming language designed by JetBrains as a drop-in replacement for Java. It has full compatibility with Java source code, and is 100% compatible with that ecosystem.

Over time, it has extended far past it’s original goals. It’s seen broad industry support, and has proven to be a versatile language for full-stack development.

• As of 2019, it’s Google’s recommended language for building Android applications. Android development, including third-party libraries, is done in Kotlin¹.
• It’s fully capable back-end language for building servers and services, and it’s compatibile with existing service frameworks e.g., Spring Boot, Ktor.
• Finally, it’s designed as a cross-platform language, with native compilers for many platforms . We use it in this course for desktop and Android. iOS support is in beta², and WASM/web is also on the horizon.

Compiling

The Kotlin programming language includes a set of compilers that can be used to target different platforms.

flowchart LR
    kotlin([Kotlin Code])
    jvm([Kotlin/JVM])
    native([Kotlin/Native])
    js([Kotlin/JS])
    wasm([Kotlin/WASM])
    kotlin --> jvm
    kotlin --> native
    kotlin --> js
    kotlin --> wasm

• Kotlin/JVM compiles Kotlin to JVM bytecode, which can be interpreted on a Java virtual machine. This supports running Kotlin code anywhere a JVM is supported (typically desktop, server).
• Kotlin/Native compiles Kotlin to native executables. This provides support for iOS and other targets.
• Kotlin/JS transpiles Kotlin to JavaScript. The current implementation targets ECMAScript 5.1.
• Kotlin/WASM adds support for WASM virtual machine standard, allowing Kotlin to run on the web.

Kotlin/JVM and Kotlin/Native for Android are the best-supported targets at this time, and what we will focus on in this course.

A Kotlin application can be compiled to JVM or native code. Kotlin’s application code looks a little like C, or Java. Here’s the world’s simplest Kotlin program, consisting of a single main method.

fun main() {
	val message="Hello Kotlin!"
	println(message)
}

To compile from the command-line, we can use the Kotlin compiler, kotlinc. By default, it takes Kotlin source files (.kt) and compiles them into corresponding class files (.class) that can be executed on the JVM.

$ kotlinc Hello.kt

$ ls
Hello.kt HelloKt.class

$ kotlin HelloKt
Hello Kotlin!

Notice that the compiled class is named slightly differently than the source file. If your code isn’t contained in a class, Kotlin wraps it in an artificial class so that the JVM (which requires a class) can load it properly. Later when we use classes, this won’t be necessary.

This example compiles Hello.kt into Hello.jar and then executes it:

$ kotlinc Hello.kt -include-runtime -d Hello.jar

$ ls
Hello.jar Hello.kt

$ java -jar Hello.jar
Hello Kotlin!

Most of the time, we won’t compile from the command-line; instead, we will use a build system to generate an installable package for our platform e.g., desktop or Android. See Gradle.

Modularity

Packages

When defining related classes or functions within an application, it’s best to group them together in a package.

Packages are meant to be a collection of related classes. e.g. graphics classes. They are conceptually similar to namepsaces in C++. We use them to enforce a clear separation of concerns within code.

Packages are named using a reverse DNS name, typically the reverse-domain name of company that owns the code, followed by a unique name related to the code’s functionality. e.g. com.sun.graphics for a graphics package developed at Sun Microsystems, or ca.uwaterloo.cs346 for code related to this course.

To create and use a package, you need to:

1. Add the package declaration to the top of a source file to assign that file to a namespace. Classes or modules in the same package have full visibility to each other.

For example, we have declared this file to be included in the cs.uwaterloo.cs346 package. This means that any classes in this file will be included in that package, and will be visible to any other functions or classes in that package, even if they are in different files.

package ca.uwaterloo.cs346

data class Point(x: Float, y: Float) { }
data class Vector(x: Float, y: Float, m: Float) { }

1. Use the import keyword to bring classes from other packages into the current namespace. This allows you to use classes from other packages in your code.

For example, to use our Point class in another file, we would import it like this. Note that the Vector class isn’t imported, so it won’t be available in this file.

import ca.uwaterloo.cs346.Point

fun main() {
  val p1 = Point(5.0, 10.0)
  val p2 = Point(15.0, 20.0)
}

To include all classes in a package, you can use the * wildcard. This would make both Point and Vector visible.

import ca.uwaterloo.cs346.*

Modules

Modules serve a different purpose than packages: they are intended to expose permissions for external dependencies. Using modules, you can create higher-level constructs (modules) and have higher-level permissions that describe how that module can be reused. Modules are intended to support a more rigorous separation of concerns that you can obtain with packages.

A simple application would be represented as a single module containing one or more packages, representing different parts of our application.

JAR Files

A Kotlin application might consist of many classes, each compiled into it’s own .class file. To make these more manageable, the Java platform includes the jar utility, which can be used to combine your classes into a single archive file i.e. a jar file. Your application can be executed directly from that jar file. This is the standard mechanism in the Java ecosystem for distributing applications.

A jar file is just a compressed file (like a zip file) which has a specific structure and contents, and is created using the jar utility. This example compiles Hello.kt, and packages the output in a Hello.jar file.

$ kotlinc Hello.kt -include-runtime -d Hello.jar

$ ls
Hello.jar Hello.kt

The -d option tells the compiler to package all of the required classes into our jar file. The -include-runtime flag tells the compiler to also include the Kotlin runtime classes (which contain things like the Garbage Collector).

These classes are needed for all Kotlin applications, and they’re small, so you should always include them in your distribution (if you fail to include them, you app won’t run unless your user has Kotlin installed).

To run from a jar file, use the java command.

$ java -jar Hello.jar
Hello Kotlin!

In the same way that the Java compiler compiles Java code into IR code that will run on the JVM, the Kotlin compiler also compiles Kotlin code into compatible IR code. The java command will execute any IR code, regardless of which programming language and compiler was used to produce it.

Our JAR file from above looks like this if you uncompress it:

$ unzip Hello.jar -d contents
Archive:  Hello.jar
  inflating: contents/META-INF/MANIFEST.MF
  inflating: contents/HelloKt.class
  inflating: contents/META-INF/main.kotlin_module
  inflating: contents/kotlin/collections/ArraysUtilJVM.class
  ...

$ tree -L 2 contents/
.
├── META-INF
│   ├── MANIFEST.MF
│   ├── main.kotlin_module
│   └── versions
└── kotlin
    ├── ArrayIntrinsicsKt.class
    ├── BuilderInference.class
    ├── DeepRecursiveFunction.class
    ├── DeepRecursiveKt.class
    ├── DeepRecursiveScope.class
    ...

The JAR file contains these main features:

• HelloKt.class – a class wrapper generated by the compiler
• META-INF/MANIFEST.MF – a file containing metadata.
• kotlin/ – Kotlin runtime classes not included in the JDK.

The MANIFEST.MF file is autogenerated by the compiler, and included in the JAR file. It tells the runtime which main method to execute. e.g. HelloKt.main().

$ cat contents/META-INF/MANIFEST.MF
Manifest-Version: 1.0
Created-By: JetBrains Kotlin
Main-Class: HelloKt

Library support

Kotlin benefits from some of the best library support in the industry, since it can leverage any existing Java libraries as well as Kotlin standard libraries. This means that you can use any Java library in your Kotlin code, and you can also use any Kotlin library that has been developed.

Kotlin Standard Library

The Kotlin Standard Library is included with the Kotlin language, and contained in the kotlin package. This is automatically imported and does not need to be specified in an import statement.

Some of the features that will be discussed below are actually part of the standard library (and not part of the core language). This includes essential classes, such as:

• Higher-order scope functions that implement idiomatic patterns (let, apply, use, etc).
• Extension functions for collections (eager) and sequences (lazy).
• Various utilities for working with strings and char sequences.
• Extensions for JDK classes making it convenient to work with files, IO, and threading.

Java Standard Library

Kotlin is completely 100% interoperable with Java, so all of the classes available in Java/JVM can also be imported and used in Kotlin.

// import all classes in the java.io package
// this allows us to reference any classes in this namespace
import java.io.*

// we can also just import a single class
// this allows us to refer to just the ListView class in code
import javafx.scene.control.ListView

// Kotlin code calling Java IO libraries
import java.io.FileReader
import java.io.BufferedReader
import java.io.FileNotFoundException
import java.io.IOException
import java.io.FileWriter
import java.io.BufferedWriter

if (writer != null) {
	writer.write(
  	row.toString() + delimiter +
    	s + row + delimiter +
      pi + endl
  )

Importing a class requires your compiler to locate the file containing these classes! The Kotlin Standard Library can always be referenced by the compiler, and as long as you’re compiling to the JVM, the Java class libraries will also be made available.

Third-Party Libraries

To use any other Java or Kotlin library, you will to add it to your Gradle dependencies in your build.gradle.kts. This makes them available to import in your source code

// build.gradle.kts

dependencies {
  implementation("com.github.ajalt.clikt:clikt:4.2.2")
}

Once the library is added to your dependencies, you can import it in your source code:

import com.github.ajalt.clikt.core.CliktCommand

fun process() {
  val command = object : CliktCommand() {
    override fun run() {
      echo("Hello, world!")
    }
  }
  command.main(arrayOf())
}

See Gradle > Dependency Management for more information on how to manage dependencies in Kotlin.

Kotlin Basics

Kotlin is a general-purpose, class-based object-oriented language. Although not a functional language, it includes some functional design elements (e.g. higher order functions). Syntactically, it is similar to other C-style programming languages, with a number of modern advances.

Practically, it is an excellent general-purpose language that can replace Java in production environments.

Types

A type system is the set of rules that are applied to expressions in a propgramming language language. We differentiate different type systems by the types of rules that they apply:

• Strong typing: The language has strict typing rules, which are typically enforced at compile-time. The exact type of variable must be declared or fixed before the variable is used. This has the advantage of catching many types of errors at compile-time (e.g. type-mismatch).
• Weak typing: These languages have looser typing rules, and will often attempt to infer types based on runtime usage. This means that some categories of errors are only caught at runtime.

Kotlin is a strongly typed language, where variables need to be declared, or the type known, at compile time. If a type isn’t strictly provided, Kotlin will infer the type at compile time (similar to ‘auto‘ in C++). The compiler is strict about this: if the type cannot be inferred at compile-time, an error will be thrown.

Variables

Kotlin uses the var keyword to indicate a variable, and Kotlin expects variables to be declared before use. Types are always placed to the right of the variable name. Types can be declared explicitly, but will be inferred if the type isn’t provided.

fun main() {
  var a:Int = 10
  var b:String = "Jeff"
  var c:Boolean = false

  var d = "abc"	   // inferred as a String
  var e = 5        // inferred as Int
  var f = 1.5      // inferred as Float
}

All standard data-types are supported, and unlike Java, all types are objects with properties and behaviours. This means that your variables are objects with methods! E.g. "10".toInt() does what you would expect.

Supported Types

Integers

Floating Point Types

Boolean

The type Boolean represents boolean objects that can have two values: true and false. Boolean has a nullable counterpart Boolean? that also has the null value.

Built-in operations on booleans include:

• || – disjunction (logical OR)
• && – conjunction (logical AND)
• !- negation (logical NOT)

• || and && work lazily.

Strings

Strings are often a more complex data type to work with. In Kotlin, they are represented by the String type, and are immutable. Elements of a string are characters that can be accessed by the indexing operation: s[i], and you can iterate over a string with a for-loop:

fun main() {
  val str = "Sam"
  for (c in str) {
      println(c)
  }
}

click to run

You can concatenate strings using the + operator. This also works for concatenating strings with values of other types, as long as the first element in the expression is a string (in which case the other element will be case to a String automatically):

fun main() {
  val s = "abc" + 1
  println(s + "def")
}

click to run

Kotlin supports the use of string templates, so we can perform variable substitution directly in strings. It’s a minor but incredibly useful feature that replaces the need to concatenate and build up strings to display them.

fun main() {
  println("> Kotlin ${KotlinVersion.CURRENT}")

  val str = "abc"
  println("$str.length is ${str.length}")

  var n = 5
  println("n is ${if(n > 0) "positive" else "negative"}")
}

click to run

is and !is operator

To perform a runtime check whether an object conforms to a given type, use the is operator or its negated form !is:

fun main() {
  val obj = "abc"

  if (obj is String) {
    print("String of length ${obj.length}")
  } else {
    print("Not a String")
  }
}

click to run

In most cases, you don’t need to use explicit cast operators in Kotlin because the compiler tracks the is checks and explicit casts for immutable values and inserts (safe) casts automatically when needed:

fun main() {
  val x = "abc"
  if (x !is String) return
  println("x=${x.length}") // x is automatically cast to String

  val y = "defghi"
  // y is automatically cast to string on the right-hand side of `||`
  if (y !is String || y.length == 0) return
  println("y=${y.length}") // y must be a string with length > 0
}

click to run

Immutability

Kotlin supports the use of immutable variables and data structures (mutable means that it can be changed; immutable structures cannot be changed after they are initialized). This follows best-practices in other languages (e.g. use of final in Java, const in C++), where we use immutable structures to avoid accidental mutation.

• var: a standard mutable variable that can be changed or reassigned.
• val: an immutable variable that cannot be changed once initialized.

var a = 0       // type inferred as Int
a = 5           // a is mutable, so reassignment is ok

val b = 1	    // type inferred as Int as well
// b = 2	    // error because b is immutable

var c:Int = 10  // explicit type provided in this case

click to run

Operators

Kotlin supports a wide range of operators. The full set can be found on the Kotlin Language Guide.

• +, -, *, /, %- mathematical operators
• = assignment operator
• &&, ||, !- logical ‘and,’ ‘or,’ ‘not’ operators
• ==, != — structural equality operators compare members of two objects for equality
• ===, !==- referential equality operators are true when both sides point to the same object.
• [, ] — indexed access operator (translated to calls of get and set)

NULL Safety

NULL is a special value that indicates that there is no data present (often indicated by the null keyword in other languages). NULL values can be difficult to work with in other programming languages, because once you accept that a value can be NULL, you need to check all uses of that variable against the possibility of it being NULL.

NULL values are incredibly difficult to manage, because to address them properly means doing constant checks against NULL in return values, data, and so on. They add inherent instability to any type system.

Tony Hoare invented the idea of a NULL reference. In 2009, he apologized for this, famously calling it his “billion-dollar mistake.”

In Kotlin, every type is non-nullable by default. This means that if you attempt to assign a NULL to a normal data type, the compiler is able to check against this and report it as a compile-time error. If you need to work with NULL data, you can declare a nullable variable using the ? annotation. Once you do this, you need to use specific ? methods. You may also need to take steps to handle NULL data when appropriate.

Conventions

• By default, a variable cannot be assigned a NULL value.
• ? suffix on the type indicates that it’s NULL-able.
• ?. accesses properties/methods if the object is not NULL (“safe call operator”)
• ?: elvis operator is a ternary operator for NULL data
• !! override operator (calls a method without checking for NULL, bad idea)

fun main() {
	// name is nullable
	var name:String? = null

	// only returns value if name is not null
	var length = name?.length
	println(length) // null

	// elvis operator provides an `else` value
	length = name?.length ?: 0
	println(length) // 0
}

click to run

Generics

Generics are extensions to the type system that allows us to parameterize classes or functions across different types. Generics expand the reusability of your class definitions because they allow your definitions to work with many types.

We’ve already seen generics when dealing with collections:

val list: List<Int> = listOf(5, 10, 15, 20)

In this example, <Int> is specifying the type that is being stored in the list. Kotlin infers types where it can, so we typically write this as:

val list = listOf(5, 10, 15, 20)

We can use a generic type parameter in the place of a specific type in many places, which allows us to write code towards a generic type instead of a specific type. This prevents us from writing methods that might only differ by parameter or return type.

A generic type is a class that accepts an input of any type in its constructor. For instance, we can create a Table class that can hold differing values.

You define the class and make it generic by specifying a generic type to use in that class, written in angle brackets < >. The convention is to use T as a placeholder for the actual type that will be used.

fun main() {
	class Table<T>(t: T) {
	    var value = t
	}

	val table1: Table<Int> = Table<Int>(5)
	val table2 = Table<Float>(3.14f)
}

click to run

A more complete example:

import java.util.*

class Timeline<T>() {
    val events : MutableMap<Date, T> = mutableMapOf()

    fun add(element: T) {
        events.put(Date(), element)
    }

    fun getLast(): T {
        return events.values.last()
    }
}

fun main() {
	val timeline = Timeline<Int>()
	timeline.add(5)
	timeline.add(10)
}

click to run

Control Flow

Kotlin supports the style of control flow that you would expect in an imperative language, but it uses more modern forms of these constructs

if then else

if... then has both a statement form (no return value) and an expression form (return value).

fun main() {
  val a=5
  val b=7

  // we don't return anything, so this is a statement
  println("a=$a, b=$b")
  if (a > b) {
      println("a is larger")
  } else {
      println("b is larger")
  }

  val number = 6

  // the value from each branch is considered a return value
  // this is an expression that returns a result
  println("number=$number")
  val result =
    if (number > 0)
      "$number is positive"
    else if (number < 0)
      "$number is negative"
    else
      "$number is zero"

  println(result)
}

// a=5, b=7
// b is larger
// number=6
// 6 is positive

click to run

This is why Kotlin does not have a ternary operator: if used as an expression serves the same purpose.

for in

A for in loop steps through any collection that provides an iterator. This is equivalent to the for each loop in languages like C#.

fun main() {
  val items = listOf("apple", "banana", "kiwifruit")
  for (item in items) {
  	println(item)
  }

  for (index in items.indices) {
  	println("item $index is ${items[index]}")
  }

  for (c in "Kotlin") {
    print("$c ")
  }
}

// apple
// banana
// kiwifruit
// item 0 is apple
// item 1 is banana
// item 2 is kiwifruit
// K o t l i n

click to run

Kotlin doesn’t support a C/Java style for loop. Instead, we use a range collection .. that generates a sequence of values.

fun main() {
  // invalid in Kotlin
  // for (int i=0; i < 10; ++i)

  // range provides the same funtionality
  for (i in 1..3) {
    print(i)
  }
  println() // space out our answers

  // descending through a range, with an optional step
  for (i in 6 downTo 0 step 2) {
    print("$i ")
  }
  println()

  // we can step through character ranges too
  for (c in 'A'..'E') {
    print("$c ")
  }
  println()

  // Check if a number is within range:
  val x = 10
  val y = 9
  if (x in 1..y+1) {
    println("fits in range")
  }
}

// 123
// 6 4 2 0
// A B C D E
// fits in range

click to run

while

while and do... while exist and use familiar syntax.

fun main() {
  var i = 1
  while ( i <= 10) {
    print("$i ")
    i++
  }
}

// 1 2 3 4 5 6 7 8 9 10

click to run

when

when replaces the switch operator of C-like languages:

fun main() {
  val x = 2
  when (x) {
    1 -> print("x == 1")
    2 -> print("x == 2")
    else -> print("x is neither 1 nor 2")
  }
}

// x == 2

click to run

fun main() {
    val x = 13
    val validNumbers = listOf(11,13,17,19)

    when (x) {
    	0, 1 -> print("x == 0 or x == 1")
    	in 2..10 -> print("x is in the range")
    	in validNumbers -> print("x is valid")
    	!in 10..20 -> print("x is outside the range")
    	else -> print("none of the above")
  }
}

// x is valid

click to run

We can also return a value from when. Here’s a modified version of this example:

fun main() {
    val x = 13
    val validNumbers = listOf(11,13,17,19)

    val response = when (x) {
        0, 1 -> "x == 0 or x == 1"
        in 2..10 -> "x is in the range"
        in validNumbers -> "x is valid"
        !in 10..20 -> "x is outside the range"
        else -> "none of the above"
    }
    println(response)
}

// x is valid

click to run

When is flexible. To evaluate any expression, you can move the comparison expressions into when statement itself:

fun main() {
    val x = 13

    val response = when {
        x < 0 -> "negative"
        x >= 0 && x <= 9 -> "small"
        x >=10 -> "large"
        else -> "how do we get here?"
    }
    println(response)
}

// large

click to run

return

Kotlin has three structural jump expressions:

• return by default returns from the nearest enclosing function or anonymous function
• break terminates the nearest enclosing loop
• continue proceeds to the next step of the nearest enclosing loop

Functions

Functions are preceded with the fun keyword. Function parameters require types, and are immutable. Return types should be supplied after the function name, but in some cases may also be inferred by the compiler.

Named Functions

Named functions has a name assigned to them that can be used to invoke them directly (this is the expected form of a “function” in most cases, and the form that you’re probably expecting).

// no parameters required
fun main() {
    println(sum1(1, 2))
    println(sum1(3,4))
}

// parameters which require type annotations
fun sum1(a: Int, b: Int): Int {
    return a + b
}

// return types can be inferred based on the value you return
// it's better form to explicitly include the return type in the signature
fun sum2(a: Int, b: Int) {
    a + b // Kotlin knows that (Int + Int) -> Int
}

// 3
// 7

click to run

Single-Expression Functions

Simple functions in Kotlin can sometimes be reduced to a single line aka a single-expression function.

// previous example
fun sumOf(a: Int, b: Int):Int {
    return a + b
}

// this works since we evaluate a single expression
fun minOf(a: Int, b: Int) = if (a < b) a else b

fun main() {
    println(sumOf(5,10))
    println(minOf(10,20))
}

// 15
// 10

click to run

Default arguments

We can use default arguments for function parameters. When called, a parameter with a default value is optional; if the caller does not provide the value, then the default will be used.

// Second parameter has a default value, so it’s optional
fun mult(a:Int, b:Int = 5): Int {
	return a * b
}

fun main() {
	println(mult(1)) // a=1, b=5 default
	println(mult(5,2)) // a=5, b=2
	// mult() would throw an error, since `a` must be provided
}

click to run

Named parameters

You can (optionally) provide the parameter names when you call a function. If you do this, you can even change the calling order!

fun repeat(str:String="*", count:Int=1):String {
    return str.repeat(count)
}

fun main() {
	println(repeat()) // *
	println(repeat(str="#")) // *
	println(repeat(count=3)) // ***
	println(repeat(str="#", count=5)) // #####
	println(repeat(count=5, str="#")) // #####
}

// *
// #
// ***
// #####
// #####

click to run

Variable-length arguments

Finally, we can have a variable length list of arguments:

fun sum(vararg numbers: Int): Int {
    var sum = 0
    for(number in numbers) {
        sum += number
    }
  return sum
}

fun main() {
    println(sum(1)) // 1
    println(sum(1,2,3)) // 6
    println(sum(1,2,3,4,5,6,7,8,9,10)) // 55
}

// 1
// 6
// 55

click to run

Collections

A collection is a finite group of some variable numbers of items (possibly zero) of the same type. Objects in a collection are called elements.

Collections in Kotlin are contained in the kotlin.collections package, which is part of the Kotlin Standard Library.

These collection classes exist as generic containers for a group of elements of the same type e.g. List<Int> would be an ordered list of integers. Collections have a finite size, and are eagerly evaluated.

Kotlin offers functional processing operations (e.g. filter, map and so on) on each of these collections.

fun main() {
  val list = (1..10).toList() // generate list of 1..10
  println( list.take(5).map{it * it} ) // square the first 5 elements
}

// [1, 4, 9, 16, 25]

click to run

Under-the-hood, Kotlin uses Java collection classes, but provides mutable and immutable interfaces to these classes. Kotlin best-practice is to use immutable for read-only collections whenever possible (since mutating collections is often very costly in performance).

{.compact}

A tuple is a data structure representing a sequence of n elements.

Pair

A Pair is a tuple of two values. Use var or val to indicate mutability. Theto keyword can be used to indicate a Pair.

fun main() {
  // mutable
  var nova_scotia = "Halifax Airport" to "YHZ"
  var newfoundland = Pair("Gander Airport", "YQX")
  var ontario = Pair("Toronto Pearson", "YYZ")
  ontario = Pair("Billy Bishop", "YTZ") // reassignment is ok

  // accessing elements
  val canadian_exchange = Pair("CDN", 1.38)
  println(canadian_exchange.first) // CDN
  println(canadian_exchange.second) // 1.38

  // destructuring
  val (first, second) = Pair("Calvin", "Hobbes") // split a Pair
  println(first) // Calvin
  println(second) // Hobbes
}

// CDN
// 1.38
// Calvin
// Hobbes

click to run

Pairs are extremely useful when working with data that is logically grouped into tuples, but where you don’t need the overhead of a custom class., e.g. Pair for 2D points.

List

A List is an ordered collection of objects.

fun main() {
  // define an immutable list
  var fruits = listOf( "advocado", "banana")
  println(fruits.get(0))
  // advocado

  // add elements
  var mfruits = mutableListOf( "advocado", "banana")
  mfruits.add("cantaloupe")
  mfruits.forEach { println(it) }

  // sorted/sortedBy returns ordered collection
  val list = listOf(2,3,1,4).sorted() // [1, 2, 3, 4]
  list.sortedBy { it % 2 } // [2, 4, 1, 3]

  // groupBy groups elements on collection by key
  list.groupBy { it % 2 } // Map: {1=[1, 3], 0=[2, 4]}

  // distinct/distinctBy returns unique elements
  listOf(1,1,2,2).distinct() // [1, 2]
}

// advocado
// advocado
// banana
// cantaloupe

click to run

Set

A Set is a generic unordered collection of unique elements (i.e. it does not support duplicates, unlike a List which does). Sets are commonly constructed with helper functions:

fun main() {
	val numbersSet = setOf("one", "two", "three", "four")
    println(numbersSet)

	val emptySet = mutableSetOf<String>()
    println(emptySet)
}

// [one, two, three, four]
// []

click to run

Map

A Map is an associative dictionary containing Pairs of keys and values.

fun main() {
  // immutable reference, immutable map
  val imap = mapOf(Pair(1, "a"), Pair(2, "b"), Pair(3, "c"))
  println(imap)
  // {1=a, 2=b, 3=c}

  // immutable reference, mutable map (so contents can change)
  val mmap = mutableMapOf(5 to "d", 6 to "e")
  mmap.put(7,"f")
  println(mmap)
  // {5=d, 6=e, 7=f}

  // lookup a value
  println(mmap.get(5))
  // d

  // iterate over key and value
  for ((k, v) in imap) {
    print("$k=$v ")
  }
  // 1=a 2=b 3=c

  // alternate syntax
  imap.forEach { k, v -> print("$k=$v ") }
  // 1=a 2=b 3=c

  // `it` represents an implicit iterator
  imap.forEach {
    print("${it.key}=${it.value} ")
  }
  // 1=a 2=b 3=c
}

// {1=a, 2=b, 3=c}
// {5=d, 6=e, 7=f}
// d
// 1=a 2=b 3=c 1=a 2=b 3=c 1=a 2=b 3=c

click to run

Functions

Collection classes (e.g. List, Set, Map, Array) have built-in functions for working with the data that they contain. These functions frequently accept other functions as parameters.

Filter

filter produces a new list of those elements that return true from a predicate function.

fun main() {
	val list = (1..100).toList()
	val filtered = list.filter { it % 5 == 0 }
    println(filtered)
	// 5 10 15 20 ... 100

	val below50 = filtered.filter { it in 0..49 }
    println(below50)
	// [5, 10, 15, 20]
}

// [5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100]
// [5, 10, 15, 20, 25, 30, 35, 40, 45]

click to run

Map

map produces a new list that is the results of applying a function to every element that it contains.

fun main() {
	val list = (1..100).toList()
	val doubled = list.map { it * 2 }
    println(doubled)
}

// 2 4 6 8 ... 200

click to run

Reduce

reduce accumulates values starting with the first element and applying an operation to each element from left to right.

fun main() {
	val strings = listOf("a", "b", "c", "d")
	println(strings.reduce { acc, string -> acc + string }) // abcd
}

// abcd

click to run

Zip

zip combines two collections, associating their respective pairwise elements.

fun main() {
	val foods = listOf("apple", "kiwi", "broccoli", "carrots")
	val fruit = listOf(true, true, false, false)
    println(fruit)

	val results = foods.zip(fruit)
	println(results)
}

// [true, true, false, false]
// [(apple, true), (kiwi, true), (broccoli, false), (carrots, false)]

click to run

A more realistic scenario might be where you want to generate a pair based on the results of the list elements:

fun main() {
	val list = listOf("123", "", "456", "def")
	val exists = list.zip(list.map { !it.isBlank() })
    println(exists)

	val numeric = list.zip(list.map { !it.isEmpty() && it[0] in ('0'..'9') })
    println(numeric)
}

// [(123, true), (, false), (456, true), (def, true)]
// [(123, true), (, false), (456, true), (def, false)]

click to run

ForEach

forEach calls a function for every element in the collection.

fun main() {
	val fruits = listOf("advocado", "banana", "cantaloupe" )
	fruits.forEach { print("$it ") }
}

// advocado banana cantaloupe

click to run

We also have helper functions to extract specific elements from a list.

Take

take returns a collection containing just the first n elements. drop returns a new collection with the first n elements removed.

fun main() {
	val list = (1..50)
	val first10 = list.take(10)
	println(first10)

	val last40 = list.drop(10)
	println(last40)
}

// [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
// [11, 12, 13, 14, 15, 16, 17, 18, 19, ...]

click to run

First, Last, Slice

first and last return those respective elements. slice allows us to extract a range of elements into a new collection.

fun main() {
	val list = (1..50)
	val even = list.filter { it % 2 == 0 }

	println(even.first()) // 2
	println(even.last()) // 50
	println(even.slice(1..3)) // 4 6 8
}

// 2
// 50
// [4, 6, 8]

click to run

Object-Oriented Kotlin

Object-oriented programming is a refinement of the structured programming model that was discovered in 1966 by Ole-Johan Dahl and Kristen Nygaard.

It is characterized by the use of classes as a template for common behaviour (methods) and data (state) required to model that behaviour.

• Abstraction: Model entities in the system to match real world objects. Each object exposes a stable high-level interface that can be used to access its data and behaviours. “Changing the implementation should not break anything.”
• Encapsulation: Keep state and implementation private.
• Inheritance: Specialization of classes. Create a specialized (child) class by deriving from another (parent) class, and reusing the parent’s fields and methods.
• Polymorphism: Base and derived classes share an interface, but have specialized implementations.

Object-oriented programming has benefits over iterative programming:

• It supports abstraction and allows us to express computation in a way that models our problem domain (e.g. customer classes, file classes).
• It handles complex state more effectively, by delegating to classes.
• It’s also a method of organizing your code. This is critical as programs grow.
• There is some suggestion that it makes code reuse easier (debatable).
• It became the dominant programming model in the 80s, and our most used languages are OO languages. e.g. C++, Java, Swift, Kotlin.

Classes

Kotlin is a class-based object-oriented language, with some advanced features that it shares with some other recent languages. The class keyword is used to define a Class. You create an instance of the class using the class name (no new keyword required!)

 // define class
 class Person

 // create two instances and assign to p, q
 // note that we have an implicit no-arg constructor
 val p = Person()
 val q = Person()

Classes include properties (values) and methods.

Properties

A property is a variable declared in a class, but outside methods or functions. They are analogous to class members, or fields in other languages.

class Person() {
   var firstName = "Vanilla"
   var lastName = "Ice"
}

fun main() {
	val p = Person()

    // we can access properties directly
    // this calls an implicit get() method; default returns the value
    println("${p.firstName} ${p.lastName} ${p.lastName} Baby")
}

click to run

Properties have implicit backing fields that store their data. We can override the get() and set methods to determine how our properties interact with the backing fields.

For example, for a City class, we can decide that we want the city name always reported in uppercase, and we want the population always stored as thousands.

 // the backing field is just referred to as `field`
 // in the set() method, we use `value` as the argument
 class City() {
   var name = ""
     get() = field.uppercase()
     set(value) {
       field = value
     }
   var population = 0
     set(value) {
       field = value/1_000
     }
 }

 fun main() {
     // create our city, using properties to access values
     val city = City()
     city.name = "Halifax"
     city.population = 431_000
     println("${city.name} has a population of ${city.population} thousand people")
 }

click to run

Behind-the-scenes, Kotlin is actually creating getter and setter methods, using the convention of getField and setField. In other words, you always have corresponding methods that are created for you. If you directly access the field name, these methods are actually getting called in the background.

Venkat Subramaniam has an excellent example of this (Subramaniam 2019). Write the class Car in a separate file named Car.kt:

 class Car(val yearOfMake: Int, var color: String)

Then compile the code and take a look at the bytecode using the javap tool, by running these commands:

 $ kotlinc-jvm Car.kt
 $ javap -p Car.class

This will display the bytecode generated by the Kotlin Compiler for the Car class:

 public final class Car {
   private final int yearOfMake;
   private java.lang.String color;
   public final int getYearOfMake();
   public final java.lang.String getColor();
   public final void setColor(java.lang.String);
   public Car(int, java.lang.String);
 }

That concise single line of Kotlin code for the Car class resulted in the creation of two fields—the backing fields for properties, a constructor, two getters, and a setter.

Constructors

Like other OO languages, Kotlin supports explicit constructors that are called when objects are created.

Primary Constructors

A primary constructor is the main constructor that your class will support (representing how you want it to be instantiated most of the time). You define it by expanding the class definition:

 // class definition includes the primary constructor
 class Person constructor() { }

 // we can collapse this to define an explicit no-arg constructor
 class Person() {}

In the example above, the primary constructor is called when this class is instantiated.

Optionally, you can include parameters in the primary constructor, and use these to initialize parameters in the constructor body.

 // constructor with arguments
 // this uses the parameters to initialize properties (i.e. variables)
 class Person (first:String, last:String) {
   val firstName = first.take(1).uppercase() + first.drop(1).lowercase()
   val lastName = last.take(1).uppercase() + last.drop(1).lowercase()

   // adding a statement like this will prevent the code from compiling
   // println("${firstname} ${lastname}")  // will not compile
 }

 fun main() {
	 // this does not work! we do not have a no-arg constructor
	 // val person = Person() // error since no matching constructor

	 // this works and demonstrates the properties
	 val person = Person("JEFF", "AVERY")
	 println("${person.firstName} ${person.lastName}") // Jeff Avery
 }

click to run

Constructors are designed to be minimal:

• Parameters can only be used to initialize properties. They go out of scope immediately after the constructor executes.
• You cannot invoke any other code in your constructor (there are other ways to handle that, which we will discuss below).

Secondary Constructors

What if you need more than a single constructor?

You can define secondary constructors in your class. Secondary constructors must delegate to the primary constructor. Let’s rewrite this class to have a primary no-arg constructor, and a second constructor with parameters.

// primary constructor
class Person() {
	// initialize properties
   	var firstName = "PAULA"
   	var lastName = "ABDUL"

   	// secondary constructor
   	// delegates to the no-arg constructor, which will be executed first
   	constructor(first: String, last: String) : this() {
   		// assign to the properties defined in the primary constructor
   		firstName = first.take(1).uppercase() + first.drop(1).lowercase()
   		lastName = last.take(1).uppercase() + last.drop(1).lowercase()
   	}
}

fun main() {
	val person1 = Person() // primary constructor using default property values
	println("${person1.firstName} ${person1.lastName}")

	val person2 = Person("JEFF", "AVERY") // secondary constructor
	println("${person2.firstName} ${person2.lastName}")
}

click to run

Init Blocks

How do we execute code in the constructor? We often want to do more than initialize properties.

Kotlin has a special method called init() that is used to manage initialization code. You can have one or more of these init blocks in your code, which will be called in order after the primary constructor (they’re actually considered part of the primary constructor). The order of initialization is (1) primary constructor, (2) init blocks in listed order, and then finally (3) secondary constructor.

class InitOrderDemo(name: String) {
   val first = "$name".also(::println)

   init {
     println("First init: ${first.length}")
   }

   val second = "$name".also(::println)
   init {
     println("Second init: ${second.length}")
     }
 }

fun main() {
    InitOrderDemo("Jeff")
}

Why does Kotlin split the constructor up like this? It’s a way to enforce that initialization MUST happen first, which results in cleaner and safer code.

Class Methods

Similarly to other programming languages, functions defined inside a class are called methods.

class Person(var firstName: String, var lastName: String) {
   fun greet() {
     println("Hello! My name is $firstName")
   }
 }

 fun main() {
 	val person = Person ("Jeff", "Avery")
 	println("${person.firstName} ${person.lastName}")
 }

Inheritance

To derive a class from a supertype, we use the colon : operator. We also need to delegate to the base class constructor using ().

By default, classes and methods are closed to inheritance. If you want to extend a class or method, you need to explicitly mark it as open for inheritance.

 class Base
 class Derived : Base() // error!

 open class Base
 class Derived : Base() // ok

Kotlin supports single-inheritance.

open class Person(val name: String) {
   open fun hello() = "Hello, I am $name"
 }

 class PolishPerson(name: String) : Person(name) {
   override fun hello() = "Dzien dobry, jestem $name"
 }

 fun main() {
     val p1 = Person("Jerry")
     val p2 = PolishPerson("Beth")
     println(p1.hello())
     println(p2.hello())
 }

If the derived class has a primary constructor, the base class can (and must) be initialized, using the parameters of the primary constructor. If the derived class has no primary constructor, then each secondary constructor has to initialize the base type using the super keyword, or to delegate to another constructor which does that.

 class MyView : View {
     constructor(ctx: Context) : super(ctx)
     constructor(ctx: Context, attrs: AttributeSet) : super(ctx, attrs)
 }

All classes in Kotlin have a common superclass Any, that is the default superclass for a class with no supertypes declared:

 class Example // Implicitly inherits from Any

Any has three methods: equals(), hashCode() and toString(). Thus, they are defined for all Kotlin classes.

Abstract Classes

Classes can be declared abstract, which means that they cannot be instantiated, only used as a supertype. The abstract class can contain a mix of implemented methods (which will be inherited by subclasses) and abstract methods, which do not have an implementation.

// useful way to represent a 2D point
 data class Point(val x:Int, val y:Int)

 abstract class Shape() {
   // we can have a single representation of position
   var x = 0
   var y = 0

   fun position(): Point {
     return Point(x, y)
   }
   // subtypes will have their own calculations for area
   abstract fun area():Int
 }

 class Rectangle (var width: Int, var height: Int): Shape() {
   constructor(x: Int, y: Int, width: Int, height: Int): this(width, height) {
     this.x = x
     this.y = y
   }
   // must be overridden since our base is abstract
   override fun area():Int {
     return width * height
   }
 }

 fun main() {
	// this won't compile, since Shape() is abstract
	// val shape = Shape()

	// this of course is fine
	val rect = Rectangle(10, 20, 50, 10)
	println("Rectangle at (${rect.position().x},${rect.position().y}) with area ${rect.area()}")
	// => Rectangle at (10,20) with area 500
 }

Interfaces

Interfaces in Kotlin are similar to abstract classes, in that they can contain a mix of abstract and implemented methods. What makes them different from abstract classes is that they cannot store state. They can have properties but these need to be abstract or to provide accessor implementations.

Data Classes

A data class is a special type of class, which primarily exists to hold data, and does not have custom methods. Classes like this are more common than you expect – we often create trivial classes to just hold data, and Kotlin makes them simple to create.

Why would you use a data class over a regular class? It generates a lot of useful methods for you:

• hashCode()
• equals() // compares fields
• toString()
• copy() // using fields
• destructuring

Here’s an example of how useful this can be:

data class Person(val name: String, var age: Int)

fun main() {
	val mike = Person("Mike", 23)

	// toString() displays all properties
	println(mike.toString())

	// structural equality (==) compares properties
	println(mike == Person("Mike", 23)) // True
	println(mike == Person("Mike", 21)) // False

	// referential equality (===) compares object references
	println(mike === Person("Mike", 23)) // False

	// hashCode based on primary constructor properties
	println(mike.hashCode() == Person("Mike", 23).hashCode()) // True
	println(mike.hashCode() == Person("Mike", 21).hashCode()) // False

	// destructuring based on properties
	val (name, age) = mike
	println("$name $age") // Mike 23

	// copy that returns a copy of the object
	// with concrete properties changed
	val jake = mike.copy(name = "Jake") // copy
}

Enum Classes

Enums in Kotlin are classes, so enum classes support type safety.

We can use them in expected ways. Enum num constants are separated with commas. We can also do interesting things with our enums, e.g. use them in when clauses (Example from Sommerhoff 2020).

 enum class Suits {
     HEARTS, SPADES, DIAMONDS, CLUBS
 }

fun main() {
 	val color = when(Suits.SPADES) {
   		Suits.HEARTS, Suits.DIAMONDS -> "red"
   		Suits.SPADES, Suits.CLUBS -> "black"
 	}
 	println(color)
}

Each enum constant is an object, and can be instantiated.

enum class Direction(val degrees: Double) {
	NORTH(0.0), SOUTH(180.0), WEST(270.0), EAST(90.0)
}

fun main() {
	val direction = Direction.EAST
	print(direction.degrees)
}

Visibility Modifiers

Classes, objects, interfaces, constructors, functions, properties and their setters can have visibility modifiers. Getters always have the same visibility as the property. Kotlin defaults to public access if no visibility modifier is provided.

The possible visibility modifiers are:

• public: visible to any other code.
• private: visible inside this class only (including all its members).
• protected: visible to any derived class, otherwise private.
• internal: visible within the same module, where a module is a set of Kotlin files compiled together.

Operator Overloading

Kotlin allows you to provide custom implementations for the predefined set of operators. These operators have predefined symbolic representation (like + or *) and precedence if you combine them.

Basically, you use the operator keyword to define a function, and provide a member function or an extension function with a specific name for the corresponding type. This type becomes the left-hand side type for binary operations and the argument type for the unary ones.

Here’s an example that extends a class named ClassName by overloading the + operator.

data class Point(val x: Double, val y: Double)

// -point
operator fun Point.unaryMinus() = Point(-x, -y)

// p1+p2
operator fun Point.plus(other: Point) = Point(this.x + other.x, this.y + other.y)

// p1*5
operator fun Point.times(scalar: Int) = Point(this.x * scalar, this.y * scalar)
operator fun Point.times(scalar: Double) = Point(this.x * scalar, this.y * scalar)

fun main() {
    val p1 = Point(5.0, 10.0)
	val p2 = Point(10.0, 12.0)

    println("p1=${p1}")
    println("p2=${p2}\n")
    println("-p1=${-p1}")
    println("p1+p2=${p1+p2}")
    print("p2*5=${p2*5}")
}

We can override any operators by using the keyword that corresponds to the symbol we want to override.

Note that this is the reference object on which we are calling the appropriate method. Parameters are available as usual.

Infix Functions

Functions marked with the infix keyword can also be called using the infix notation (omitting the dot and the parentheses for the call). Infix functions must meet the following requirements:

• They must be member functions or extension functions.
• They must have a single parameter.
• The parameter must not accept variable number of arguments and must have no default value.

For example, we can add a “shift left” function to the built-in Int class:

infix fun Int.shl(x: Int): Int {
	return (this shl x)
}

fun main() {
	// calling the function using the infix notation
	// shl 1 multiples an int by 2
 	println(212 shl 1)

 	// is the same as
 	println(212.shl(1))
}

Extension Functions

Kotlin supports extension functions: the ability to add functions to existing classes, even when you don’t have access to the original class’s source code, or cannot modify the class for some reason. This is also a great alternative to inheritance when you cannot extend a class.

For a simple example, imagine that you want to determine if an integer is even. The “traditional” way to handle this is to write a function:

fun isEven(n: Int): Boolean = n % 2 == 0

fun main() {
	println(isEven(4))
	println(isEven(5))
}

In Kotlin, the Int class already has a lot of built-in functionality. It would be a lot more consistent to add this as an extension function to that class.

fun Int.isEven() = this % 2 == 0

fun main() {
    println(4.isEven())
    println(5.isEven())
}

You can use extensions with your own types and also types you do not control, like List, String, and other types from the Kotlin standard library.

Extension functions are defined in the same way as other functions, with one major difference: When you specify an extension function, you also specify the type the extension adds functionality to, known as the receiver type. In our earlier example, Int.isEven(), we need to include the class that the function extends, or Int.

Note that in the extension body, this refers to the instance of the type (or the receiver for this method invocation).

fun String.addEnthusiasm(enthusiasmLevel: Int = 1) = this + "!".repeat(enthusiasmLevel)

fun main() {
    val s1 = "I'm so excited"
    val s2 = s1.addEnthusiasm(5)
    println(s2)
}

Defining an extension on a superclass

Extensions do not rely on inheritance, but they can be combined with inheritance to expand their scope. If you extend a superclass, all of its subclasses will inherit the extension method that you defined.

Define an extension on the Any class called print. Because it is defined on Any, it will be directly callable on all types.

// Any is the top-level class from which all classes derive i.e. the ultimate superclass.

fun Any.print() {
  println(this)
}

fun main() {
	"string".print()
	42.print()
}

Extension Properties

In addition to adding functionality to a type by specifying extension functions, you can also define extension properties.

For example, here is an extension property that counts a string’s vowels:

val String.numVowels
    get() = count { it.lowercase() in "aeiou" }

fun main() {
    println("abcd".numVowels)
}

Destructuring

Sometimes it is convenient to destructure an object into a number of variables. This syntax is called a destructuring declaration. A destructuring declaration creates multiple variables at once. In the example below, you declared two new variables: name and age, and can use them independently:

data class Person(val name: String, val age: Int)

fun main() {
  val p = Person("Janine", 38)
	val (name, age) = p  // destructuring
	println(name)
	println(age)
}

A destructuring declaration is compiled down to the following code:

 val name = person.component1()
 val age = person.component2()

component1(), component2() are aliases to the named properties in this class, in the order they were declared (and, of course, there can be component3() and component4() and so on). You would never normally refer to them using these aliases.

Here’s an example from the Kotlin documentation on how to use this to return multiple values from a function:

 // data class with properties `result` and `status`
 data class Result(val result: Int, val status: Status)

 fun function(...): Result {
     // computations
     return Result(result, status)
 }

 // Destructure into result and status
 val (result, status) = function(...)

 // We can also choose to not assign fields
 // e.g. we could just return `result` and discard `status`
 val (result, _) = function(...)

Companion Objects

OO languages typically have some idea of static members: methods that are associated with a class instead of an instance of a class. Static methods can be useful when attempting to implement the singleton pattern, for instance.

Kotlin doesn’t support static members directly. To get something comparable in Kotlin, you need to declare a companion object as an inner class of an existing class. Any methods that are created as part of the companion object are considered to be static methods in the enclosing class.

The examples below are taken from: https://livevideo.manning.com/module/59_5_16/kotlin-for-android-and-java-developers

class House(val numberOfRooms: Int, val price: Double) {
   companion object {
     val HOUSES_FOR_SALE = 10
     fun getNormalHouse() = House(6, 599_000.00)
     fun getLuxuryHouse() = House(42, 7_000_000.00)
   }
 }

 fun main() {
   val normalHouse = House.getNormalHouse()  // works
   println(normalHouse.price)
   println(House.HOUSES_FOR_SALE)
 }

We can also use object types to implement singletons. All we need to do is use the object keyword.

 class Country(val name:String) {
     var area = 0.0
 }

 // there can be only one
 object CountryFactory {
     fun createCountry() = Country("Canada")
 }

 fun main() {
     val obj = CountryFactory.createCountry()
     println(obj.name)
 }
``

Functional Kotlin

Functional programming is a programming style where programs are constructed by compositing functions together. Functional programming treats functions as first-class citizens: they can be assigned to a variable, passed as parameters, or returned from a function.

Functional programming also specifically avoids mutation: functions transform inputs to produce outputs, with no internal state. Functional programming can be described as declarative (describe what you want) instead of imperative (describe how to accomplish a result).

Kotlin is considered a hybrid language: it provides mechanisms for you to write in a functional style, but it also doesn’t prevent you from doing non-functional things. As a developer, it’s up to you to determine the most appropriate approach to a given problem.

Here are some common properties that we talk about when referring to “functional programming”:

• First-class functions means that functions are treated as first-class citizens. We can pass them as to another function as a parameter, return functions from other functions, and even assign functions to variables. This allows us to treat functions much as we would treat any other variable.

• Pure functions are functions that have no side effects. More formally, the return values of a pure function are identical for identical arguments (i.e. they don’t depend on any external state). Also, by having no side effects, they do not cause any changes to the system, outside their return value. Functional programming attempts to reduce program state, unlike other programming paradigms (imperative or object-oriented) which are based on careful control of program state.

• Immutable data means that we do not modify data in-place. We prefer immutable data that cannot be accidentally changed, especially as a side effect of a function. Instead, if we need to mutate data, we pass it to a function that will return a new data structure containing the modified data, leaving the original data intact. This avoids unintended state changes.

• Lazy evaluation is the notion that we only evaluate an expression when we need to operate on it (and we only evaluate what we need to evaluate at the moment!) This allows us to express and manipulate some expressions that would be extremely difficult to actually represent in other paradigms.

Function Types

Functions in Kotlin are “first-class citizens” of the language. This means that we can define functions, assign them to variables, pass functions as arguments to other functions, or return functions! Functions are types in Kotlin, and we can use them anywhere we would expect to use a regular type.

Dave Leeds on Kotlin presents the following excellent example: Bert’s Barber shop is creating a program to calculate the cost of a haircut, and they end up with 2 almost-identical functions.

fun main() {
    val taxMultiplier = 1.10

    fun calculateTotalWithFiveDollarDiscount(initialPrice: Double): Double {
      val	priceAfterDiscount = initialPrice - 5.0
      val total = priceAfterDiscount * taxMultiplier
      return total
    }

    fun calculateTotalWithTenPercentDiscount(initialPrice: Double): Double {
      val priceAfterDiscount = initialPrice * 0.9
      val total = priceAfterDiscount * taxMultiplier
      return total
    }
}

These functions are identical except for the line that calculates priceAfterDiscount. If we could somehow pass in that line of code as an argument, then we could replace both with a single function that looks like this, where applyDiscount() represents the code that we would dynamically replace:

// applyDiscount = initialPrice * 0.9, or
// applyDiscount = initialPrice - 5.0
fun calculateTotal(initialPrice: Double, applyDiscount: ???): Double {
    val priceAfterDiscount = applyDiscount(initialPrice)
    val total = priceAfterDiscount * taxMultiplier
    return total
}

This is a perfect scenario for passing in a function!

Assign a function to a variable

fun discountFiveDollars(price: Double): Double = price - 5.0
val applyDiscount = ::discountFiveDollars

In this example, applyDiscount is now a reference to the discountFiveDollars function (note the :: notation when we have a function on the RHS of an assignment). We can even call it.

val discountedPrice = applyDiscount(20.0) // Result is 15.0

So what is the type of our function? The type of function is the function signature, but with a different syntax that you might be accustomed to seeing.

// this is the original function signature
fun discountFiveDollars(price: Double): Double = price - 5.0
val applyDiscount = ::discountFiveDollars

// applyDiscount accepts a Double as an argument and returns a Double
// we use this format when specifying the type
val applyDiscount: (Double) -> Double

For functions with multiple parameters, separate them with a comma.

We can use this notation when explicitly specifying type.

fun discountFiveDollars(price: Double): Double = price - 5.0

// specifying type is not necessary since type inference works too
// we'll just do it here to demonstrate how it would appear
val applyDiscount : (Double) -> Double = ::discountFiveDollars

Pass a function to a function

We can use this information to modify the earlier example, and have Bert’s calculation function passed into the second function.

fun discountFiveDollars(price: Double): Double = price - 5.0
fun discountTenPercent(price: Double): Double = price * 0.9
fun noDiscount(price: Double): Double = price

fun calculateTotal(initialPrice: Double, applyDiscount: (Double) -> Double): Double {
    val priceAfterDiscount = applyDiscount(initialPrice)
    val total = priceAfterDiscount * taxMultiplier
    return total
}

val withFiveDollarsOff = calculateTotal(20.0, ::discountFiveDollars) // $16.35
val withTenPercentOff  = calculateTotal(20.0, ::discountTenPercent)  // $19.62
val fullPrice          = calculateTotal(20.0, ::noDiscount)          // $21.80

Returning Functions from Functions

Instead of typing in the name of the function each time he calls calculateTotal(), Bert would like to just enter the coupon code from the bottom of the coupon that he receives from the customer. To do this, he just needs a function that accepts the coupon code and returns the right discount function.

fun discountForCouponCode(couponCode: String): (Double) -> Double = when (couponCode) {
    "FIVE_BUCKS" -> ::discountFiveDollars
    "TAKE_10"    -> ::discountTenPercent
    else         -> ::noDiscount
}

I’ve taken liberties with Dave Leed’s example, but my notes can’t do it justice. I’d highly recommend a read through his site - he’s building an outstanding Kotlin book chapter-by-chapter with cartoons and illustrations.

Lambdas

We can use this same notation to express the idea of a function literal, or a function as a value.

val applyDiscount: (Double) -> Double = { price: Double -> price - 5.0 }

The code on the RHS of this expression is a function literal, which captures the body of this function. We also call this a lambda. A lambda is just a function, but written in this form:

• the function is enclosed in curly braces { }
• the parameters are listed, followed by an arrow
• the body comes after the arrow

What makes a lambda different from a traditional function is that it doesn’t have a name. In the expression above, we assigned the lambda to a variable, which we could them use to reference it, but the function itself isn’t named.

Note that due to type inference, we could rewrite this example without the type specified on the LHS. This is the same thing!

val applyDiscount = { price: Double -> price - 5.0 }

The implicit ‘it’ parameter

In cases where there’s only a single parameter for a lambda, you can omit the parameter name and the arrow. When you do this, Kotlin will automatically make the name of the parameter it.

val applyDiscount: (Double) -> Double = { it - 5.0 }

Lambdas and Higher-Order Functions

Passing Lambdas as Arguments

Higher-order functions have a function as an input or output. We can rewrite our earlier example to use lambdas instead of function references:

// fun discountFiveDollars(price: Double): Double = price - 5.0
// fun discountTenPercent(price: Double): Double = price * 0.9
// fun noDiscount(price: Double): Double = price

fun calculateTotal(initialPrice: Double, applyDiscount: (Double) -> Double): Double {
    val priceAfterDiscount = applyDiscount(initialPrice)
    val total = priceAfterDiscount * taxMultiplier
    return total
}

val withFiveDollarsOff = calculateTotal(20.0, { price - 5.0 }) // $16.35
val withTenPercentOff  = calculateTotal(20.0, { price * 0.9 }) // $19.62
val fullPrice          = calculateTotal(20.0, { price })       // $21.80

In cases where function’s last parameter is a function type, you can move the lambda argument outside of the parentheses to the right, like this:

val withFiveDollarsOff = calculateTotal(20.0) { price -> price - 5.0 }
val withTenPercentOff  = calculateTotal(20.0) { price -> price * 0.9 }
val fullPrice          = calculateTotal(20.0) { price -> price }

This is meant to be read as two arguments: one inside the brackets, and the lambda as the second parameter.

Returning Lambdas as Function Results

We can easily modify our earlier function to return a lambda as well.

fun discountForCouponCode(couponCode: String): (Double) -> Double = when (couponCode) {
    "FIVE_BUCKS" -> { price -> price - 5.0 }
    "TAKE_10"    -> { price -> price * 0.9 }
    else         -> { price -> price }
}

Scope Functions