Staff

Students taking this course are formally enrolled in one of four course numbers:

• CS 451: this is the version for computer science undergraduates. Most students will be enrolled in this version.
• CS 651: this is the version for computer science graduate students and MDSAI graduate students. Students enrolled in this version complete all CS 451 assignments plus an additional final project.
• CS 431: this is the version for undergraduates who are not majoring in computer science.
• CS 631: This course has been discontinued.

We use the same website for all courses, so make sure you're doing the right assignments!

The instructor for this course is Dan Holtby.

If you have general questions, post on Piazza. If you have personal concerns, send us a private post on Piazza (private posts are only visible by me and TAs).

Course Materials

The most recent version of all materials for this course will be posted on this website, including the syllabus, readings, slides, and assignments. Please check the site frequently for updates.

CS451/651 students, see the "software" page for instructions on getting access to MapReduce and Spark. Throughout this course, you'll also be using Bespin, which is a software library that contains reference implementations of "big data" algorithms in MapReduce and Spark.

The textbooks for this course are:

• Data-Intensive Text Processing with MapReduce by Jimmy Lin and Chris Dyer
• Hadoop: The Definitive Guide (4th Edition) by Tom White [Book website]
• Learning Spark by Holden Karau, Andy Konwinski, Patrick Wendell, Matei Zaharia
• Spark: The Definitive Guide by Matei Zaharia, Bill Chambers

Note that the three O'Reilly books are optional but recommended. They are mostly there to help you with the mechanics of MapReduce and Spark (API, command-line invocations, etc.). However, the same material can be found from a multitude of freely-available sources on the web (although not organized as well).

Readings from other sources such as research papers will be assigned as appropriate.

Grading

Components of the final grade are as follows:

For Undergraduate Students (CS451/CS431): For Graduate Students (CS651):

In order to pass the course, students must pass the final exam.

The homework assignments are to be completed individually. See below section on Academic Integrity.

Late submission policy: Assignment deadlines are firm (but see "accommodations" below). Late assignments will recieve a grade of 0.

Assignment marking reappraisal requests: If you believe we have made an error marking your assignment, you may request that your assignment be reappraised. Please send a private post on Piazza with subject "Remark request-A?-userid", for example, "Remark request-A4-a2abedi". In your post, clearly describe the reason for the remark request. Note that for each request, the entire assignment will be reconsidered, in addition to the highlighted issues. This means that your grade might be adjusted up or down—the latter case if we found an error that was missed the first time.

Late enrollment policy: Students who enroll late into the course must follow the lectures and hand in all assignments (an extension will be granted if necessary based on time of enrollment, but not an exemption!)

Academic Integrity

All work in this course is to be done individually unless otherwise noted (e.g., the group final project).

It is not permitted to use Generative AI to generate code for your solutions. This includes but is not limited to CoPilot and ChatGPT.

Incidents of suspected plagiarism or unauthorized collaboration will be reported to the office of the Associate Dean for Undergraduate Studies. The standard penalty for a first offence is a grade of zero for the assessment in question and a 5-mark deduction from the course grade. Note that in cases in which one individual provides assistance to another, the same penalty is normally applied to each. Subsequent offences are likely to result in more severe penalties, possibly including suspension or even expulsion.

To avoid inadvertently plagiarizing, you should discuss assignment issues with other students only in a very broad and high-level fashion. Do not take notes during such discussions and do not looking at anyone else's assignment code, on screen or on paper. If you find yourself stuck, contact the TAs or the instructor for help. You are allowed to search the web for information about general issues, but do not try to search for solutions online (or ask for assignment help online on mailing lists or in forums such as StackOverflow, etc., or from generative AI like ChatGPT). The assignments are designed so that solutions are not available online, but if you inadvertently stumble onto a solution to any of the assignments, do not look at it. If you do find a solution online, however, please let us know—we will appreciate it and not construe it as plagiarism (unless, of course, you actually do copy the solution).

Assignment Screening

Measure of Software Similarities (MOSS) is used in this course as a means of comparing students' assignments to ensure academic integrity. We will report suspicious activity, and penalties for plagiarism/cheating are severe. Please read the available information about academic integrity very carefully. Discipline cases involving any automated marking system such as Marmoset or MarkUs include, but are not limited to, printing or returning values in order to match expected test results rather than making an actual reasonable attempt to solve the problem as required in the assignment question specification.

Senate Undergraduate Council has asked us to post the following paragraphs:

Academic Integrity: In order to maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility.

Grievance: A student who believes that a decision affecting some aspect of his/her university life has been unfair or unreasonable may have grounds for initiating a grievance. Read Policy 70 - Student Petitions and Grievances, Section 4.

Discipline: A student is expected to know what constitutes academic integrity, to avoid committing academic offenses, and to take responsibility for his/her actions. A student who is unsure whether an action constitutes an offense, or who needs help in learning how to avoid offenses (e.g., plagiarism, cheating) or about "rules" for group work/collaboration should seek guidance from the course professor, academic advisor, or the Undergraduate Associate Dean. When misconduct has been found to have occurred, disciplinary penalties will be imposed under Policy 71 - Student Discipline. For information on categories of offenses and types of penalties, students should refer to that policy.

Avoiding Academic Offenses: Most students are unaware of the line between acceptable and unacceptable academic behaviour, especially when discussing assignments with classmates and using the work of other students. For information on commonly misunderstood academic offenses and how to avoid them, students should refer to the Faculty of Mathematics Academic Integrity Policy.

Appeals: A student may appeal the finding and/or penalty in a decision made under Policy 70 - Student Petitions and Grievances (other than regarding a petition) or Policy 71 - Student Discipline if a ground for an appeal can be established. Read Policy 72 - Student Appeals.

Accommodations

Illness policy: From time to time students become ill or have ongoing medical conditions that prevent them from meeting academic obligations. For these cases, please consult the university policy.

Short Term Absence Policy: The university allows students to delcare a short term absence (see the registrar's note here). If you use this then you will get a 48-hour extension for the assignment due that week. You must contact me after declaring a STA.

Flex Days: Students in this course all have a bank of 5 "flex" days. You may use a flex day to get an extra 24 hours added to the deadline of one assignment. You can use all of your flex days on a single assignment, or spread them between assignments. These are "no questions asked" but you must tell me you're using them using the Assignment Extensions Page.

Accommodations for Religious Holidays and Other Special Circumstances. Students wishing to discuss accommodations for religious holidays on dates that assignments are due, or other circumstances not addressed in this course information page, should discuss those circumstances with me before the third class session in order to permit adequate time for planning. Only accommodations for unforeseeable circumstances will be considered after that date.

Note for students with disabilities: AccessAbility Services, located in Needles Hall, collaborates with all academic departments to arrange appropriate accommodations for students with disabilities without compromising the academic integrity of the curriculum. If you require academic accommodations to lessen the impact of your disability, please register with them at the beginning of each academic term.