Week 9 To-Do’s

Readings

Each student must individually select at least one starred item and at least one other item from this week’s reading list to read/watch. More details are available on the Information page under Deliverable 1c.

Plan your user evaluation

There are 5 common parts to this planning process for both prototypes:

1. Develop a test plan. For each prototype you created last week, what tasks you will you ask people to do? Why these tasks? IMPORTANT: you will likely need to refine and expand the prototypes you created last week to cover the tasks you want to test.
2. Create a script of instructions for the facilitator. Write down a list of questions that the facilitator may ask the user during an evaluation session. How long should the facilitator wait in silence without offering help, when a user is stuck at a particular step, unable to proceed?
3. Create a script of instructions for the participant. Write down the list of tasks they will be asked to perform and instructions to will explain what they will be doing during the evaluation session and why (e.g., think aloud, the fidelity of the prototype, etc.).
4. Prepare your ethics consent materials. Remember you absolutely must obtain consent from anyone participating in your evaluation if they are not a student in this term’s offering of the CS449/649 class.
5. Decide on the roles each team member will play during the evaluation: facilitator, wizard/technician, or observer. The technician handles the setup and trouble shooting of the interactive wireframe prototype and the wizard operates the paper prototype as the “human computer”. You need 3 team members, 1 person can’t perform all 3 roles during the evaluation.

For the paper prototype, there are 2 additional planning steps to create the wizard guide:

1. For the wizard, compile a list of user actions with appropriate “system” responses for each action. Make sure your paper prototype components are organized and labelled so the wizard can find them quickly.
2. Write down the procedure the wizard should follow to properly respond to unexpected input from a participant. For example, what should the wizard do if the participant does something unexpected?

Pilot your user evaluation

After planning the evaluation, run a mock “pilot” evaluation with 1 person from the class. Include all the steps you planned, even the step where you obtain consent. The recommended order is to test your paper prototype first, then the interactive wireframe prototype, but you can use the other order if it makes sense for the tasks in your prototypes.

Your design document should include:

A Protocol section:

1. Your goals, procedure, tasks to test, etc. Add justifications were appropriate.
2. The list of people who took on the role of facilitator, observer, and wizard/technician.
3. A link to your facilitator script .

Two Prototype subsections:

1. Descriptions with photos/screencaps showing any major changes to your prototypes since last week. A major change is a new screen, change to the navigation structure, significant addition to content on a screen, etc.
2. Include a shared link to the Figma file for the interactive wireframe prototype.
3. The list of wizard actions and wizard procedure for paper prototype.

Conduct your user evaluation

Based on the pilot evaluation feedback, run a full user evaluation with both prototypes with 2 people who are outside of this class (e.g., friends, acquaintances, extended family, etc.). Try your best to find target users (e.g., people who fit your personas) to perform the evaluation. Note the paper prototype really must be done in person, and in general in-person evaluations are better: they can be more convenient for you, and often much more informative and revealing since you can observe users’ facial expressions and body postures more closely. Re-read the notes about safety and consent for in-person sessions.

Special considerations due to COVID pandemic. If team members are feeling sick or test positive for COVID, please contact your TA: it’s important that you don’t unnecessarily expose people to COVID. Regardless of how you feel, you should wear a mask during the in-person evaluation and find a large room with good airflow (i.e. empty classroom on the weekend).

After the evaluations are completed, document the Key Findings in your design document for each participant:

1. Any points of confusion or hesitation from the user (with the user’s consent, include closeup photos/screenshots of the user performing the action on the prototype at these points of confusion/hesitation; the photos and screenshots should not have any identifying information).
2. Your questions to the user and their responses.
3. How you think your feature can be re-designed based on these findings.

Team meeting and design document

Conduct this week’s team meeting or discussion in and/or out of class. Update your design document with the meeting minutes and the outcomes of this week’s activities. More details are available on the Information page under Deliverable 2a.

Due next week

• Week 9 reading reflections (1c)
• Design document section #8 (2a)

---