Observation and diary studies

Observation shows what people do rather than what they say

Intuition

Ask someone how they use their phone on the train and you get a tidy answer. Watch them and you see the rest. Observation covers that gap, either live or through the records they leave.

Lecture 4 splits observation in two. Direct observation is watching people perform their activities. It needs time and commitment from a design team member and can produce a huge amount of data. Indirect observation uses records of the activity studied afterwards: diaries, interaction logging, and video and photographs collected remotely.

Compare

Direct techniques

Silent observation

Think aloud

Constructive interaction

What the observer does

The designer watches the user work. No communication.

The user is directed to say their thought process aloud.

Two people work on the task together and their conversation is observed.

Frameworks keep notes from becoming noise

The simple framework has three easy-to-remember parts: the person (who), the place (where) and the thing (what). The detailed framework from Robson (2014) has nine.

Mechanism

• Space: what is the physical space like and how is it laid out?
• Actors: names and relevant details of the people involved.
• Activities: what are the actors doing and why?
• Objects: physical objects present, such as furniture.
• Acts: specific individual actions.
• Events: is what you observe part of a special event?
• Time: the sequence of events.
• Goals: what the actors are trying to accomplish.
• Feelings: the mood of the group and of individuals.

Exam detail

Sample exam question 20 lists the practical considerations when planning an observation: observer presence may influence behaviour, overt versus covert observation, recording accurately and unobtrusively, and the duration and timing of sessions. “Only observing in laboratory settings” is the wrong option.

Pitfall

Acts and activities are different Robson items. Activities are what actors are doing and why. Acts are the specific individual actions. Goals are what they are trying to accomplish.

Logs and diaries record behaviour you cannot watch

Interaction logging records user activity in a log for later examination, such as time spent on a page and mouse clicks. It is unobtrusive but raises participant privacy issues if done without their knowledge.

Diaries suit usage patterns and private activity. Users log information at regular intervals or when an event happens, and the time recording is more accurate than a survey. There are two types: feedback diaries, where the data gives feedback to researchers, and elicitation diaries, where the data is used to gather more information about the event later.

To conduct a diary study: identify the purpose and participants, decide what to collect (text, video, voice or images, whichever is most natural for the diarists), choose the entry structure (fully structured with a set of questions each entry, or unstructured free-form), keep the burden low, recruit, and instruct participants on the goal, the activities of interest, when to record, the terms and the duration.

Recall

Name the two types of diary and one practical issue with interaction logging.

Feedback and elicitation diaries. Logging raises privacy issues if participants do not know about it.