Field studies and in-the-wild evaluation

Field studies watch technology in the settings where it is used

Field studies are done in natural settings. “In the wild” is a term for prototypes being used freely in natural settings. They seek to understand what users do naturally and how technology impacts them. In product design they are used to identify opportunities for new technology, determine design requirements, decide how best to introduce new technology, and evaluate technology in use.

They range from a few minutes to longitudinal studies of a few years. Data collection should be unobtrusive but informative: self-reports of problems when they occur, interval logging triggered by smartphone notifications, and logging software that monitors the frequency and patterns of daily activities.

Three conundrums come with studying people in their own lives

Informing participants: knowing they are studied makes people conscious of how they behave, yet without telling them, how do you get consent? Privacy: studies in homes are always intrusive, so participant and researcher agree which activities may or may not be recorded. The prototype: what to do if it breaks down, and what security is needed if deployed in public spaces.

Worked example from the lecture: does phone use drive emotion?

The lecture presents a field study asking whether smartphone usage drives emotions or the reverse. A standalone Android app tracked app usage, tracked facial expressions and logged emotions (Affectiva API), and collected self-reported emotions through questionnaires six times a day, running as a background service. The study lasted two weeks with 30 participants (15 female, 15 male). Independent variables: gender, day of the week, application use. Dependent variables: emotions (sadness, joy, fear, anger, surprise, disgust, valence) and self-reported valence and arousal. Self-reported valence correlated with the mean Affectiva valence (r = 0.3, p < 0.01). Using convergent cross mapping (Sugihara et al., 2012), for some participants particular apps caused certain emotions, while for others emotions drove app use.