Interaction types and object/operation analysis

Intuition

Once you know what the system is, you decide how people will act on it. The five interaction types are five verbs for that: tell it, talk to it, move things in it, wander through it, or let it nudge you.

Five interaction types describe what the user is doing

The lecture defines interaction types as the ways a person interacts with a product or application.

Instructing means telling the system what to do by typing commands, selecting menus, speaking commands, gesturing, pressing buttons or using function-key combinations. Conversing is a dialogue with the system as the partner, from simple menu-based dialogues to voice recognition, audio or text, for example Apple Siri, Amazon Alexa and virtual agents.

Manipulating relies on moving, selecting, dragging, opening and closing digital elements, with zooming and stretching as extensions. It draws on experience with real objects (affordances). Exploring is moving through virtual or physical environments, now popular with immersive VR. Responding is the system taking the initiative to alert, describe or show something relevant to the time or context.

Which type fits depends on the application domain and product, and different parts of one interaction can use different types.

Direct manipulation is the principle behind manipulating

Early guidance for GUIs was direct manipulation (DM), introduced by Ben Shneiderman. The lecture gives three core principles.

Exam detail

1. Continuous representation of the objects and actions of interest.
2. Rapid, reversible, incremental actions with immediate feedback about the object of interest.
3. Physical actions and button pressing instead of issuing commands with complex syntax.

Suitable uses: instructing for repetitive tasks; conversing for phone services, users with disabilities, children or elderly; manipulating for many applications; exploring for VR, AR and tele-existence; responding for contextual applications and recommendations.

Pitfall

Interface type is NOT interaction type. “The user is dragging a file” is an interaction type (manipulating). “A touchscreen GUI” is an interface type. Questions swap them to see whether you notice.

The conceptual model is expanded with functions, information and objects

The lecture then asks what functions the product performs and who does each (human or system), how functions relate (sequential or parallel, or grouped such as all privacy actions on a smartphone) and what information is needed and how the system transforms it. Weeks 6 and 7 worksheets ask you to do this for a project, including a task allocation of which functions are under system control and which under user control.

Object/operation analysis produces a list of the user-visible object types, the attributes of each, and the operations users perform on each. Objects are nouns from the scenario, operations are verbs, and interface furniture such as buttons, menus and buffers is excluded. The lecture’s example is a calendar app.

Try it: worked example

Scenario: A student adds a lecture to the calendar, sets a reminder, and deletes an old event. List the objects and operations.

Show answer and steps

AnswerObjects: calendar, event (attributes such as title, date, time) and reminder. Operations: add, set, delete.

1. Pick out the nouns: student, lecture, calendar, reminder, event. The student is the user, so the user-visible object types are the event (the lecture is a kind of event), the calendar and the reminder.
2. Give each object type attributes, for example event: title, date, start time, location.
3. Pick out the verbs: add, set, delete. These are the operations users perform on the objects.
4. Check nothing on the list is a button, menu or buffer. The calendar is a domain object, so it stays; a Save button would be excluded.

Recall

What are the three core principles of direct manipulation?

Continuous representation of objects and actions; rapid, reversible, incremental actions with immediate feedback; physical actions and button presses instead of complex command syntax.