Uncovering Context With Mobile Diary Studies

Mobile user research cannot afford to be confined by physical space. People are on the move and research should meet users where they are.

Mobile user research can no longer afford to be confined by physical space and geographic boundaries. People are on the move. If we as researchers are to to understand their true behaviors, we need a robust toolset to meet them where they are and understand where they are going.

We are constantly working to improve our understanding of users. Our user research methodologies include rapid ethnography, participatory design, prototyping, usability testing, diary studies and everything in between. Yet not one of them is used the same way today as it was five years ago. In order to better understand emergent user behavior, we must evolve traditional research methods and keep pace with the changing technological landscape.

Recently, Punchcut has been exploring new ways of leveraging the age-old diary study from a mobile user research context. In general, we run diary studies alongside other contextual research methods, including contextual inquiries, interviews and shadowing. Diary studies augment the grounded viewpoint of contextual methods with a more intimate look into participants’ attitudes towards technology. In rethinking the methods surrounding diary studies, we sought a way to simultaneously reveal contextual influences while becoming more tactical about eliciting how participants reported their behaviors.

The following five insights encapsulate our efforts to better capture contextual information in real-time. We believe these insights can provoke innovations and improvements on how we approach user research.

Divider Line

One

Empower users to capture their life where it happens, as it happens.

Spontaneity counts. Users' daily lives are sequences of one in-the-moment activity after another. The most valuable data collection happens within this stream of activity. If it cannot be captured directly, researchers need to help participants provide feedback in a way that is quick and easy, at a time and place that is convenient for participants. For diary studies, opportunities to gather information are lost when users forget to make an entry or when the burden is too large. Mobile devices offer an ideal solution.

Punchcut's mobile diary study tools allow real-time entries in the field.

Divider Line

Two

Context is a source of implicit user data. Uncover it.

With the research tools available now, we can build more intelligence into how we collect data without intruding unnecessarily on participants' privacy. Methods such as careful randomization of prompts and variable reminder schedules that catch users at different times of the day can add richness to the data that users explicitly provide. Even looking at a participant's public social media output can add another dimension through which to analyze user feedback.

Divider Line

Three

Capture the richness of communication.

A voice-based diary study captures the richness of verbal communication. When participants speak rather than write their entries, their feedback takes the form of stories and full scenarios. People are conversational when talking in a way that gets lost in writing. We can hear the emotion and nuances in their speech, and can consider the depth of descriptions that are full and vivid. By using the appropriate tools and methods, it is easy to turn qualitative feedback into quantitative data. The way people tell their stories is as important as the content of their stories.

Divider Line

Four

Don't exclude participants unnecessarily.

The reality of timing and budgets forces many limitations on user research. In designing for consumer products, we are working with a user base that typically includes a wide variety of people. They differ in when, where and how they use their devices. Studies in the environment often exclude certain people simply by nature of the devices or contexts that are part of the study. By finding creative ways to adjust our toolset, we can gain efficiency and help our studies become more representative. We continue to evolve our research methods to make it as easy as possible for users to supply feedback in natural and convenient situations using devices that they already have, know how to use, and have readily available. We believe it’s important not to exclude valuable participants just because they don’t have the right equipment or feel burdened by contributing.

Divider Line

Five

Innovation is affordable.

The barrier for improving user research methods is low. Significant improvements can be made easily because the tools for advancing much of our research are readily available at low cost. This lets us build custom tools quickly to match the specific needs of a particular study. Because of the reduced overhead in developing new tools, we are able to allocate more time to designing the study, spend more time with our participants, and analyze more thoroughly the data collected.

Divider Line

Punchcut's Mobile Diary Study Tool

Our user research team was recently commissioned to discover the devices people use throughout the day and uncover how they were used.

The team identified topics for investigation and built a mobile diary tool using the Twilio platform. While we cannot share the findings of that study, we can share the new methods we uncovered and some thoughts about the tool we created.

The tool offered a a number of benefits and addressed a variety of pain points found in existing diary study methods. With the new tool:

  1. Participants can create diary entries at any time, from any place where they have their mobile phones, either by leaving a spoken message or sending a text message.
  2. A call results in multiple pieces of related data: a transcription of the voice message, an audio file, a timestamp, the participant ID, and the duration of the message.
  3. Results are collected in real time, allowing for quick iterations on the tool and research method. This is great for customizing to individual studies.
  4. Automated text messages are sent to participants as gentle reminders to prompt an entry as well as to encourage ongoing participation.

Easing The Burden On Participants

Participants are traditionally required to carry a diary around that might not fit within their normal daily routine. But with Punchcut’s Mobile Diary tool, entries can be created using any phone, a device most people carry with them throughout the day.

Participants find it difficult to remember to make entries. Studies that require scheduled entries suffer from gaps in activity because participants often forget or lose motivation. Using our mobile diary tool, we were able to push SMS reminders to them if they had not done so recently. We could schedule reminders intelligently so they did not become intrusive. Our team also found reminders to be a great opportunity to provide encouragement, helping participants maintain high levels of engagement.

Enabling Researchers

Distributing and then collecting physical diaries is cumbersome. When not all diaries get returned, valuable data is lost. With Punchcut’s mobile diary tool, distribution of study materials and delivery of diary entries are done digitally. Because everything is electronic, our team eliminated a costly data entry process. Intelligent tracking, tagging, and data processing are easy because everything is digital.

Traditionally, data is collected all at once, at the end of the study. With our mobile tool, the entries come to the researchers in real-time. Research can start to get a sense of the data set immediately. This has the added benefit of providing an up-to-date picture of how engaged each participant is.

Researchers cannot verify the integrity of manual entries in traditional diary studies. There is no way of knowing whether individual entries are created in the moment or if they are created all at once. But the data collected with our digital tool includes a timestamp to let the researcher know exactly when an entry is created. Collecting location-based information for each entry could also provide even richer context.

Punchcut's mobile diary study tool prompts users to leave a diary entry wherever they are.

Traditional diary studies come with some inherent drawbacks.

A web interface allows researchers to create reports in real-time.

Divider Line

User research can enlighten design by illuminating behaviors, use cases and environments that are not always understood by the researcher or the participant. Punchcut is continually refining our research methods to meet the demands of streamlined projects and uncover emergent user behaviors. We strive to synthesize not only what users say, but what they do, and increasingly what their environment can reveal implicitly. Guided by established and perpetually evolving techniques, our user research provides companies with dynamic insight into the shifting contexts of mobility.

Interested in
speaking with us?

give us a ring at +1 415.445.8855
or email us at business@punchcut.com

Punchcut is a human interface design company specializing in mobile, connected products and services.

© 2014 Punchcut LLC All rights reserved.

View photo attributions