Our experience and insights define our perspectives
on the evolving digital world.

We are constantly working to improve our understanding of users. Our user research methodologies include rapid ethnography, participatory design, prototyping, usability testing, diary studies and everything in between. Yet not one of them is used the same way today as it was five years ago. In order to better understand emergent user behavior, we must evolve traditional research methods and keep pace with the changing technological landscape.

Recently, Punchcut has been exploring new ways of leveraging the age-old diary study from a mobile user research context. In general, we run diary studies alongside other contextual research methods, including contextual inquiries, interviews and shadowing. Diary studies augment the grounded viewpoint of contextual methods with a more intimate look into participants’ attitudes towards technology. In rethinking the methods surrounding diary studies, we sought a way to simultaneously reveal contextual influences while becoming more tactical about eliciting how participants reported their behaviors.

The following five insights encapsulate our efforts to better capture contextual information in real-time. We believe these insights can provoke innovations and improvements on how we approach user research.

1 // Empower users to capture their life where it happens, as it happens.

Punchcut's mobile diary study tools
allow real-time entries in the field.
(Photo: Ed Yourdon)

Spontaneity counts. Users' daily lives are sequences of one in-the-moment activity after another. The most valuable data collection happens within this stream of activity. If it cannot be captured directly, researchers need to help participants provide feedback in a way that is quick and easy, at a time and place that is convenient for participants. For diary studies, opportunities to gather information are lost when users forget to make an entry or when the burden is too large. Mobile devices offer an ideal solution.

2 // Context is a source of implicit user data. Uncover it.

With the research tools available now, we can build more intelligence into how we collect data without intruding unnecessarily on participants' privacy. Methods such as careful randomization of prompts and variable reminder schedules that catch users at different times of the day can add richness to the data that users explicitly provide. Even looking at a participant's public social media output can add another dimension through which to analyze user feedback.

3 // Capture the richness of communication.

A voice-based diary study captures the richness of verbal communication. When participants speak rather than write their entries, their feedback takes the form of stories and full scenarios. People are conversational when talking in a way that gets lost in writing. We can hear the emotion and nuances in their speech, and can consider the depth of descriptions that are full and vivid. By using the appropriate tools and methods, it is easy to turn qualitative feedback into quantitative data. The way people tell their stories is as important as the content of their stories.

4 // Don't exclude participants unnecessarily.

The reality of timing and budgets forces many limitations on user research. In designing for consumer products, we are working with a user base that typically includes a wide variety of people. They differ in when, where and how they use their devices. Studies in the environment often exclude certain people simply by nature of the devices or contexts that are part of the study. By using finding creative ways to adjust our toolset, we can gain efficiency and help our studies become more representative. We continue to evolve our research methods to make it as easy as possible for users to supply feedback in natural and convenient situations using devices that they already have, know how to use, and have readily available. We believe it’s important not to exclude valuable participants just because they don’t have the right equipment or feel burdened by contributing.

5 // Innovation is affordable.

The barrier for improving user research methods is low. Significant improvements can be made easily because the tools for advancing much of our research are readily available at low cost. This lets us build custom tools quickly to match the specific needs of a particular study. Because of the reduced overhead in developing new tools, we are able to allocate more time to designing the study, spend more time with our participants, and analyze more thoroughly the data collected.

Punchcut's Mobile Diary Study Tool Our user research team was recently commissioned to discover the devices people use throughout the day and uncover how they were used. The team identified topics for investigation and built a mobile diary tool using the Twilio platform. While we cannot share the findings of that study, we can share the new methods we uncovered and some thoughts about the tool we created. Punchcut's mobile diary study tool prompts users to leave a diary entry wherever they are. Traditional diary studies come with some inherent drawbacks. (Photo: Eli Duke) A web interface allows researchers to create reports in real-time. // PUNCHCUT'S MOBILE DIARY TOOL The tool offered a a number of benefits and addressed a variety of pain points found in existing diary study methods. With the new tool: Participants can create diary entries at any time, from any place where they have their mobile phones, either by leaving a spoken message or sending a text message. A call results in multiple pieces of related data: a transcription of the voice message, an audio file, a timestamp, the participant ID, and the duration of the message. Results are collected in real time, allowing for quick iterations on the tool and research method. This is great for customizing to individual studies. Automated text messages are sent to participants as gentle reminders to prompt an entry as well as to encourage ongoing participation. // EASING THE BURDEN ON PARTICIPANTS Participants are traditionally required to carry a diary around that might not fit within their normal daily routine. But with Punchcut’s Mobile Diary tool, entries can be created using any phone, a device most people carry with them throughout the day. Participants find it difficult to remember to make entries. Studies that require scheduled entries suffer from gaps in activity because participants often forget or lose motivation. Using our mobile diary tool, we were able to push SMS reminders to them if they had not done so recently. We could schedule reminders intelligently so they did not become intrusive. Our team also found reminders to be a great opportunity to provide encouragement, helping participants maintain high levels of engagement. // ENABLING RESEARCHERS Distributing and then collecting physical diaries is cumbersome. When not all diaries get returned, valuable data is lost. With Punchcut’s mobile diary tool, distribution of study materials and delivery of diary entries are done digitally. Because everything is electronic, our team eliminated a costly data entry process. Intelligent tracking, tagging, and data processing are easy because everything is digital. Traditionally, data is collected all at once, at the end of the study. With our mobile tool, the entries come to the researchers in real-time. Research can start to get a sense of the data set immediately. This has the added benefit of providing an up-to-date picture of how engaged each participant is. Researchers cannot verify the integrity of manual entries in traditional diary studies. There is no way of knowing whether individual entries are created in the moment or if they are created all at once. But the data collected with our digital tool includes a timestamp to let the researcher know exactly when an entry is created. Collecting location-based information for each entry could also provide even richer context.

User research can enlighten design by illuminating behaviors, use cases and environments that are not always understood by the researcher or the participant. Punchcut is continually refining our research methods to meet the demands of streamlined projects and uncover emergent user behaviors. We strive to synthesize not only what users say, but what they do, and increasingly what their environment can reveal implicitly. Guided by established and perpetually evolving techniques, our user research provides companies with dynamic insight into the shifting contexts of mobility.

52

Responses / Topics: Mobile, Social Networking, Ubiquitous Computing, Observations