RESEARCH METHOD
05 Automated Remote Research
Automated remote research is a method that can reveal statistically relevant data about what people are doing on your website, to help identify the usability enhancements with the biggest impact.
Automated remote research enables design teams to leverage web-based research tools and services in order to collect statistically significant information about what people are doing on your website or web application.1 Once there is enough quantitative data about what users are doing, the research team can triangulate research findings with observed behavioral data to decide which usability enhancements to make to the site.
In automated remote research experiments, the research team’s focus shifts from recruiting and observing usability sessions with participants, to planning the appropriate strategy for the study, and then accurately selecting the right automated remote research tools and configuring the logistics of the study. Because there are an ever-growing number of web-based research tools available that are both quantitative and qualitative, it is important to invest some time to understand the automated research landscape.2
Many of these automated research tools can be used to further understand specific usability issues the research team is interested in, and to help you to collect quantitative data such as:3
• Are participants able to perform a certain task on the website?
• If so, how long does it take them to complete the task?
• If they have trouble completing the task, where are they abandoning the process?
• What is the most common click path that users take through the interface to complete a task?
If your organization puts an emphasis on the value of quantitative information, or if you have enough activity on your site that lends itself to a statistically significant sample, automated remote research may be a good fit for you.4 However, it should not be used as a replacement or alternative to more qualitative research methods that serve to provide rich insight into why site visitors behave the way that they do. It also should not be used interchangeably with remote moderated research, which provides qualitative behavior-based data. Even though both are deployed remotely, the resulting data from each method will be very different.5 It is important to understand these distinctions when selling remote methods internally, to appropriately set stakeholder expectations on research outcomes.
1. Bolt, Nate, and Tony Tulathimutte. Remote Research: Real Users, Real Time, Real Research. New York: Rosenfeld Media, 2010.
2. Bolt | Peters maintains an updated list of automated remote research products and services at http://www.remoteresear.ch
3. See note 1 above.
4. See note 1 above.
5. See note 1 above.
Further Reading
Tullis, Tom, and Bill Albert. Measuring the User Experience. San Francisco, CA: Morgan Kaufmann, 2008.
Tullis, Tom, Donna Tedesco, and William Albert. Beyond the Usability Lab: Conducting Large-Scale User Experience Studies. San Francisco, CA: Morgan Kaufmann, 2010.
In their book Remote Research, Nate Bolt and Tony Tulathimutte provide a framework for thinking about the different types of remote research tools and applications available to design teams. Automated remote methods are shown towards the bottom half of the diagram.
Courtesy of Nate Bolt, CEO, Bolt | Peters User Experience