Do you need real users in user studies, user experiments, and user surveys?

When you design user studies, user experiments, and user surveys (a.k.a. user research), there is a need to sample from the user population – that is selecting real users. Selecting real users is often considered vital for ensuring the validity, usefulness, and applicability of user research. But what are real users? Real users are participants … Read more

User Study Capabilities at the Qatar Computing Research Institute (QCRI)

We conducted a user study this week, 22-26 January 2023, collecting eye, mouse, verbal, and survey data — all within a single system!  In case a user study is something that you or others within your organization might be interested in, reach out!  If there are research implications, we are open to collaboration. Reach out! … Read more

Fair compensation of crowdsourcing work: the problem of flat rates

Compensating crowdworkers for their research participation often entails paying a flat rate to all participants, regardless of the amount of time they spend on the task or skill level. If the actual time required varies considerably between workers, flat rates may yield unfair compensation. To study this matter, we analyzed three survey studies with varying … Read more

Who is in the sample? An analysis of real and surrogate users as participants in user study research in the information technology fields

This research aims to determine how often user studies reported in peer-reviewed information technology literature sample real users or surrogate users as participants. Constructing a sample of real users as participants in user studies is considered by most researchers to be vital for the validity, usefulness, and applicability of research findings. However, how often user … Read more

Generating Cultural Personas From Social Data: A Perspective of Middle Eastern Users

We conduct a mixed-method study to better understand the content consumption patterns of Middle Eastern social media users and to explore new ways to present online data by using automatic persona generation. First, we analyze millions of content interactions on YouTube to dynamically generate personas describing behavioral patterns of different demographic groups. Second, we analyze … Read more

Who are your users?: comparing media professionals’ preconception of users to data-driven personas

One of the reasons for using personas is to align user understandings across project teams and sites. As part of a larger persona study, we conducted 16 qualitative interviews with media producers, the end users of persona descriptions. We asked the participants about their understanding of a typical AJE media consumer, and the variety of … Read more

Fixation and Confusion – Investigating Eye-tracking Participants’ Exposure to Information in Personas

To more effectively convey relevant information to end users of persona profiles, we conducted a user study consisting of 29 participants engaging with three persona layout treatments. We were interested in confusion engendered by the treatments on the participants, and conducted a within-subjects study in the actual work environment, using eye-tracking and talk-aloud data collection. … Read more

Findings of a User Study of Automatically Generated Personas

We report findings and implications from a seminaturalistic user study of a system for Automatic Persona Generation (APG) using large-scale audience data of an organization’s social media channels conducted at the workplace of a major international corporation. Thirteen participants from a range of positions within the company engaged with the system in a use case … Read more

Persona Perception Scale: Developing and Validating an Instrument for Human-Like Representations of Data

Personas are widely used in software development, system design, and HCI studies. Yet, their evaluation is difficult, and there are no recognized and validated measurement scales to date. To improve this condition, this research develops a persona perception scale based on reviewing relevant literature. We validate the scale through a pilot study with 19 participants, … Read more