Using METRIC to Evaluate User Interactions with Images

Evaluating user interaction with images is important for online marketing, advertising, news and content creation, and webpage design. METRIC (Measuring Engagement Through Remote Interactions of Customers) provides an advanced capability to do just this! As shown in Figure 1, you can set up Areas of Interest (AOIs) via a drag-and-drop method and give each AOI … Read more

Some insights and recommendations about user studies

We have published a series of research articles examining user studies, including implementation suggestions. Some of our findings and recommendations are: Read more in these articles! Salminen, J., Sayed Kamel, A.M., Jung, S.G., Mustak, M. and Jansen, B. J. (2022) Fair compensation of crowdsourcing work: the problem of flat rates. Behaviour & Information Technology, 27 … Read more

Do you need real users in user studies, user experiments, and user surveys?

When you design user studies, user experiments, and user surveys (a.k.a. user research), there is a need to sample from the user population – that is selecting real users. Selecting real users is often considered vital for ensuring the validity, usefulness, and applicability of user research. But what are real users? Real users are participants … Read more

User Study Capabilities at the Qatar Computing Research Institute (QCRI)

We conducted a user study this week, 22-26 January 2023, collecting eye, mouse, verbal, and survey data — all within a single system!  In case a user study is something that you or others within your organization might be interested in, reach out!  If there are research implications, we are open to collaboration. Reach out! … Read more

Fair compensation of crowdsourcing work: the problem of flat rates

Compensating crowdworkers for their research participation often entails paying a flat rate to all participants, regardless of the amount of time they spend on the task or skill level. If the actual time required varies considerably between workers, flat rates may yield unfair compensation. To study this matter, we analyzed three survey studies with varying … Read more

Who is in the sample? An analysis of real and surrogate users as participants in user study research in the information technology fields

This research aims to determine how often user studies reported in peer-reviewed information technology literature sample real users or surrogate users as participants. Constructing a sample of real users as participants in user studies is considered by most researchers to be vital for the validity, usefulness, and applicability of research findings. However, how often user … Read more

Generating Cultural Personas From Social Data: A Perspective of Middle Eastern Users

We conduct a mixed-method study to better understand the content consumption patterns of Middle Eastern social media users and to explore new ways to present online data by using automatic persona generation. First, we analyze millions of content interactions on YouTube to dynamically generate personas describing behavioral patterns of different demographic groups. Second, we analyze … Read more

Who are your users?: comparing media professionals’ preconception of users to data-driven personas

One of the reasons for using personas is to align user understandings across project teams and sites. As part of a larger persona study, we conducted 16 qualitative interviews with media producers, the end users of persona descriptions. We asked the participants about their understanding of a typical AJE media consumer, and the variety of … Read more

Fixation and Confusion – Investigating Eye-tracking Participants’ Exposure to Information in Personas

To more effectively convey relevant information to end users of persona profiles, we conducted a user study consisting of 29 participants engaging with three persona layout treatments. We were interested in confusion engendered by the treatments on the participants, and conducted a within-subjects study in the actual work environment, using eye-tracking and talk-aloud data collection. … Read more