{"id":82,"date":"2022-11-29T02:35:57","date_gmt":"2022-11-29T02:35:57","guid":{"rendered":"https:\/\/metric.qcri.org\/blog\/?p=82"},"modified":"2022-11-29T02:36:02","modified_gmt":"2022-11-29T02:36:02","slug":"fair-compensation-of-crowdsourcing-work-the-problem-of-flat-rates","status":"publish","type":"post","link":"https:\/\/metric.qcri.org\/blog\/2022\/11\/29\/fair-compensation-of-crowdsourcing-work-the-problem-of-flat-rates\/","title":{"rendered":"Fair compensation of crowdsourcing work: the problem of flat rates"},"content":{"rendered":"\n

Compensating crowdworkers for their research participation often entails paying a flat rate to all participants, regardless of the amount of time they spend on the task or skill level. If the actual time required varies considerably between workers, flat rates may yield unfair compensation.<\/p>\n\n\n\n

\"\"
Fair compensation of crowdsourcing work: the problem of flat rates<\/a><\/figcaption><\/figure>\n\n\n\n

To study this matter, we analyzed three survey studies with varying complexity. Based on the United Kingdom minimum wage and actual task completion times, we found that more than 3 in 4 (76.5%) of the crowdworkers studied were paid more than the intended hourly wage, and around one in four (23.5%) was paid less than the intended hourly wage when using a flat rate compensation model based on estimated completion time.<\/p>\n\n\n\n

The results indicate that the popular flat rate model falls short as a form of equitable remuneration, when perceiving fairness in the form of compensating one\u2019s time. Flat rate compensation would not be problematic if the workers\u2019 completion times were similar, but this is not the case in reality, as skills and
motivation can vary.<\/p>\n\n\n\n

To overcome this problem, the study proposes three alternative compensation models: Compensation by Normal Distribution, Multi-Objective Fairness, and Post-Hoc Bonuses.<\/p>\n\n\n\n

Salminen, J., Sayed Kamel, A.M., Jung, S.G., Mustak, M. and\u00a0Jansen, B. J.\u00a0(2022)\u00a0Fair compensation of crowdsourcing work: the problem of flat rates<\/a>,\u00a0Behaviour & Information Technology. DOI: 10.1080\/0144929X.2022.2150564<\/p>\n","protected":false},"excerpt":{"rendered":"

Compensating crowdworkers for their research participation often entails paying a flat rate to all participants, regardless of the amount of time they spend on the task or skill level. If the actual time required varies considerably between workers, flat rates may yield unfair compensation. To study this matter, we analyzed three survey studies with varying … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-82","post","type-post","status-publish","format-standard","hentry","category-user-study"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":168,"url":"https:\/\/metric.qcri.org\/blog\/2024\/03\/14\/some-insights-and-recommendations-about-user-studies\/","url_meta":{"origin":82,"position":0},"title":"Some insights and recommendations about user studies","date":"March 14, 2024","format":false,"excerpt":"We have published a series of research articles examining user studies, including implementation suggestions. Some insights and recommendations about user studies Some of our findings and recommendations are: Fair pay: Paying crowdworkers adequately is task dependent Real Users: Students are overused in academic studies; there are workable alternates to get\u2026","rel":"","context":"Similar post","img":{"alt_text":"Some insights and recommendations about user studies","src":"https:\/\/i0.wp.com\/metric.qcri.org\/blog\/wp-content\/uploads\/2024\/03\/insights_into_user_studies_jansen.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":75,"url":"https:\/\/metric.qcri.org\/blog\/2022\/09\/30\/who-is-in-the-sample-an-analysis-of-real-and-surrogate-users-as-participants-in-user-study-research-in-the-information-technology-fields\/","url_meta":{"origin":82,"position":1},"title":"Who is in the sample? An analysis of real and surrogate users as participants in user study research in the information technology fields","date":"September 30, 2022","format":false,"excerpt":"This research aims to determine how often user studies reported in peer-reviewed information technology literature sample real users or surrogate users as participants. Constructing a sample of real users as participants in user studies is considered by most researchers to be vital for the validity, usefulness, and applicability of research\u2026","rel":"","context":"In "user study"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":14,"url":"https:\/\/metric.qcri.org\/blog\/2022\/07\/28\/which-message-which-channel-which-customer-exploring-response-rates-in-multi-channel-marketing-using-short-form-advertising\/","url_meta":{"origin":82,"position":2},"title":"Which Message? Which Channel? Which Customer?: Exploring Response Rates in Multi-Channel Marketing Using Short Form Advertising","date":"July 28, 2022","format":false,"excerpt":"Formulating short form advertising messages with little ad content that work and choosing high-performing channels to disseminate them are persistent challenges in multichannel marketing. Drawing on the persuasive systems design model, we conducted an experiment with 33,848 actual customers of an international telecom company. In a real-life setting, we compared\u2026","rel":"","context":"In "user study"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":46,"url":"https:\/\/metric.qcri.org\/blog\/2022\/07\/28\/personas-and-analytics-a-comparative-user-study-of-efficiency-and-effectiveness-for-a-user-identification-task\/","url_meta":{"origin":82,"position":3},"title":"Personas and Analytics: A Comparative User Study of Efficiency and Effectiveness for a User Identification Task","date":"July 28, 2022","format":false,"excerpt":"Personas are a well-known technique in human computer interaction. However, there is a lack of rigorous empirical research evaluating personas relative to other methods. In this 34-participant experiment, we compare a persona system and an analytics system, both using identical user data, for efficiency and effectiveness for a user identification\u2026","rel":"","context":"In "user study"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":231,"url":"https:\/\/metric.qcri.org\/blog\/2024\/06\/25\/benefits-of-metric-in-teaching-user-study-analytics\/","url_meta":{"origin":82,"position":4},"title":"Benefits of METRIC in Teaching User Study Analytics","date":"June 25, 2024","format":false,"excerpt":"Analytics Illustration [Source: ACUA] Various industries benefit from user analytic studies, both with and without knowledge. E-Commerce, logistics and transportation, hospitality, web development, real estate, healthcare, public services, and even education. Many industries need user feedback to improve their services, so implementing user analytic studies in academia benefits students. Here\u2026","rel":"","context":"In "analytics"","img":{"alt_text":"Benefits of METRIC in Teaching User Study Analytics","src":"https:\/\/i0.wp.com\/metric.qcri.org\/blog\/wp-content\/uploads\/2024\/06\/Screen-Shot-2024-06-13-at-5.59.38-PM.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":36,"url":"https:\/\/metric.qcri.org\/blog\/2022\/07\/28\/comparing-persona-analytics-and-social-media-analytics-for-a-user-centric-task-using-eye-tracking-and-think-aloud\/","url_meta":{"origin":82,"position":5},"title":"Comparing Persona Analytics and Social Media Analytics for a User-Centric Task Using Eye-Tracking and Think-Aloud","date":"July 28, 2022","format":false,"excerpt":"We compare a data-driven persona system and an analytics system for efficiency and effectiveness for a user identification task. Findings from the 34-participant experiment show that the data-driven persona system affords faster task completion, is easier for users to engage with, and provides better user identification accuracy. Eye-tracking data indicates\u2026","rel":"","context":"In "user study"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/posts\/82","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/comments?post=82"}],"version-history":[{"count":1,"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/posts\/82\/revisions"}],"predecessor-version":[{"id":84,"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/posts\/82\/revisions\/84"}],"wp:attachment":[{"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/media?parent=82"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/categories?post=82"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/metric.qcri.org\/blog\/wp-json\/wp\/v2\/tags?post=82"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}