Abstract
Locating reliable sources of generalizable longitudinal data is an extremely important issue for business research. The aim of this paper was to empirically verify that crowdsourcing can be used to source longitudinal samples. Specifically, three studies assess reliability of the Amazon Mechanical Turk Marketplace (MTurk). All three studies demonstrate that MTurk is a reliable, inexpensive source for generalizable longitudinal data. Study 1 (n = 752) examines the two-month re-response rate (study 1, n = 752; 75%) of a US MTurk sample. Study 2 (n = 373) investigates the four- and eight-month re-response rate (56 and 38%, respectively) of a US immigrant sample. Study 3 examines the thirteen-month re-response rate (47%). Each study demonstrates minimal non-response biases and longitudinal response consistency, in terms of both demographics and personality traits. This study also independently verifies the accuracy of self-report state of residence for 94% of the participants.
1. Swapping bricks for clicks: Crowdsourcing longitudinal data collection with Amazon Mechanical Turk
An opportunity for improving cross-sectional business research lies in the potential to further explore theories and issues with longitudinal research designs. Indeed, some theories and models inherently rely upon time-separated data from individuals. For example, brand loyalty and brand switching are vitally important to branding research, but are almost impossible to access without some type of temporally-separated design (e.g., Dawes, Meyer-Waarden, & Driesener, 2015). This type of research typically includes a true-panel design where the diagonal elements represent brand loyalty and the off-diagonal ones indicate extents of brand switching. Similarly, technology acceptance (e.g., Brown, Venkatesh, & Goyal, 2014; Venkatesh, Thong, & Xu, 2012), test–retest for scale development (see MacKenzie, Podsakoff, & Podsakoff, 2011), purchase intentionto-behavior relationships (e.g., Pavlou, Liang, & Xue, 2007), and pre- and post-communication campaign research (e.g., Johnston & Warkentin, 2010) are among other research topics that depend on multiple timepoints.
6.4. Conclusion
Longitudinal studies (true-panel designs in particular) in business research are strongly needed. Indeed, a deeper understanding of branding contexts, consumption over time, investment behaviors over time, organizational behaviors over time, and assessing temporal efficacy of programs would benefit from longitudinal research. Unfortunately, the commonly used captive student samples and commercial research panels (now mostly online) have major disadvantages; external validity concerns with the former and cost concerns with the latter. In contrast, MTurk can be used to access longitudinal data that is reliable, valid, consistent, and inexpensive without relying on student samples. The studies here provide preliminary but meaningful testimony to this generalization.
