1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Organizational Psychology and Organizational Behavior
  4. Volume 4, 2017
  5. Article

Annual Review of Organizational Psychology and Organizational Behavior

Volume 4, 2017

Field Experiments in Organizations

Abstract

Field experimentation, although rare, is the sterling-gold standard of organizational research methods. It yields the best internally valid and generalizable findings compared to more fallible methods. Reviewers in many psychology specialties, including organizational psychology, synthesize largely nonexperimental research, warn of causal ambiguity, and call for experimental replication. These calls go mostly unheeded. Practical application is a raison d'être for much organizational research. With the emergence of evidence-based management, field experiments enable us to deliver the most actionable tools to practitioners. This review explicates the role of experimental control and randomization and enumerates some of the factors that mitigate field experimentation. It describes, instantiates, and evaluates true field experiments, quasi-experiments, quasi-fields, combo designs, and triangulation. It also provides practical tips for overcoming deterrents to field experimentation. The review ends describing the merging of new technologies with classical experimental design and prophesying the bright future of organizational field experimentation.

Keyword(s): experimental control, field experiment, little data, quasi-fields, quasification, randomization
Loading

Article metrics loading...

/content/journals/10.1146/annurev-orgpsych-041015-062400
2017-03-21
2024-04-20
Loading full text...

Full text loading...

/deliver/fulltext/orgpsych/4/1/annurev-orgpsych-041015-062400.html?itemId=/content/journals/10.1146/annurev-orgpsych-041015-062400&mimeType=html&fmt=ahah

Literature Cited

  1. Adair JG. 1984. The Hawthorne effect: a reconsideration of the methodological artifact. J. Appl. Psychol. 69:334–45 [Google Scholar]
  2. Agerström J, Rooth D. 2011. The role of automatic obesity stereotypes in real hiring discrimination. J. Appl. Psychol. 96:790–805 [Google Scholar]
  3. Aguinis H, Bradley KJ. 2014. Best-practice recommendations for designing and implementing experimental vignette methodology studies. Organ. Res. Methods 17:351–71 [Google Scholar]
  4. Aguinis H, Lawal SO. 2012. Conducting field experiments using eLancing's natural environment. J. Bus. Venturing 27:493–505 [Google Scholar]
  5. Aguinis H, Mazurkiewicz MD, Heggestad ED. 2009. Using web-based frame-of-reference training to decrease biases in personality-based job analysis: an experimental field study. Pers. Psychol. 62:405–38 [Google Scholar]
  6. Aguinis H, Molina-Azorín JF. 2015. Using multilevel modeling and mixed methods to make theoretical progress in microfoundations for strategy research. Strateg. Org. 13:353–64 [Google Scholar]
  7. Aguinis H, Pierce CA, Bosco FA, Muslin IS. 2009. First decade of Organizational Research Methods: trends in design, measurement, and data-analysis topics. Organ. Res. Methods 12:69–112 [Google Scholar]
  8. Aguinis H, Vandenberg RJ. 2014. An ounce of prevention is worth a pound of cure: improving research quality before data collection. Annu. Rev. Organ. Psychol. Organ. Behav. 1:569–95 [Google Scholar]
  9. Alderfer CP. 1968. Comparison of questionnaire responses with and without preceding interviews. J. Appl. Psychol. 52:335–40 [Google Scholar]
  10. Allen TD, Rush MC. 1998. The effects of organizational citizenship—field study and a laboratory experiment. J. Appl. Psychol. 83:247–60 [Google Scholar]
  11. Anderson CA, Lindsay JJ, Bushman BJ. 1999. Research in the psychological laboratory: truth or triviality?. Curr. Dir. Psychol. Sci. 8:3–9 [Google Scholar]
  12. Antonakis J, d'Adda G, Weber R, Zehnder C. 2015. “Just words? Just speeches?” On the economic value of charismatic leadership Work. Pap., Dep. Organ. Behavior, Univ. Lausanne
  13. Antonakis J, Fenley M, Liechti S. 2011. Can charisma be taught? Tests of two interventions. Acad. Manag. Learn. Educ. 10:374–96 [Google Scholar]
  14. Aquino K, Tripp TM, Bies RJ. 2006. Getting even or moving on? Power, procedural justice, and types of offense as predictors of revenge, forgiveness, reconciliation, and avoidance in organizations. J. Appl. Psychol. 91:653–68 [Google Scholar]
  15. Ariely D. 2008. What's the value of a big bonus?. New York TimesNov. 19 A43. http://www.nytimes.com/2008/11/20/opinion/20ariely.html?_r=0
  16. Aronson E, Carlsmith JM. 1968. Experimentation and social psychology. The Handbook of Social Psychology 21–79 G Lindzey, E Aronson. Reading, MA: Addison-Wesley, 2nd ed.. [Google Scholar]
  17. Austin J, Scherbaum C, Mahlman. 2002. History of research methods in industrial organizational psychology: measurement, design, analysis. Handbook of Research Methods in Industrial and Organizational Psychology SG Rogelberg 77–98 Malden, MA: Blackwell [Google Scholar]
  18. Austin PC. 2011. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav. Res. 46:399–424 doi:10.1080/00273171.2011.568786 [Google Scholar]
  19. Avolio BJ, Sosik JJ, Kahai S, Baker B. 2014. E-leadership: Re-examining transformations in leadership source and transmission. Leadersh. Q. 25:105–31 [Google Scholar]
  20. Azar OH, Yosef S, Bar-Eli M. 2013. Do customers return excessive change in a restaurant? A field experiment on dishonesty. J. Econ. Behav. Organ. 93:219–26 [Google Scholar]
  21. Babad EY, Inbar J, Rosenthal R. 1982. Pygmalion, Galatea, and the Golem: investigations of biased and unbiased teachers. J. Educ. Psychol. 74:459–74 [Google Scholar]
  22. Bandiera O, Barankay I, Rasul I. 2011. Field experiments with firms. J. Econ. Perspect. 25:363–82 [Google Scholar]
  23. Bandura A. 1997. Self-Efficacy: The Exercise of Control New York: Freeman
  24. Bareket-Bojmel L, Hochman G, Ariely D. 2014. It's (not) all about the Jacksons: testing different types of short-term bonuses in the field. J. Manag. https://doi.org/10.1177/0149206314535441
  25. Barling J, Kelloway EK, Weber T. 1996. Effects of transformational leadership training on attitudinal and financial outcomes: a field experiment. J. Appl. Psychol. 81:827–32 [Google Scholar]
  26. Bass BM, Bass R. 2008. The Bass Handbook of Leadership: Theory, Research, and Managerial Applications New York: The Free Press
  27. Bass BM, Cascio WF, McPherson JW, Tragash HJ. 1976. PROSPER—training and research for increasing management awareness of affirmative action in race relations. Acad. Manag. J. 19:353–69 [Google Scholar]
  28. Beal DJ, Weiss HM. 2003. Methods of ecological momentary assessment in organizational research. Organ. Res. Methods 6:440–64 [Google Scholar]
  29. Beardsworth A, Bryman A. 1994. Late modernity and the dynamics of quasification: the case of the themed restaurant. Sociol. Rev.228–57
  30. Becker LJ. 1978. Joint effect of feedback and goal setting on performance: a field study of residential energy conservation. J. Appl. Psychol. 63:428–33 [Google Scholar]
  31. Becker TE, Atinc G, Breaugh JA, Carlson KD, Edwards JR, Spector PE. 2015. Statistical control in correlational studies: 10 essential recommendations for organizational researchers. J. Organ. Behav. 37:157–67 [Google Scholar]
  32. Beer M, Cannon MD. 2004. Promise and peril in implementing pay-for-performance. Hum. Resour. Manag. 43:3–20 [Google Scholar]
  33. Bellé N. 2015. Performance‐related pay and the crowding out of motivation in the public sector: a randomized field experiment. Public Adm. Rev. 75:230–41 [Google Scholar]
  34. Bernerth JB, Aguinis H. 2016. A critical review and best-practice recommendations for control variable usage. Pers. Psychol. 69:229–83 [Google Scholar]
  35. Bertrand M, Djankov S, Hanna R, Mullainathan S. 2007. Obtaining a driver's license in India: an experimental approach to studying corruption. Q. J. Econ. 122:1639–76 [Google Scholar]
  36. Borgman CL. 2015. Big Data, Little Data, No Data: Scholarship in the Networked World Cambridge, MA: MIT Press.
  37. Boruch RF, Wothke W. 1985. Seven kinds of randomization plans for designing field experiments. Randomization and Field Experimentation RF Boruch, W Wothke 95–118 San Francisco: Jossey-Bass [Google Scholar]
  38. Brown TC, Latham GP. 2006. The effect of training in verbal self-guidance on performance effectiveness in a MBA program. Can. J. Behav. Sci. 38:11–11 [Google Scholar]
  39. Bruning NS, Frew DR. 1987. Effects of exercise, relaxation, and management skills training on physiological stress indicators: a field experiment. J. Appl. Psychol. 72:515–21 [Google Scholar]
  40. Buhrmester M, Kwang T, Gosling SD. 2011. Amazon's Mechanical Turk: a new source of inexpensive, yet high-quality, data. Perspect. Psychol. Sci. 6:3–5 [Google Scholar]
  41. Campbell DT, Stanley JC. 1966. Experimental and Quasi-Experimental Designs for Research Chicago: Rand McNally
  42. Caplan RD, Vinokur AD, Price RH, van Ryn M. 1989. Job seeking, reemployment, and mental health: a randomized field experiment in coping with job loss. J. Appl. Psychol. 74:759–69 [Google Scholar]
  43. Carlson KD, Wu J. 2012. The illusion of statistical control: control variable practice in management research. Organ. Res. Methods 15:413–35 [Google Scholar]
  44. Carroll GR, Hannan MT. 2000. The Demography of Corporations and Industries Princeton, NJ: Princeton Univ. Press
  45. Cascio WF, Aguinis H. 2008. Research in industrial and organizational psychology from 1963 to 2007. J. Appl. Psychol. 93:1062–81 [Google Scholar]
  46. Caza A, McCarter MW, Northcraft GB. 2015. Performance benefits of reward choice: a procedural justice perspective. Hum. Resour. Manag. J. 25:184–99 [Google Scholar]
  47. Chandler D, Kapelner A. 2013. Breaking monotony with meaning: motivation in crowdsourcing markets. J. Econ. Behav. Organ. 90:123–33 [Google Scholar]
  48. Chen S, Westman M, Eden D. 2009. Impact of enhanced resources on anticipatory stress and adjustment to new information technology: a field-experimental test of Conservation of Resources Theory. J. Occup. Health Psychol. 14:219–348 [Google Scholar]
  49. Chen Y, Brockner J, Greenberg J. 2003. When is it “A pleasure to do business with you”? The effects of relative status, outcome favorability, and procedural fairness. Organ. Behav. Hum. Decis. Processes 92:1–21 [Google Scholar]
  50. Cialdini RB. 2009. We have to break up. Perspect. Psychol. Sci. 4:5–6 [Google Scholar]
  51. Coch L, French JRP Jr.. 1948. Overcoming resistance to change. Hum. Relat. 1:512–32 [Google Scholar]
  52. Coman A, Hirst W. 2015. Social identity and socially shared retrieval-induced forgetting: the effects of group membership. J. Exp. Psychol.: Gen. 144:717–22 [Google Scholar]
  53. Cook TD, Campbell DT. 1979. Quasi-Experimentation: Design and Analysis Issues for Field Settings Chicago: Rand McNally
  54. Creswell JW. 2003. Research Design: Quantitative, Qualitative, and Mixed Methods Approaches Thousand Oaks, CA: Sage, 2nd ed..
  55. Cruz-Cunha MM. 2012. Handbook of Research on Serious Games as Educational, Business and Research Tools Hershey, PA: IGI Global
  56. D'Agostino RB, Rubin DB. 2000. Estimating and using propensity scores with partially missing data. J. Am. Stat. Assoc. 95:749–59 [Google Scholar]
  57. Davenport S, Daellenbach U. 2011. “Belonging” to a virtual research centre: exploring the influence of social capital formation processes on member identification in a virtual organization. Br. J. Manag. 22:54–76 [Google Scholar]
  58. Davidson OB, Eden D. 2000. Remedial self-fulfilling prophecy: two field experiments to prevent Golem effects among disadvantaged women. J. Appl. Psychol. 85:386–98 [Google Scholar]
  59. de Leeuw JR. 2015. jsPsych: a JavaScript library for creating behavioral experiments in a Web browser. Behav. Res. Methods 47:1–12 [Google Scholar]
  60. De Paola M, Gioia F, Scoppa V. 2014. Overconfidence, omens and gender heterogeneity: results from a field experiment. J. Econ. Psychol. 45:237–52 [Google Scholar]
  61. Deci EL, Koestner R, Ryan RM. 1999. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol. Bull. 125:627–68 [Google Scholar]
  62. Dewar AJ, Kavussanu M, Ring C. 2013. The effects of achievement goals on emotions and performance in a competitive agility task. Sport Exercise Perform. Psychol. 2:4250–64 [Google Scholar]
  63. Di Stefano G, King AA, Verona G. 2014. Kitchen confidential? Norms for the use of transferred knowledge in gourmet cuisine. Strateg. Manag. J. 35:1645–70 [Google Scholar]
  64. Dipboye RL. 1990. Laboratory versus field research in industrial-organizational psychology. International Review of Industrial and Organizational Psychology 5 CL Cooper, IT Robertson 1–34 New York: Wiley [Google Scholar]
  65. Ditlmann R, Paluck EL. 2015. Field experiments. International Encyclopedia of the Social and Behavioral Sciences 9 JD Wright 128–34 Oxford: Elsevier, 2nd ed.. [Google Scholar]
  66. Dvir T, Eden D, Avolio B, Shamir B. 2002. Impact of transformational leadership on follower development and performance: a field experiment. Acad. Manag. J. 45:735–44 [Google Scholar]
  67. Earley PC, Lind EA. 1987. Procedural justice and participation in task selection: the role of control in mediating justice judgments. J. Pers. Soc. Psychol. 52:1148–60 [Google Scholar]
  68. Eden D. 1985. Team development: a true field experiment employing three levels of rigor. J. Appl. Psychol. 70:94–100 [Google Scholar]
  69. Eden D. 1986. Team development: quasi-experimental confirmation among combat companies. Group Organ. Stud. 11:133–46 [Google Scholar]
  70. Eden D, Aviram A. 1993. Self-efficacy training to speed reemployment: helping people to help themselves. J. Appl. Psychol. 78:352–60 [Google Scholar]
  71. Eden D, Geller D, Gewirtz A, Gordon-Terner R, Inbar I. et al. 2000. Implanting Pygmalion leadership style through workshop training: seven field experiments. Leadersh. Q. 11:171–210 [Google Scholar]
  72. Eden D, Moriah L. 1996. Impact of internal auditing on branch bank performance: a field experiment. Organ. Behav. Hum. Decis. Process. 68:262–71 [Google Scholar]
  73. Eden D, Stone-Romero EF, Rothstein H. 2015. Synthesizing the results of multiple randomized experiments to establish causality in mediation testing. Hum. Resour. Manag. Rev.
  74. Edwards JR. 2008. To prosper, organizational psychology should...overcome methodological barriers to progress. J. Organ. Behav. 29:469–91 [Google Scholar]
  75. Ein-Gar D, Steinhart Y. 2011. The “Sprinter effect”: when self-control and involvement stand in the way of sequential performance. J. Consum. Psychol. 21:240–55 [Google Scholar]
  76. Fan J, Lai L. 2014. Pre-training perceived social self-efficacy accentuates the effects of a cross-cultural coping orientation program: evidence from a longitudinal field experiment. J. Organ. Behav. 35:831–50 [Google Scholar]
  77. Fan J, Wanous JP. 2008. Organizational and cultural entry: a new type of orientation program for multiple boundary crossings. J. Appl. Psychol. 93:1390–400 [Google Scholar]
  78. Fisher GG, Matthews RA, Gibbons AM. 2016. Developing and investigating the use of single-item measures in organizational research. J. Occup. Health Psychol. 21:3–23 [Google Scholar]
  79. Flores C, Bui D, Lilienthal L, Myerson J, Hale S. 2014. Age-related differences in simple and complex spans in online samples. Presented at Annu. Meet. Psychonom. Soc. 55th, Long Beach, CA, Nov. 20–23
  80. French JRP Jr, Israel J, Ås D. 1960. An experiment on participation in a Norwegian factory. Hum. Relat. 13:3–20 [Google Scholar]
  81. Frese M. 2009. Towards a psychology of entrepreneurship: an action theory perspective. Found. Trends Entrep. 5:435–94 [Google Scholar]
  82. Frese M, Bausch A, Schmidt P, Rauch A, Kabst R. 2012. Evidence-based entrepreneurship (EBE): cumulative science, action principles, and bridging the gap between science and practice. Found. Trends Entrep. 8:1–62 [Google Scholar]
  83. Friedman S, Ronen S. 2015. The effect of implementation intentions on transfer of training. Eur. J. Soc. Psychol. 45:409–16 [Google Scholar]
  84. Ganzach Y, Pazy A, Ohayun Y, Brainin E. 2002. Social exchange and organizational commitment: decision-making training for job choice as an alternative to the realistic job preview. Pers. Psychol. 55:613–37 [Google Scholar]
  85. Giessner SR, van Knippenberg D, Sleebos E. 2009. License to fail? How leader group prototypicality moderates the effects of leader performance on perceptions of leadership effectiveness. Leadersh. Q. 20:434–51 [Google Scholar]
  86. Gilliland SW, Groth M, Baker RC, Dew AE, Polly LM, Langdon JC. 2001. Improving applicants' reactions to rejection letters: an application of fairness theory. Pers. Psychol. 54:669–703 [Google Scholar]
  87. Glaser M, Langer T, Weber M. 2013. True overconfidence in interval estimates: evidence based on a new measure of miscalibration. J. Behav. Decis. Making 26:405–17 [Google Scholar]
  88. Glaub M, Frese M, Fischer S, Hoppe M. 2014. Increasing personal initiative in small business managers or owners leads to entrepreneurial success: a theory-based controlled randomized field intervention for evidence-based management. Manag. Learn. Educ. 13:354–79 [Google Scholar]
  89. Gordijn EH, Stapel DA. 2008. When controversial leaders with charisma are effective: the influence of terror on the need for vision and impact of mixed attitudinal messages. Eur. J. Soc. Psychol. 38:389–411 [Google Scholar]
  90. Graen GB, Scandura TA, Graen MR. 1986. A field experimental test of the moderating effects of growth need strength on productivity. J. Appl. Psychol. 71:484–91 [Google Scholar]
  91. Grant AM. 2008. The significance of task significance: job performance effects, relational mechanisms, and boundary conditions. J. Appl. Psychol. 93:108–24 [Google Scholar]
  92. Grant AM, Rothbard NP. 2013. When in doubt, seize the day? Security values, prosocial values, and proactivity under ambiguity. J. Appl. Psychol. 98:810–19 [Google Scholar]
  93. Grant AM, Wall TD. 2009. The neglected science and art of quasi-experimentation: why-to, when-to, and how-to advice for organizational researchers. Organ. Res. Methods 12:653–86 [Google Scholar]
  94. Gray D. 1983. A job club for older job seekers: an experimental evaluation. J. Gerontol. 38:363–68 [Google Scholar]
  95. Greenberg J. 1988. Equity and workplace status: a field experiment. J. Appl. Psychol. 73:606–13 [Google Scholar]
  96. Greenberg J. 1990. Employee theft as a reaction to underpayment inequity: the hidden cost of pay cuts. J. Appl. Psychol. 75:561–68 [Google Scholar]
  97. Greenberg J. 1993. Stealing in the name of justice: informational and interpersonal moderators of theft reactions to underpayment inequity. Organ. Behav. Hum. Decis. Process. 54:81–103 [Google Scholar]
  98. Greenberg J, Tomlinson EC. 2004. Situated experiments in organizations: transplanting the lab to the field. J. Manag. 30:703–24 [Google Scholar]
  99. Giluk TL, Rynes SL. 2012. Research findings practitioners resist: lessons for management academics from evidence-based medicine. See Rousseau 2012 130–64
  100. Goodman PS. 1970. The natural controlled experiment in organizational research. Hum. Organ. 29:197–203 [Google Scholar]
  101. Hägg I, Johanson J. 1975. The business game as a research tool: experiments on the effects of received information on the decision process: II. Swedish J. Econ. 77:351–58 [Google Scholar]
  102. Harrison DA, McLaughlin ME. 1993. Cognitive processes in self-report responses: tests of item context effects in work attitude measures. J. Appl. Psychol. 78:129–40 [Google Scholar]
  103. Harrison GW, List JA. 2004. Field experiments. J. Econ. Lit. 42:1009–55 [Google Scholar]
  104. Hebl MR, King EB, Glick P, Singletary SL, Kazama S. 2007. Hostile and benevolent reactions toward pregnant women: complementary interpersonal punishments and rewards that maintain traditional roles. J. Appl. Psychol. 92:1499–511 [Google Scholar]
  105. Hezkiau-Ludwig R, Eden D. 2011. What to do when a field experiment goes awry? Presented at Annu. Conf. Soc. Ind. Org. Psychology, 26th, Chicago [Google Scholar]
  106. Highhouse S. 2009. Designing experiments that generalize. Organ. Res. Methods 12:554–66 [Google Scholar]
  107. Ho DE, Rubin DB. 2011. Credible causal inference for empirical legal studies. Annu. Rev. Law Soc. Sci. 7:17–40 [Google Scholar]
  108. Jackson SE, Hall NC, Rowe PM, Daniels LM. 2009. Getting the job: attributional retraining and the employment interview. J. Appl. Soc. Psychol. 39:973–98 [Google Scholar]
  109. Jick TD. 1979. Mixing qualitative and quantitative methods: triangulation in action. Adm. Sci. Q. 24:602–11 [Google Scholar]
  110. Jones DA, Willness CR, Madey S. 2014. Why are job seekers attracted by corporate social performance? Experimental and field tests of three signal-based mechanisms. Acad. Manag. J. 57:383–404 [Google Scholar]
  111. Kahneman D. 2011. Thinking, Fast and Slow New York: Farrar, Straus & Giroux
  112. Kenny DA. 1979. Correlation and Causality New York: Wiley-Intersci. Revis, ed..
  113. King EB, Hebl MR, Morgan WB, Ahmad AS. 2012. Field experiments on sensitive organizational topics. Organ. Res. Methods 16:501–21 [Google Scholar]
  114. Kluger AN, DeNisi A. 1996. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 119:254–84 [Google Scholar]
  115. Kraut AI. 2013. Theory, schmeory. Let's keep our models simple and strong. Ind. Organ. Psychol. 6:188–90 [Google Scholar]
  116. Lam SSK, Schaubroeck J. 2000. A field experiment testing frontline opinion leaders as change agents. J. Appl. Psychol. 85:987–95 [Google Scholar]
  117. Landers RN, Behrend TS. 2015. An inconvenient truth: arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples. Ind. Organ. Psychol.: Perspect. Sci. Practice 8:142–64 [Google Scholar]
  118. Lawler EE III. 1977. Adaptive experiments: an approach to organizational behavior research. Acad. Manag. Rev. 2:576–85 [Google Scholar]
  119. Lawler EE III, Hackman JR. 1969. Impact of employee participation in the development of pay incentive plans: a field experiment. J. Appl. Psychol. 53:467–71 [Google Scholar]
  120. Lawler EE, Hackman JR, Kaufman S. 1973. Effects of job redesign: a field experiment. J. Appl. Soc. Psychol. 3:49–62 [Google Scholar]
  121. Lee JJ, Gino F, Staats BR. 2014. Rainmakers: why bad weather means good productivity. J. Appl. Psychol. 99:504–13 [Google Scholar]
  122. Lee TW. 1999. Using Qualitative Methods in Organizational Research Thousand Oaks, CA: Sage
  123. Levitt SD, List JA. 2009. Field experiments in economics: the past, the present, and the future. Eur. Econ. Rev. 53:1–18 [Google Scholar]
  124. Lewin K, Lippitt R. 1938. An experimental approach to the study of autocracy and democracy: a preliminary note. Sociometry 1:292–300 [Google Scholar]
  125. Lewin K, Lippitt R, White RK. 1939. Patterns of aggressive behavior in experimentally created “social climates.”. J. Soc. Psychol. 10:271–99 [Google Scholar]
  126. Liang LH, Adair WL, Hideg I. 2014. When should we disagree? The effect of relationship conflict on team identity in East Asian and North American teams. Negotiation Confl. Manag. Res. 7:a282–89 [Google Scholar]
  127. Light RJ, Singer JD, Willett JB. 1990. By Design: Planning Research in Higher Education Cambridge, MA: Harvard Univ. Press
  128. Lipsey MW, Cordray DS. 2000. Evaluation methods for social intervention. Annu. Rev. Psychol. 51:345–75 [Google Scholar]
  129. Locke EA. 1986. Generalizing from laboratory to field settings: research findings from industrial-organizational psychology, organizational behavior, and human resource management. Lexington, MA: Lexington Books
  130. Locke EA, Latham GP. 1990. A Theory of Goal Setting and Task Performance Englewood Cliffs, NJ: Prentice-Hall
  131. Long EC, Christian MS. 2015. Mindfulness buffers retaliatory responses to injustice: a regulatory approach. J. Appl. Psychol. 100:1409–22 [Google Scholar]
  132. Lykken DT. 1968. Statistical significance in psychological research. Psychol. Bull. 70:151–59 [Google Scholar]
  133. Marr JC, Thau S. 2014. Falling from great (and not-so-great) heights: how initial status position influences performance after status loss. Acad. Manag. J. 57:223–48 [Google Scholar]
  134. Marrow AJ. 1969. The Practical Theorist: The Life and Work of Kurt Lewin New York: Basic Books
  135. Martin S, Liao H, Campbell EM. 2013. Directive versus empowering leadership: a field experiment comparing impacts on task proficiency and proactivity. Acad. Manag. J. 56:1372–95 [Google Scholar]
  136. Mathias BD, Smith AD. 2016. Autobiographies in organizational research: using leaders’ life stories in a triangulated research design. Organ. Res. Methods 19:204–30 [Google Scholar]
  137. Martocchio JJ. 1994. Effects of conceptions of ability on anxiety, self-efficacy, and learning in training. J. Appl. Psychol. 79:819–25 [Google Scholar]
  138. Meglino BM, DeNisi AS, Youngblood SA, Williams KJ. 1988. Effects of realistic job previews: a comparison using an “enhancement” and a “reduction” preview. J. Appl. Psychol. 73:259–66 [Google Scholar]
  139. Meindl JR, Ehrlich SB, Dukerich JM. 1985. The romance of leadership. Adm. Sci. Q. 30:78–102 [Google Scholar]
  140. Mertens DM. 2015. Mixed methods and wicked problems. J. Mixed Methods Res. 9:13–6 [Google Scholar]
  141. Milkman KL, Akinola M, Chugh D. 2012. Temporal distance and discrimination: an audit study in academia. Psychol. Sci. 23:710–17 [Google Scholar]
  142. Milkman KL, Akinola M, Chugh D. 2015. What happens before? A field experiment exploring how pay and representation differentially shape bias on the pathway into organizations. J. Appl. Psychol. 100:1678–1712 [Google Scholar]
  143. Mitchell MS, Ambrose ML. 2012. Employees' behavioral reactions to supervisor aggression: an examination of individual and situational factors. J. Appl. Psychol. 97:61148–70 [Google Scholar]
  144. Mitchell TR, Thompson L, Peterson E, Cronk R. 1997. Temporal adjustments in the evaluation of events: the “rosy view.”. J. Exp. Soc. Psychol. 33:421–48 [Google Scholar]
  145. Molina-Azorín JF. 2012. Mixed methods research in strategic management: impact and applications. Organ. Res. Methods 15:33–56 [Google Scholar]
  146. Mook DG. 1983. In defense of external invalidity. Am. Psychol. 38:379–87 [Google Scholar]
  147. Morgan KL, Rubin DB. 2012. Rerandomization to improve covariate balance in experiments. Ann. Stat. 40:1263–82 [Google Scholar]
  148. Morgan WB, Walker SS, Hebl MR, King EB. 2013. A field experiment: reducing interpersonal discrimination toward pregnant job applicants. J. Appl. Psychol. 98:799–809 [Google Scholar]
  149. Morse NC, Reimer E. 1956. The experimental change of a major organizational variable. J. Abnorm. Soc. Psychol. 52:120–29 [Google Scholar]
  150. Moulton S, Collins JM, Loibl C, Samek A. 2015. Effects of monitoring on mortgage delinquency: evidence from a randomized field study. J. Policy Anal. Manag. 34:184–207 [Google Scholar]
  151. Münsterberg H. 1913. Psychology and Industrial Efficiency Boston, New York: Houghton Mifflin
  152. Nahum-Shani I, Qian M, Almirall D, Pelham WE, Gnagy B. et al. 2012. Experimental design and primary data analysis methods for comparing adaptive interventions. Psychol. Methods 17:4457–77 [Google Scholar]
  153. Noordzij G, van Hooft EAJ, van Mierlo H, van Dam A, Born MPh A. 2013. The effects of a learning‐goal orientation training on self‐regulation: a field experiment among unemployed job seekers. Pers. Psychol. 66:723–55 [Google Scholar]
  154. Oreg S, Berson Y. 2015. Personality and charismatic leadership in context: the moderating role of situational stress. Pers. Psychol. 68:49–77 [Google Scholar]
  155. Oz S, Eden D. 1994. Restraining the golem: boosting performance by changing the interpretation of low scores. J. Appl. Psychol. 79:744–54 [Google Scholar]
  156. Pager D. 2007. The use of field experiments for studies of employment discrimination: contributions, critiques, and directions for the future. Ann. Am. Acad. Pol. Soc. Sci. 609:104–33 [Google Scholar]
  157. Pastor JC, Mayo M, Shamir B. 2007. Adding fuel to fire: the impact of followers' arousal on ratings of charisma. J. Appl. Psychol. 92:1584–96 [Google Scholar]
  158. Pelligra V, Stanca L. 2013. To give or not to give? Equity, efficiency and altruistic behavior in an artefactual field experiment. J. Soc. Econ. 46:1–9 [Google Scholar]
  159. Petersen JA, Kumar V. 2015. Perceived risk, product returns, and optimal resource allocation: evidence from a field experiment. J. Mark. Res. 52:268–85 [Google Scholar]
  160. Pinker S. 2011. The Better Angels of Our Nature: Why Violence Has Declined New York: Penguin Books
  161. Podsakoff NP, Maynes TD, Whiting SW, Podsakoff PM. 2015. One (rating) from many (observations): factors affecting the individual assessment of voice behavior in groups. J. Appl. Psychol. 100:1189–202 [Google Scholar]
  162. Podsakoff PM, Dalton DR. 1987. Research methodology in organizational studies. J. Manag. 13:419–41 [Google Scholar]
  163. Porter LW, Schneider B. 2014. What was, what is, and what may be in OP/OB. Annu. Rev. Organ. Psychol. Organ. Behav. 1:1–21 [Google Scholar]
  164. Quillian L. 2006. New approaches to understanding racial prejudice and discrimination. Annu. Rev. Sociol. 32:299–328 [Google Scholar]
  165. Rausch JR, Maxwell SE, Kelley K. 2003. Analytic methods for questions pertaining to a randomized pretest, posttest, follow-up design. J. Clin. Child Adolesc. Psychol. 32:467–86 [Google Scholar]
  166. Reynolds GS, Bennett JB. 2015. A cluster randomized trial of alcohol prevention in small businesses: a cascade model of help seeking and risk reduction. Am. J. Health Promot. 29:3182–91 [Google Scholar]
  167. Rife JC, Belcher JR. 1994. Assisting unemployed older workers to become reemployed: an experimental evaluation. Res. Soc. Work Practice 4:3–13 [Google Scholar]
  168. Ringelmann M. 1913. Recherches sur les moteurs animés: travail de l'homme. Ann. Inst. Nat. Agron. 12:21–40 http://gallica.bnf.fr/ark:/12148/bpt6k54409695.image.f14.langEN [Google Scholar]
  169. Roethlisberger FJ, Dickson WJ. 1939. Management and the Worker Cambridge, MA: Harvard Univ. Press
  170. Rosen NA. 1970. Demand characteristics in a field experiment. J. Appl. Psychol. 54:163–68 [Google Scholar]
  171. Rosenbaum PR, Rubin D. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70:41–55 [Google Scholar]
  172. Rosenthal R. 1979. An introduction to the file drawer problem. Psychol. Bull. 86:638–41 [Google Scholar]
  173. Rubin DB. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66:688–701 [Google Scholar]
  174. Rousseau DM. 2012. Oxford Handbook of Evidence-Based Management Oxford, UK: Oxford Univ. Press
  175. Rynes SL. 2007. Let's create a tipping point: what academics and practitioners can do, alone and together. Acad. Manag. J. 50:1046–54 [Google Scholar]
  176. Rynes SL, Bartunek JM. 2017. Evidence-based management: foundations, development, controversies, and future. Annu. Rev. Organ. Psychol. Organ. Behav 4:235–61 [Google Scholar]
  177. Scandura TA, Graen GB. 1984. Moderating effects of initial leader–member exchange status on the effects of a leadership intervention. J. Appl. Psychol. 69:428–36 [Google Scholar]
  178. Scandura TA, Williams EA. 2000. Research methodology in management: current practices, trends, and implications for future research. Acad. Manag. J. 43:1248–64 [Google Scholar]
  179. Schaubroeck J, Ganster DC, Sime WE, Ditman D. 1993. A field experiment testing supervisory role clarification. Pers. Psychol. 46:1–25 [Google Scholar]
  180. Scheflen KC, Lawler EE III, Hackman JR. 1971. Long-term impact of employee participation in the development of pay incentive plans: a field experiment revisited. J. Appl. Psychol. 55:182–86 [Google Scholar]
  181. Schein EH. 2015. Organizational psychology then and now: some observations. Annu. Rev. Organ. Psychol. Organ. Behav. 2:1–19 [Google Scholar]
  182. Schraven DFJ, Hartmann A, Dewulf GPMR. 2015. Resuming an unfinished tale: applying causal maps to analyze the dominant logics within an organization. Organ. Res. Methods 18:326–49 [Google Scholar]
  183. Seijts GH, Latham GP. 2011. The effect of commitment to a learning goal, self-efficacy, and the interaction between learning goal difficulty and commitment on performance in a business simulation. Hum. Perform. 24:3189–204 [Google Scholar]
  184. Shadish WR. 2002. Revisiting field experimentation: field notes for the future. Psychol. Methods 7:3–18 [Google Scholar]
  185. Shadish WR. 2010. Campbell and Rubin: a primer and comparison of their approaches to causal inference in field settings. Psychol. Methods 15:3–17 [Google Scholar]
  186. Shadish WR, Clark MH, Steiner PM. 2008. Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment. J. Am. Stat. Assoc. 103:1334–43 [Google Scholar]
  187. Shadish WR, Cook TD. 2009. The renaissance of field experimentation in evaluating interventions. Annu. Rev. Psychol. 60:607–29 [Google Scholar]
  188. Shadish WR, Cook TD, Campbell DT. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton-Mifflin
  189. Shavit T, Rosenboim M, Shani Y. 2014. Time preference before and after a risky activity. J. Econ. Psychol. 43:30–36 [Google Scholar]
  190. Spera SP, Buhrfeind ED, Pennebaker JW. 1994. Expressive writing and coping with job loss. Acad. Manag. J. 37:722–33 [Google Scholar]
  191. Stajkovic AD, Lee D, Nyberg AJ. 2009. Collective efficacy, group potency, and group performance: meta-analyses of their relationships, and test of a mediation model. J. Appl. Psychol. 94:3814–28 [Google Scholar]
  192. Stajkovic AD, Luthans F. 1998. Self-efficacy and work-related performance: a meta-analysis. Psychol. Bull. 124:2240–61 [Google Scholar]
  193. Stone-Romero EF. 2011. Research strategies in industrial and organizational psychology: nonexperimental, quasi-experimental, and randomized experimental research in special purpose and nonspecial purpose settings. APA Handbook of Industrial and Organizational Psychology, Vol. 1, Building and Developing the Organization S Zedeck 37–72 Washington, DC: Am. Psychol. Assoc. [Google Scholar]
  194. Stone-Romero EF, Weaver AE, Glenar JL. 1995. Trends in research design and data analytic strategies in organizational research. J. Manag. 21:141–57 [Google Scholar]
  195. Stoop J. 2014. From the lab to the field: envelopes, dictators and manners. Exp. Econ. 17:304–13 [Google Scholar]
  196. Taverniers J, Van Ruysseveldt J, Smeets J, von Grumbkow J. 2010. High-intensity stress elicits robust cortisol increases, and impairs working memory and visuo-spatial declarative memory in Special Forces candidates: a field experiment. Stress 13:324–34 [Google Scholar]
  197. Teddlie C, Tashakkori A. 2008. Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences New York: Sage
  198. Terpstra DE. 1981. Relationship between methodological rigor and reported outcomes in organization development evaluation research. J. Appl. Psychol. 66:541–43 [Google Scholar]
  199. Thatcher SMB, Brown SA, Jenkins JL. 2012. E-collaboration media use and diversity perceptions: an evolutionary perspective of virtual organizations. Int. J. e-Collaboration 8:228–46 [Google Scholar]
  200. Thau S, Derfler-Rozin R, Pitesa M, Mitchell MS, Pillutla MM. 2015. Unethical for the sake of the group: risk of social exclusion and pro-group unethical behavior. J. Appl. Psychol. 100:98–113 [Google Scholar]
  201. Thibaut J, Friedland N, Walker L. 1974. Compliance with rules: some social determinants. J. Pers. Soc. Psychol. 30:792–801 [Google Scholar]
  202. Tucker DH, Rowe PM. 1977. Consulting the application form prior to the interview: an essential step in the selection process. J. Appl. Psychol. 62:283–87 [Google Scholar]
  203. Tziner A, Eden D. 1985. Effects of crew composition on crew performance: Does the whole equal the sum of its parts. J. Appl. Psychol. 70:85–93 [Google Scholar]
  204. Tziner A, Kopelman R. 1988. Effects of rating format on goal-setting dimensions: a field experiment. J. Appl. Psychol. 73:323–26 [Google Scholar]
  205. Uy MA, Foo MD, Aguinis H. 2010. Using experience sampling methodology to advance entrepreneurship theory and research. Org. Res. Methods 13:31–54 [Google Scholar]
  206. Vancouver JB, Carlson BW. 2015. All things in moderation, including tests of mediation (at least some of the time). Organ. Res. Methods 18:170–91 [Google Scholar]
  207. Vinokur AD, Price RH, Caplan RD. 1991. From field experiments to program implementation: assessing the potential outcomes of an experimental intervention program for unemployed persons. Am. J. Community Psychol. 19:543–62 [Google Scholar]
  208. Vinokur AD, Price RH, Schul Y. 1995. Impact of the JOBS intervention on unemployed workers varying in risk for depression. Am. J. Community Psychol. 23:39–74 [Google Scholar]
  209. Wanous JP, Reichers AE. 2000. New employee orientation programs. Hum. Resour. Manag. Rev. 10:435–51 [Google Scholar]
  210. Waung M. 1995. The effects of self-regulatory coping orientation on newcomer adjustment and job survival. Pers. Psychol. 48:633–50 [Google Scholar]
  211. Weick KE. 1967. Organizations in the laboratory. Methods of Organizational Research VH Vroom 1–56 Pittsburgh: Univ. Pittsb. Press [Google Scholar]
  212. Wheeler AR, Shanine KK, Leon MR, Whitman MV. 2014. Student-recruited samples in organizational research: a review, analysis, and guidelines for future research. J. Occup. Organ. Psychol. 87:1–26 [Google Scholar]
  213. Xu J, Wu Y. 2015. Using Twitter in crisis management for organizations bearing different country-of-origin perceptions. J. Commun. Manag. 19:239–53 [Google Scholar]
  214. Yanar B, Budworth M, Latham GP. 2009. The effect of verbal self-guidance training for overcoming employment barriers: a study of Turkish women. Appl. Psychol. 58:586–601 [Google Scholar]
  215. Yeow J, Martin R. 2013. The role of self-regulation in developing leaders: a longitudinal field experiment. Leadersh. Q. 24:625–37 [Google Scholar]
  216. Zielhorst T, van den Brule D, Visch V, Melles M, van Tienhoven S. et al. 2015. Using a digital game for training desirable behavior in cognitive–behavioral therapy of burnout syndrome: a controlled study. Cyberpsychol. Behav. Soc. Netw. 18:2101–11 [Google Scholar]
/content/journals/10.1146/annurev-orgpsych-041015-062400
Loading
  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error