Theory and Technology in Organizational Psychology: A Review of Technology Integration Paradigms and Their Effects on the Validity of Theory

Literature Cited

1. Aguinis H, Pierce CA. 2008. Enhancing the relevance of organizational behavior by embracing performance management research. J. Organ. Behav. 29:1139–45
   [Google Scholar]
2. Arthur W, Day EA, McNelly TL, Edens PS 2003. A meta-analysis of the criterion-related validity of assessment center dimensions. Pers. Psychol. 56:1125–53
   [Google Scholar]
3. Arthur W, Doverspike D, Muñoz GJ, Taylor JE, Carr AE 2014. The use of mobile devices in high-stakes remotely delivered assessments and testing. Int. J. Sel. Assess. 22:2113–23
   [Google Scholar]
4. Arthur W, Keiser NL, Doverspike D 2018. An information-processing-based conceptual framework of the effects of unproctored Internet-based testing devices on scores on employment-related assessments and tests. Hum. Perform. 31:11–32
   [Google Scholar]
5. Arthur W, Villado AJ. 2008. The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. J. Appl. Psychol. 93:2435–42
   [Google Scholar]
6. Bartels LK, Nordstrom CR. 2012. Examining big brother's purpose for using electronic performance monitoring. Perform. Improv. Q. 25:65–77
   [Google Scholar]
7. Bedwell WL, Pavlas D, Heyne K, Lazzara EH, Salas E 2012. Toward a taxonomy linking game attributes to learning: an empirical study. Simul. Gaming 43:6729–60
   [Google Scholar]
8. Beekun RI. 1989. Assessing the effectiveness of sociotechnical interventions: Antidote or fad. ? Hum. Relat. 42:10877–97
   [Google Scholar]
9. Behrend TS, Thompson LF. 2012. Using animated agents in learner-controlled training: the effects of design control. Int. J. Train. Dev. 16:4263–83
   [Google Scholar]
10. Bell BS, Kozlowski SWJ. 2008. Active learning: effects of core training design elements on self-regulatory processes, learning, and adaptability. J. Appl. Psychol. 93:296–316
    [Google Scholar]
11. Bell BS, Tannenbaum SI, Ford JK, Noe RA, Kraiger K 2017. 100 years of training and development research: what we know and where we should go. J. Appl. Psychol. 102:3305–23
    [Google Scholar]
12. Bertolotti F, Mattarelli E, Vignoli M, Macrì DM 2015. Exploring the relationship between multiple team membership and team performance: the role of social networks and collaborative technology. Res. Policy 44:4911–24
    [Google Scholar]
13. Bhave DP. 2014. The invisible eye? Electronic performance monitoring and employee job performance. Pers. Psychol. 67:605–35
    [Google Scholar]
14. Bjögvinsson E, Ehn P, Hillgren P-A 2012. Design things and design thinking: contemporary participatory design challenges. Des. Issues 28:3101–16
    [Google Scholar]
15. Bonaccio S, O'Reilly J, O'Sullivan SL, Chiocchio F 2016. Nonverbal behavior and communication in the workplace: a review and an agenda for research. J. Manag. 42:51044–74
    [Google Scholar]
16. Borrego M, Newswander L. 2010. Definitions of interdisciplinary research: toward graduate-level interdisciplinary learning outcomes. Rev. Higher Educ. 34:161–84
    [Google Scholar]
17. Bostrom RP, Gupta S, Thomas D 2009. A meta-theory for understanding information systems within sociotechnical systems. J. Manag. Inf. Syst. 26:117–48
    [Google Scholar]
18. Burtt HE. 1926. Principles of Employment Psychology Cambridge, MA: Houghton Mifflin Co.
19. Campbell JP. 1990. Modeling the performance prediction problem in industrial and organizational psychology. Handbook of Industrial and Organizational Psychology MD Dunnette, LM Hough 687–732 Mountain View, CA: Consult. Psychol. Press
    [Google Scholar]
20. Campbell JP. 2012. Leadership, the old, the new, and the timeless: a commentary. The Oxford Handbook of Leadership MG Rumsey 401–22 Oxford, UK: Oxford Univ. Press
    [Google Scholar]
21. Carolan TF, Hutchins SD, Wickens CD, Cumming JM 2014. Costs and benefits of more learner freedom: meta-analyses of exploratory and learner control training methods. Hum. Factors 56:5999–1014
    [Google Scholar]
22. Cascio WF, Montealegre R. 2016. How technology is changing work and organizations. Annu. Rev. Organ. Psychol. Organ. Behav. 3:349–75
    [Google Scholar]
23. Cattell RB. 1966. Psychological theory and scientific method. Handbook of Multivariate Experimental Psychology RB Cattell 1–18 Chicago: Rand McNally
    [Google Scholar]
24. Christian MS, Edwards BD, Bradley JC 2010. Situational judgment tests: constructs assessed and a meta-analysis of their criterion-related validities. Pers. Psychol. 63:183–117
    [Google Scholar]
25. Chuah SC, Drasgow F, Roberts BW 2006. Personality assessment: Does the medium matter? No. J. Res. Pers. 40:4359–76
    [Google Scholar]
26. Demir M, McNeese NJ, Cooke NJ 2018. The impact of perceived autonomous agents on dynamic team behaviors. IEEE Trans. Emerg. Top. Comput. Intell. 2:4258–67
    [Google Scholar]
27. DeNisi AS, Murphy KR. 2017. Performance appraisal and performance management: 100 years of progress. ? J. Appl. Psychol. 102:3421–33
    [Google Scholar]
28. Fenner DB, Lerch FJ, Kulik CT 1993. The impact of computerized performance monitoring and prior performance knowledge on performance evaluation. J. Appl. Soc. Psychol. 23:7573–601
    [Google Scholar]
29. Finholt T, Sproull LS. 1990. Electronic groups at work. Organ. Sci. 1:141–64
    [Google Scholar]
30. Friend CL, Cole CL. 1990. Learner control in computer-based instruction: a current literature review. Educ. Technol. 30:1147–49
    [Google Scholar]
31. Fuller RM, Dennis AR. 2009. Does fit matter? The impact of task-technology fit and appropriation on team performance in repeated tasks. Inf. Syst. Res. 20:12–17
    [Google Scholar]
32. Gaver WW. 1991. Technology affordances. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems79–84 New York: ACM Press
    [Google Scholar]
33. Giddens L, Gonzalez E, Leidner D 2016. I track, therefore I am: exploring the impact of wearable fitness devices on employee identity and well-being. Proceedings of the 22nd Americas Conference on Information Systems1–5 Atlanta: Assoc. Inf. Syst. eLibr.
    [Google Scholar]
34. Golden TD, Veiga JF, Dino RN 2008. The impact of professional isolation on teleworker job performance and turnover intentions: Does time spent teleworking, interacting face-to-face, or having access to communication-enhancing technology matter. ? J. Appl. Psychol. 93:61412–21
    [Google Scholar]
35. Greeno JG. 1994. Gibson's affordances. Psychol. Rev. 101:2336–42
    [Google Scholar]
36. Grelle D, Gutierrez S. 2019. Developing device-equivalent and effective measures of complex thinking with an information processing framework and mobile first design principles. Pers. Assess. Decis. 5:34
    [Google Scholar]
37. Han L, Zhang Q, Chen X, Zhan Q, Yang T, Zhao Z 2017. Detecting work-related stress with a wearable device. Comput. Ind. 90:42–49
    [Google Scholar]
38. Harris AM, Gómez-Zará D, DeChurch LA, Contractor NS 2019. Joining together online: the trajectory of CSCW scholarship on group formation. Proc. ACM Hum.-Comput. Interact. 3:CSCW148
    [Google Scholar]
39. Hartman RI, Stoner CR, Arora R 1991. An investigation of selected variables affecting telecommuting productivity and satisfaction. J. Bus. Psychol. 6:2207–25
    [Google Scholar]
40. Hassan L, Xi N, Gurkan B, Koivisto J, Hamari J 2020. Gameful self-regulation: a study on how gamified self-tracking features evoke gameful experiences. Proceedings of the 53rd Hawaii International Conference on System Sciences1103–12 Atlanta: Assoc. Inf. Syst. eLibr.
    [Google Scholar]
41. Hausknecht JP, Day DV, Thomas SC 2004. Applicant reactions to selection procedures: an updated model and meta‐analysis. Pers. Psychol. 57:639–83
    [Google Scholar]
42. Hoch JE, Kozlowski SWJ. 2014. Leading virtual teams: hierarchical leadership, structural supports, and shared team leadership. J. Appl. Psychol. 99:3390–403
    [Google Scholar]
43. Howard A. 1995. The Changing Nature of Work San Francisco: Jossey-Bass
44. Hunt ST. 2011. Technology is transforming the nature of performance management. Ind. Organ. Psychol. 4:2188–89
    [Google Scholar]
45. Illingworth AJ, Morelli NA, Scott JC, Boyd SL 2015. Internet-based, unproctored assessments on mobile and non-mobile devices: usage, measurement equivalence, and outcomes. J. Bus. Psychol. 30:2325–43
    [Google Scholar]
46. Jeske D. 2011. Electronic performance monitoring: employee perceptions and reactions PhD Diss., North. Ill. Univ.
47. Karim MN, Behrend TS. 2014. Reexamining the nature of learner control: dimensionality and effects on learning and training reactions. J. Bus. Psychol. 29:187–99
    [Google Scholar]
48. Kellogg KC, Valentine MA, Christin A 2020. Algorithms at work: the new contested terrain of control. Acad. Manag. Ann. 14:1366–410
    [Google Scholar]
49. King WC, Miles EW. 1995. A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pencil measurements: a test of measurement equivalence. J. Appl. Psychol. 80:6643–51
    [Google Scholar]
50. Kraiger K, Jerden E. 2007. A meta-analytic investigation of learner control: old findings and new directions. Toward a Science of Distributed Learning SM Fiore, E Salas 65–90 Washington, DC: Am. Psychol. Assoc.
    [Google Scholar]
51. Kruglanski AW, Chernikova M, Rosenzweig E, Kopetz C 2014. On motivational readiness. Psychol. Rev. 121:3367–88
    [Google Scholar]
52. Landers RN, Reddock CM. 2017. A meta-analytic investigation of objective learner control in web-based instruction. J. Bus. Psychol. 32:4455–78
    [Google Scholar]
53. LaPort K. 2016. Mobile assessment: comparing traditional cognitive, cognitive reasoning, and non-cognitive performance. In TD McGlochlin (Chair) Mobile equivalence: expanding research across assessment methods, levels, and devices. Presented at the 31st Annual Conference of the Society for Industrial and Organizational Psychology, Anaheim, CA
54. Larson EL, Landers TF, Begg MD 2011. Building interdisciplinary research models: a didactic course to prepare interdisciplinary scholars and faculty. Clin. Transl. Sci. 4:138–41
    [Google Scholar]
55. Larson L, DeChurch LA. 2020. Leading teams in the digital age: four perspectives on technology and what they mean for leading teams. Leadersh. Q. 31:1101377
    [Google Scholar]
56. Leonardi PM. 2012. Materiality, sociomateriality, and socio-technical systems: What do these terms mean? How are they different? Do we need them?. Materiality and Organizing: Social Interaction in a Technological World PM Leonardi, BA Nardi, J Kallinikos 25–48 Oxford, UK: Oxford Univ. Press
    [Google Scholar]
57. Lievens F, Sackett PR. 2017. The effects of predictor method factors on selection outcomes: a modular approach to personnel selection procedures. J. Appl. Psychol. 102:143–66
    [Google Scholar]
58. Loevinger J. 1957. Objective tests as instruments of psychological theory. Psychol. Rep. 3:635–94
    [Google Scholar]
59. Loken E, Gelman A. 2017. Measurement error and the replication crisis. Science 355:6325584–85
    [Google Scholar]
60. Mak S, Kozlowski SWJ. 2019. Virtual teams: conceptualization, integrative review, and research recommendations. The Cambridge Handbook of Technology and Employee Behavior RN Landers 441–79 Cambridge, UK: Cambridge Univ. Press
    [Google Scholar]
61. Mallo J, Nordstrom CR, Bartels LK, Traxler A 2007. The effect of age and task difficulty. Perform. Improv. Q. 20:149–64
    [Google Scholar]
62. Maruping LM, Agarwal R. 2004. Managing team interpersonal processes through technology: a task-technology fit perspective. J. Appl. Psychol. 89:6975–90
    [Google Scholar]
63. Mathieu J, Maynard MT, Rapp T, Gilson L 2008. Team effectiveness 1997–2007: a review of recent advancements and a glimpse into the future. J. Manag. 34:3410–76
    [Google Scholar]
64. Matusik JG, Heidl R, Hollenbeck JR, Yu A, Lee HW, Howe M 2019. Wearable bluetooth sensors for capturing relational variables and temporal variability in relationships: a construct validation study. J. Appl. Psychol. 104:3357–87
    [Google Scholar]
65. McNall LA, Roch SG. 2007. Effects of electronic monitoring types on perceptions of procedural justice, interpersonal justice, and privacy. J. Appl. Soc. Psychol. 37:3658–82
    [Google Scholar]
66. Mead AD, Drasgow F. 1993. Equivalence of computerized and paper-and-pencil cognitive ability tests: a meta-analysis. Psychol. Bull. 114:3449–58
    [Google Scholar]
67. Meade AW, Michels LC, Lautenschlager GJ 2007. Are Internet and paper-and-pencil personality tests truly comparable?: An experimental design measurement invariance study. Organ. Res. Methods 10:2322–45
    [Google Scholar]
68. Morelli NA, Mahan RP, Illingworth AJ 2014. Establishing the measurement equivalence of online selection assessments delivered on mobile versus nonmobile devices. Int. J. Sel. Assess. 22:2124–38
    [Google Scholar]
69. Morgeson FP, Garza AS, Campion MA 2013. Work design. Handbook of Psychology, Vol. 12 IB Weiner, NW Schmitt, S Highhouse 525–59 Hoboken, NJ: John Wiley & Sons 2nd ed.
    [Google Scholar]
70. Morschheuser B, Hamari J, Werder K, Abe J 2017. How to gamify? A method for designing gamification. Proceedings of the 50th Hawaii International Conference on System Sciences1298–1307 Atlanta: Assoc. Inf. Syst. eLibr.
    [Google Scholar]
71. Nadolny L, Valai A, Cherrez NJ, Elrick D, Lovett A, Nowatzke M 2020. Examining the characteristics of game-based learning: a content analysis and design framework. Comput. Educ. 156:103936
    [Google Scholar]
72. Naglieri JA, Drasgow F, Schmit M, Handler L, Prifitera A et al. 2004. Psychological testing on the Internet: new problems, old issues. Am. Psychol. 59:3150–62
    [Google Scholar]
73. Orlikowski WJ. 2007. Sociomaterial practices: exploring technology at work. Organ. Stud. 28:91435–48
    [Google Scholar]
74. Orlikowski WJ, Scott SV. 2008. Sociomateriality: challenging the separation of technology, work and organization. Acad. Manag. Ann. 2:1433–74
    [Google Scholar]
75. Pasmore W, Francis C, Haldeman J, Shani A 1982. Sociotechnical systems: a North American reflection on empirical studies of the seventies. Hum. Relat. 35:121179–204
    [Google Scholar]
76. Payne SC, Horner MT, Boswell WR, Schroeder AN, Stine‐Cheyne KJ 2009. Comparison of online and traditional performance appraisal systems. J. Manag. Psychol. 24:6526–44
    [Google Scholar]
77. Peppoloni L, Filippeschi A, Ruffaldi E, Avizzano CA 2016. A novel wearable system for the online assessment of risk for biomechanical load in repetitive efforts. Int. J. Ind. Ergon. 52:1–11
    [Google Scholar]
78. Plattner H, Meinel C, Leifer L 2011. Design Thinking: Understand—Improve—Apply Berlin: Springer-Verlag
79. Pulakos ED, Arad S, Donovan MA, Plamondon KE 2000. Adaptability in the workplace: development of a taxonomy of adaptive performance. J. Appl. Psychol. 85:4612–24
    [Google Scholar]
80. Pulakos ED, Hanson RM, Arad S, Moye N 2015. Performance management can be fixed: an on-the-job experiential learning approach for complex behavior change. Ind. Organ. Psychol. 8:151–76
    [Google Scholar]
81. Ravid DM, Tomczak DL, White JC, Behrend TS 2020. EPM 20/20: a review, framework, and research agenda for electronic performance monitoring. J. Manag. 46:1100–26
    [Google Scholar]
82. Richman-Hirsch WL, Olson-Buchanan JB, Drasgow F 2000. Examining the impact of administration medium on examinee perceptions and attitudes. J. Appl. Psychol. 85:6880–87
    [Google Scholar]
83. Robinson C, Nathan M. 2001. Considerations of learning and learning research: revisiting the “media effects” debate. J. Interact. Learn. Res. 12:169–88
    [Google Scholar]
84. Sailer M, Homner L. 2020. The gamification of learning: a meta-analysis. Educ. Psychol. Rev. 32:177–112
    [Google Scholar]
85. Schleicher DJ, Baumann HM, Sullivan DW, Levy PE, Hargrove DC, Barros-Rivera BA 2018. Putting the system into performance management systems: a review and agenda for performance management research. J. Manag. 44:62209–45
    [Google Scholar]
86. Schmidt AH, Robbins AST, Combs JK, Freeburg A, Jesperson RG et al. 2012. A new model for training graduate students to conduct interdisciplinary, interorganizational, and international research. BioScience 62:3296–304
    [Google Scholar]
87. Schmidt GB. 2014. Virtual leadership: an important leadership context. Ind. Organ. Psychol. 7:2182–87
    [Google Scholar]
88. Schwab K. 2016. The Fourth Industrial Revolution New York: Crown Publ. Group
89. Schweitzer L, Duxbury L. 2010. Conceptualizing and measuring the virtuality of teams. Inf. Syst. J. 20:3267–95
    [Google Scholar]
90. Shah JY, Kruglanski AW. 2003. When opportunity knocks: bottom-up priming of goals by means and its effects on self-regulation. J. Pers. Soc. Psychol. 84:61109–22
    [Google Scholar]
91. Sheer VC. 2012. Does e-mail facilitate negative performance feedback giving?: Supervisor and subordinate responses compared via the concept of social accountability. Commun. Stud. 63:2220–42
    [Google Scholar]
92. Shute VJ. 2011. Stealth assessment in computer-based games to support learning. Computer Games and Instruction S Tobias, JD Fletcher 503–23 Charlotte, NC: Inf. Age Publ.
    [Google Scholar]
93. Shute VJ, Moore GR. 2017. Consistency and validity in game-based stealth assessment. Technology Enhanced Innovative Assessment: Development, Modeling and Scoring from an Interdisciplinary Perspective H Jiao, RW Lissitz 31–51 Charlotte, NC: Inf. Age Publ.
    [Google Scholar]
94. Stanton JM, Julian AL. 2002. The impact of electronic monitoring on quality and quantity of performance. Comp. Hum. Behav. 18:185–101
    [Google Scholar]
95. Tellegen A, Waller NG. 2008. Exploring personality through test construction: development of the Multidimensional Personality Questionnaire. The SAGE Handbook of Personality Theory and Assessment, Vol. 2 Personality Measurement and Testing, ed. GJ Boyle, G Matthews, DH Saklofske 261–92 Thousand Oaks, CA: SAGE
    [Google Scholar]
96. Tippins NT. 2009. Internet alternatives to traditional proctored testing: Where are we now?. Ind. Organ. Psychol. 2:12–10
    [Google Scholar]
97. Tippins NT. 2015. Technology and assessment in selection. Annu. Rev. Organ. Psychol. Organ. Behav. 2:551–82
    [Google Scholar]
98. Tippins NT, Beaty J, Drasgow F, Gibson WM, Pearlman K et al. 2006. Unproctored Internet testing in employment settings. Pers. Psychol. 59:1189–225
    [Google Scholar]
99. Waclawski J. 2002. Large-scale organizational change and performance: an empirical examination. Hum. Resour. Dev. Q. 13:3289–305
    [Google Scholar]
100. Walliser JC, de Visser EJ, Wiese E, Shaw TH 2019. Team structure and team building improve human-machine teaming with autonomous agents. J. Cogn. Eng. Decis. Making 13:4258–78
     [Google Scholar]
101. Wesche JS, Sonderegger A. 2019. When computers take the lead: the automation of leadership. Comput. Hum. Behav. 101:197–209
     [Google Scholar]
102. Wynn DC, Clarkson PJ. 2018. Process models in design and development. Res. Eng. Design 29:2161–202
     [Google Scholar]
103. Yigitbasioglu OM, Velcu O. 2012. A review of dashboards in performance management: implications for design and research. Int. J. Account. Inf. Syst. 13:141–59
     [Google Scholar]