1932

Abstract

We examine the phenomenon of impact assessment in the practice of scientific research, paying attention to the context in which project evaluation is used in federally funded research on plant health in the United States. Our analysis, which is derived from systems theory, carries out a particular view of the research process. For the purposes of this review, our use of the term systems theory references the body of interdisciplinary work that deals with the organization and function of complex structures in nature and human society. Key concepts in this body of theory are that both the components and the interactions among components are important in understanding behavior and that, frequently, systems are seen to be hierarchical in structure. The aim of our analysis is to bring to the attention of the plant health community several concepts from the social sciences that might help in understanding how researchers have responded to the increased expectations from funders to provide project evaluations and impact assessments. We generate a synthesis of these theories, which have not previously been used in a unified way, to explain choices in response to newly imposed goals. Although our analysis is motivated by a specific disciplinary focus on plant health, the issues we discuss are general. Thus, we hope the review is useful to a wide range of scientists, science program managers, and policymakers.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-phyto-080417-050125
2024-09-09
2024-12-12
Loading full text...

Full text loading...

/deliver/fulltext/phyto/62/1/annurev-phyto-080417-050125.html?itemId=/content/journals/10.1146/annurev-phyto-080417-050125&mimeType=html&fmt=ahah

Literature Cited

  1. 1.
    Am. Eval. Assoc. 2024.. Guiding principles for evaluators. . American Evaluation Association. https://www.eval.org/About/Guiding-Principles
    [Google Scholar]
  2. 2.
    Am. Eval. Assoc. 2024.. Program evaluation methodologies and practices. . American Evaluation Association. https://www.eval.org/Policy-Advocacy/Evaluation-Policy-Initiative/Program-Evaluation
    [Google Scholar]
  3. 3.
    Alberts B, Johnson A, Lewis J, Raff M, Roberts K, Walter P. 2002.. Molecular Biology of the Cell. New York:: Garland Sci. , 4th ed..
    [Google Scholar]
  4. 4.
    Bard JF. 1998.. Practical Bilevel Optimization, Vol. 30. Boston, MA:: Springer US
    [Google Scholar]
  5. 5.
    Beck Y, Ljubić I, Schmidt M. 2023.. A survey on bilevel optimization under uncertainty. . Eur. J. Oper. Res. 311:(2):40126
    [Crossref] [Google Scholar]
  6. 6.
    Ben-Ayed O. 1993.. Bilevel linear programming. . Comput. Oper. Res. 20:(5):485501
    [Crossref] [Google Scholar]
  7. 7.
    Bohanec M. 2021.. From data and models to decision support systems: lessons and advice for the future. . In EURO Working Group on DSS, ed. J Papathanasiou, P Zaraté, J Freire De Sousa , pp. 191211. Cham, Switz:.: Springer
    [Google Scholar]
  8. 8.
    Bornmann L. 2013.. What is societal impact of research and how can it be assessed? A literature survey. . J. Am. Soc. Inf. Sci. Technol. 64:(2):21733
    [Crossref] [Google Scholar]
  9. 9.
    Burt RS. 1992.. Structural Holes: The Social Structure of Competition. Cambridge, MA:: Harvard Univ. Press
    [Google Scholar]
  10. 10.
    Campbell DT. 1979.. Assessing the impact of planned social change. . Eval. Program Plann. 2:(1):6790
    [Crossref] [Google Scholar]
  11. 11.
    Giampietro M. 2003.. Multi-Scale Integrated Analysis of Agroecosystems. Boca Raton, FL:: CRC Press. , 1st ed..
    [Google Scholar]
  12. 12.
    Glerup C, Davies SR, Horst M. 2017.. “ Nothing really responsible goes on here”: scientists’ experience and practice of responsibility. . J. Responsib. Innov. 4:(3):31936
    [Crossref] [Google Scholar]
  13. 13.
    Goodhart C. 1975.. Monetary relationships : a view from Threadneedle Street. . Pap. Monet. Econ. 1::127
    [Crossref] [Google Scholar]
  14. 14.
    Hamilton M, Salerno J, Fisher AP. 2019.. Cognition of complexity and trade-offs in a wildfire-prone social-ecological system. . Environ. Res. Lett. 14::125017
    [Crossref] [Google Scholar]
  15. 15.
    Hedlund J, Bodin Ö, Nohrstedt D. 2021.. Assessing policy issue interdependencies in environmental governance. . Int. J. Commons 15:(1):8299
    [Crossref] [Google Scholar]
  16. 16.
    Hirsch F. 1978.. Social Limits to Growth. Abingdon, UK:: Routledge
    [Google Scholar]
  17. 17.
    Koestler A. 1967.. The Ghost in the Machine. New York:: Macmillan. , 1st Am.. ed .
    [Google Scholar]
  18. 18.
    Koestler A. 1970.. Beyond atomism and holism—the concept of the holon. . Perspect. Biol. Med. 13:(2):13154
    [Crossref] [Google Scholar]
  19. 19.
    Levy MA, Lubell MN. 2018.. Innovation, cooperation, and the structure of three regional sustainable agriculture networks in California. . Reg. Environ. Change 18:(4):123546
    [Crossref] [Google Scholar]
  20. 20.
    Levy MA, Lubell MN, McRoberts N. 2018.. The structure of mental models of sustainable agriculture. . Nat. Sustain. 1::41320
    [Crossref] [Google Scholar]
  21. 21.
    Long NE. 1958.. The local community as an ecology of games. . Am. J. Sociol. 64:(3):25161
    [Crossref] [Google Scholar]
  22. 22.
    Mattson C, Bushardt RL, Artino AR. 2021.. When a measure becomes a target, it ceases to be a good measure. . J. Grad. Med. Educ. 13:(1):25
    [Crossref] [Google Scholar]
  23. 23.
    McGlashan J, de la Haye K, Wang P, Allender S. 2019.. Collaboration in complex systems: multilevel network analysis for community-based obesity prevention interventions. . Sci. Rep. 9::12599
    [Crossref] [Google Scholar]
  24. 24.
    Patton MQ, Asibey E, Dean-Coffey J, Kelley R, Miranda R, et al. What is evaluation? Rep. , Am. Eval. Assoc., Washington, DC:. https://www.eval.org/Portals/0/What%20is%20evaluation%20Document.pdf
    [Google Scholar]
  25. 25.
    Sci. Eur. 2024.. Research assessment. . Science Europe. https://scienceeurope.org/our-priorities/research-assessment/
    [Google Scholar]
  26. 26.
    Silviu-Mihail TI. 2011.. The Romanian research performances evaluated with Dexi: program for multi-attribute decision-making. SSRN Work. Pap. 1881643
    [Google Scholar]
  27. 27.
    Smith NJ, Sage AP. 1973.. An introduction to hierarchical systems theory. . Comput. Electr. Eng. 1:(1):5571
    [Crossref] [Google Scholar]
  28. 28.
    UK Res. Innov. 2024.. What we do. . UKRI. https://www.ukri.org/what-we-do/
    [Google Scholar]
  29. 29.
    Univ. Calif. Divis. Agric. Nat. Resourc. 2024.. UC ANR condition changes. . UCANR.edu. https://ucanr.edu/sites/anrstaff/Divisionwide_Planning/UC_ANR_Condition_Changes/
    [Google Scholar]
  30. 30.
    USDA-NIFA. 2024.. Generic logic model for NIFA reporting. Rep. , USDA-NIFA, Washington, DC:. https://www.nifa.usda.gov/sites/default/files/resource/Generic%20Logic%20Model%20for%20NIFA%20Reporting.pdf
    [Google Scholar]
  31. 31.
    von Stackelberg H. 1934.. Market Structure and Equilibrium. Berlin:: Springer Sci.
    [Google Scholar]
  32. 32.
    White LA. 1938.. Science is sciencing. . Philos. Sci. 5:(4):36989
    [Crossref] [Google Scholar]
/content/journals/10.1146/annurev-phyto-080417-050125
Loading
/content/journals/10.1146/annurev-phyto-080417-050125
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error