1932
0

Annual Review of Control, Robotics, and Autonomous Systems

Volume 8, 2025

Robotics at the University of Pennsylvania from Birth to Maturity: A Review of 30 Years of Research

  • Ruzena Bajcsy1,2
  • 1Department of Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA; email: [email protected] 2Center for Information Technology Research in the Interest of Society (CITRIS), University of California, Berkeley, California, USA
  • Vol. 8:49-75 (Volume publication date May 2025)
  • First published as a Review in Advance on December 02, 2024
  • Copyright © 2025 by the author(s).
    This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See credit lines of images or other third-party material in this article for license information.

Abstract

This article is an attempt to reexamine the evolution of robotics research at the University of Pennsylvania and all that it entailed, covering the successes and struggles of PhD students, postdoctoral researchers, and a few dedicated faculty between 1972 and 2000. In 1945, Penn's Moore School of Electrical Engineering was famous for developing the first electronic digital computer, ENIAC, but after a few years, the research had diminished. In 1972, a new Department of Computer and Information Science was formed. I came to Penn from Stanford University's Artificial Intelligence Laboratory full of energy, enthusiasm, and the goal of establishing a similar lab at Penn. This article demonstrates the creativity and ingenuity of young professionals at the General Robotics, Automation, Sensing, and Perception (GRASP) Laboratory. We built hardware. We built software. We collaborated with psychologists and electrical and mechanical engineers and tried to build a community of roboticists. Our curiosity led us to build new vision and tactile systems and to investigate cooperative robotics systems on the ground and in the air. We used computational models supported and verified by experiments. Today, I am proud to say that almost all of the students who cycled through the GRASP Lab are successful in academia or industry.

Keyword(s): active perception, GRASP Laboratory, representation of the real world, segmentation of 3D scenes
Loading

Article metrics loading...

/content/journals/10.1146/annurev-control-062723-111547
2025-05-05
2025-06-22
Loading full text...

Full text loading...

/deliver/fulltext/control/8/1/annurev-control-062723-111547.html?itemId=/content/journals/10.1146/annurev-control-062723-111547&mimeType=html&fmt=ahah

Literature Cited

  1. 1.
    Stanford Univ. Sch. Eng. 2023.. 60 years of artificial intelligence at Stanford. . YouTube. https://www.youtube.com/watch?v=Cn6nmWlu1EA
    [Google Scholar]
  2. 2.
    Winston PH, Brown RH, eds. 1982.. Artificial Intelligence: An MIT Perspective, Vol. 1: Expert Problem Solving, Natural Language Understanding, Intelligent Computer Coaches, Representation and Learning. Cambridge, MA:: MIT Press
     [Google Scholar]
  3. 3.
    Newell A, Simon HA. 1972.. Human Problem Solving. Englewood Cliffs, NJ:: Prentice Hall
     [Google Scholar]
  4. 4.
    McCarthy J. 1959.. Programs with common sense. Work. Pap., Dep. Comput. Sci., Stanford Univ., Stanford, CA:
     [Google Scholar]
  5. 5.
    McCarthy J. 1960.. Recursive functions of symbolic expressions and their computation by machine, part I. . Commun. ACM 3:(4):184–95
     [Crossref] [Google Scholar]
  6. 6.
    Landsat Leg. 2022.. Fifty years of exploration and innovation: how Landsat launched the remote sensing era. . US Geological Survey. https://www.usgs.gov/landsat-legacy/fifty-years-exploration-and-innovation-how-landsat-launched-remote-sensing-era
    [Google Scholar]
  7. 7.
    Horn BKP. 2024.. Patrick Winston and the MIT AI Lab copy demo (1970). . Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory. https://people.csail.mit.edu/bkph/phw_copy_demo.shtml
    [Google Scholar]
  8. 8.
    Sobel I. 2015.. History and definition of the so-called “Sobel Operator,” more appropriately named the Sobel-Feldman Operator. https://www.researchgate.net/publication/239398674_An_Isotropic_3x3_Image_Gradient_Operator
    [Google Scholar]
  9. 9.
    Gregory RL. 1970.. The Intelligent Eye. New York:: McGraw-Hill
     [Google Scholar]
  10. 10.
    Horn B, Brooks M. 1989.. Shape from Shading. Cambridge, MA:: MIT Press
     [Google Scholar]
  11. 11.
    Bajcsy R, Lieberman L. 1976.. Texture gradient as a depth cue. . Comput. Graph. Image Process. 5:(1):52–67
     [Crossref] [Google Scholar]
  12. 12.
    Horn BKP, Schunck BG. 1981.. Determining optical flow. . Artif. Intell. 17:(1–3):185–203
     [Crossref] [Google Scholar]
  13. 13.
    Bajcsy R, Tavakoli M. 1975.. Image filtering—a context dependent process. . IEEE Trans. Circ. Syst. 22:(5):463–74
     [Crossref] [Google Scholar]
  14. 14.
    Gibson JJ. 1950.. The Perception of the Visual World. Boston:: Houghton Mifflin
     [Google Scholar]
  15. 15.
    Bajcsy R. 1972.. Computer identification of textured visual scenes. PhD Thesis , Stanford Univ., Stanford, CA:
     [Google Scholar]
  16. 16.
    Rosenfeld A. 1969.. Picture processing by computer. . ACM Comput. Surv. 1:(3):147–76
     [Crossref] [Google Scholar]
  17. 17.
    Scidmore SL. 1977.. Computer analysis and description of pottery sherd patterns. Tech. Rep. MS-CIS-78-16 , Dep. Comput. Inf. Sci., Univ. Pa., Philadelphia:
     [Google Scholar]
  18. 18.
    Binford T. 1971.. Visual perception by a computer. . In Proceedings of the IEEE Conference on Systems and Control, pp. 116–23. Piscataway, NJ:: IEEE
     [Google Scholar]
  19. 19.
    Soroka BI, Andersson RL, Bajcsy R. 1981.. Generalised cylinders from local aggregation of sections. . Pattern Recognit. 13:(5):353–63
     [Crossref] [Google Scholar]
  20. 20.
    Mohr R, Bajcsy R. 1983.. Packing volumes by spheres. . IEEE Trans. Pattern Anal. Mach. Intell. 5:(1):111–16
     [Crossref] [Google Scholar]
  21. 21.
    Allen P. 1985.. Object recognition using vision and touch. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  22. 22.
    Lederman SJ, Klatzky RL. 1987.. Hand movements: a window into haptic object recognition. . Cogn. Psychol. 19:(3):342–68
     [Crossref] [Google Scholar]
  23. 23.
    Krotkov E. 1987.. Exploratory visual sensing for determining spatial layout with an agile stereo camera system. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  24. 24.
    Bajcsy R. 1988.. Active perception. . Proc. IEEE 76:(8):966–1005
     [Crossref] [Google Scholar]
  25. 25.
    Goldberg K, Bajcsy R. 1984.. Active touch and robot perception. . Cogn. Brain Theory 7:(2):199–216
     [Google Scholar]
  26. 26.
    Sears H. 2018.. Remembering an industry leader: Stephen C. Jacobsen, PhD. . O&P Edge. https://opedge.com/remembering-an-industry-leader-stephen-c-jacobsen-phd/
    [Google Scholar]
  27. 27.
    Harmon LD. 1984.. Tactile sensing for robots. . In Robotics and Artificial Intelligence, ed. M Brady, LA Gerhardt, HF Davidson , pp. 109–57. Berlin:: Springer
     [Google Scholar]
  28. 28.
    Lumelsky V, Shur M, Wagner S. 2001.. Sensitive skin. . IEEE Sens. J. 1:(1):41–51
     [Crossref] [Google Scholar]
  29. 29.
    Kageyama R, Kagami S, Inaba M, Inoue H. 1999.. Development of soft and distributed tactile sensors and the application to a humanoid robot. . In IEEE SMC ’99 Conference Proceedings: 1999 IEEE International Conference on Systems, Man, and Cybernetics, Vol. 2, pp. 981–86. Piscataway, NJ:: IEEE
     [Google Scholar]
  30. 30.
    Agrawal S, Bajcsy R, Kumar V. 1991.. A hand-eye-arm coordinated system. Tech. Rep. MS-CIS-91-05 , Dep. Comput. Inf. Sci., Univ. Pa., Philadelphia:
     [Google Scholar]
  31. 31.
    Sobh T. 1991.. Active observer: a discrete event dynamic system model for controlling an observer under uncertainty. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  32. 32.
    Sinha PR. 1991.. Robotic exploration of surfaces and its application to legged locomotion. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  33. 33.
    Barr AH. 1981.. Superquadrics and angle-preserving transformations. . IEEE Comput. Graph. Appl. 1:(1):11–23
     [Crossref] [Google Scholar]
  34. 34.
    Pentland AP. 1986.. Parts: structured descriptions of shape. . In Proceedings of the Fifth AAAI National Conference on Artificial Intelligence, pp. 695–701. Palo Alto, CA:: AAAI Press
     [Google Scholar]
  35. 35.
    Solina F, Bajcsy R. 1988.. Shape recovery of mail pieces using deformable models. Tech. Rep. MIS-CIS-88-57 , Dep. Comput. Inf. Sci., Univ. Pa., Philadelphia:
     [Google Scholar]
  36. 36.
    Leonardis A, Gupta A, Bajcsy R. 1995.. Segmentation of range images as the search for geometric parametric models. . Int. J. Comput. Vision 14::253–77
     [Crossref] [Google Scholar]
  37. 37.
    Gupta A. 1991.. Surface and volumetric segmentation of complex 3-D objects using parametric shape models. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  38. 38.
    Rosenthal DA, Bajcsy R. 1984.. Visual and conceptual hierarchy: a paradigm for studies of automated generation of recognition strategies. . IEEE Trans. Pattern Anal. Mach. Intell. 6:(3):319–25
     [Crossref] [Google Scholar]
  39. 39.
    Bajcsy R, Kovačič S. 1989.. Multiresolution elastic matching. . Comput. Vis. Graph. Image Process. 46:(1):1–21
     [Crossref] [Google Scholar]
  40. 40.
    Mallat S, Bajcsy R. 1989.. Image wavelet decomposition and applications. Tech. Rep. MIS-CIS-89-22 , Dep. Comput. Inf. Sci., Univ. Pa., Philadelphia:
     [Google Scholar]
  41. 41.
    Koivunen V, Bajcsy R. 1995.. Spline representations in 3-D vision. . In Object Representation in Computer Vision, ed. M Hebert, J Ponce, T Boult, A Gross , pp. 177–90. Berlin:: Springer
     [Google Scholar]
  42. 42.
    Mendelsohn J, Simoncelli E, Bajcsy R. 1997.. Discrete-time rigidity-constrained optical flow. . In Computer Analysis of Images and Patterns: 7th International Conference, CAIP ’97, ed. G Sommer, K, Daniilidis, J Pauli , pp. 255–62. Berlin:: Springer
     [Google Scholar]
  43. 43.
    Lee SW. 1991.. Understanding of surface reflections in computer vision by color and multiple views. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  44. 44.
    Weber A. 2008.. GM centennial: manufacturing innovation. . Assembly, June 30. https://www.assemblymag.com/articles/85863-gm-centennial-manufacturing-innovation
    [Google Scholar]
  45. 45.
    Funka-Lea G. 1994.. The visual recognition of shadows by an active observer. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  46. 46.
    Funka-Lea G. 1992.. A proposal concerning the analysis of shadows in images by an active observer. PhD Thesis Propos. , Univ. Pa., Philadelphia:
     [Google Scholar]
  47. 47.
    Tsikos CJ, Bajcsy R. 1991.. Segmentation via manipulation. . IEEE Trans. Robot. Autom. 7:(3):306–19
     [Crossref] [Google Scholar]
  48. 48.
    Bajcsy R, Campos M. 1991.. Active and exploratory perception. Tech. Rep . MIS-CIS-91-91 , Dep. Comput. Inf. Sci., Univ. Pa., Philadelphia:
     [Google Scholar]
  49. 49.
    Ramadge PJG, Wonham WM. 1989.. The control of discrete event systems. . Proc. IEEE 77:(1):81–98
     [Crossref] [Google Scholar]
  50. 50.
    Bogoni L, Bajcsy R. 1994.. Functionality investigation using a discrete event system approach. . Robot. Auton. Syst. 13:(3):173–96
     [Crossref] [Google Scholar]
  51. 51.
    Rosch E. 1978.. Principles of categorization. . In Cognition and Categorization, ed. E Rosch, BB Lloyd , pp. 27–48. Hillsdale, NJ:: Erlbaum
     [Google Scholar]
  52. 52.
    Bogoni L, Bajcsy R. 1995.. Interactive recognition and representation of functionality. . Comput. Vision Image Underst. 62:(2):194–214
     [Crossref] [Google Scholar]
  53. 53.
    Košecká J, Bajcsy R. 1994.. Discrete event systems for autonomous mobile agents. . Robot. Auton. Syst. 12::187–98
     [Crossref] [Google Scholar]
  54. 54.
    Adams J, Bajcsy R, Košecká J, Kumar V, Mintz M, et al. 1996.. Cooperative material handling by human and robotic agents: module development and system synthesis. . Expert Syst. Appl. 11:(2):89–97
     [Crossref] [Google Scholar]
  55. 55.
    Paljug E, Yun X, Kumar V. 1994.. Control of rolling contacts in multi-arm manipulation. . IEEE Trans. Robot. Autom. 10:(4):441–52
     [Crossref] [Google Scholar]
  56. 56.
    Bajcsy R, Chance P. 1975.. Picture registration application in biochemistry. Tech. Rep., Dep. Comput. Inf. Sci., Univ. Pa., Philadelphia:
     [Google Scholar]
  57. 57.
    Gee J, Reivich M, Bajcsy R. 1993.. Elastically deforming a three-dimensional atlas to match anatomical brain images. Tech. Rep. MS-CIS-93-53 , Dep. Comput. Sci., Univ. Pa., Philadelphia:
     [Google Scholar]
  58. 58.
    Broit C. 1981.. Optimal registration of deformed images. PhD Thesis , Univ. Pa., Philadelphia:
     [Google Scholar]
  59. 59.
    Bajcsy R, Lieberson R, Reivich M. 1983.. A computerized system for the elastic matching of deformed radiographic images to idealized atlas images. . J. Comput. Assist. Tomogr. 7:(4):618–25
     [Crossref] [Google Scholar]
  60. 60.
    Kakadiaris I, Metaxas D, Bajcsy R. 1997.. Inferring 2D object structure from the deformation of apparent contours. . Comput. Vis. Image Underst. 65:(2):129–47
     [Crossref] [Google Scholar]
  61. 61.
    Venetianer PL, Large EW, Bajcsy R. 1997.. A methodology for evaluation of task performance in robotic systems: a case study in vision-based localization. . Mach. Vis. Appl. 9::304–20
     [Crossref] [Google Scholar]
  62. 62.
    Sara R, Bajcsy R. 1997.. On occluding contour artifacts in stereo vision. . In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 852–57. Piscataway, NJ:: IEEE
     [Google Scholar]
  63. 63.
    Large EW, Christensen HI, Bajcsy R. 1999.. Scaling the dynamic approach to path planning and control: competition among behavioral constraints. . Int. J. Robot. Res. 18:(1):37–58
     [Crossref] [Google Scholar]
  64. 64.
    Beauchemin SS, Bajcsy R. 1988.. The role of theory in the evaluation of image motion algorithms. . In Performance Characterization in Computer Vision, ed. R Klette, HS Stiehl, MA Viergever, KL Vincken , pp. 55–67. Dordrecht, Neth:.: Springer
     [Google Scholar]
  65. 65.
    Salganicoff M. 1992.. A robotic system for learning visually-driven grasp planning. Tech. Rep. MS-CIS-92-27 , Univ. Pa., Philadelphia:
     [Google Scholar]
  66. 66.
    Colwell R, Bertsch McGrayne S. 2020.. A Lab of One's Own: One Woman's Personal Journey Through Sexism in Science. New York:: Simon & Schuster
     [Google Scholar]
  67. 67.
    Lederman SJ, Klatzky RL. 1993.. Extracting object properties through haptic exploration. . Acta Psychol. 84::29–40
     [Crossref] [Google Scholar]
  68. 68.
    Sinha PR, Bajcsy R. 1992.. Robotic exploration of surfaces and its application to legged locomotion. . In Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Vol. 1, pp. 221–26. Piscataway, NJ:: IEEE
     [Google Scholar]
/content/journals/10.1146/annurev-control-062723-111547
Loading
Robotics at the University of Pennsylvania from Birth to Maturity: A Review of 30 Years of Research
Annual Review of Control, Robotics, and Autonomous Systems 8, 49 (2025); https://doi.org/10.1146/annurev-control-062723-111547
/content/journals/10.1146/annurev-control-062723-111547
/content/journals/10.1146/annurev-control-062723-111547
Loading

Data & Media loading...

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error