1932

Abstract

An estimated 11% of adults report experiencing some form of cognitive decline, which may be associated with conditions such as stroke or dementia and can impact their memory, cognition, behavior, and physical abilities. While there are no known pharmacological treatments for many of these conditions, behavioral treatments such as cognitive training can prolong the independence of people with cognitive impairments. These treatments teach metacognitive strategies to compensate for memory difficulties in their everyday lives. Personalizing these treatments to suit the preferences and goals of an individual is critical to improving their engagement and sustainment, as well as maximizing the treatment's effectiveness. Robots have great potential to facilitate these training regimens and support people with cognitive impairments, their caregivers, and clinicians. This article examines how robots can adapt their behavior to be personalized to an individual in the context of cognitive neurorehabilitation. We provide an overview of existing robots being used to support neurorehabilitation and identify key principles for working in this space. We then examine state-of-the-art technical approaches for enabling longitudinal behavioral adaptation. To conclude, we discuss our recent work on enabling social robots to automatically adapt their behavior and explore open challenges for longitudinal behavior adaptation. This work will help guide the robotics community as it continues to provide more engaging, effective, and personalized interactions between people and robots.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-control-042920-093225
2022-05-03
2024-03-29
Loading full text...

Full text loading...

/deliver/fulltext/control/5/1/annurev-control-042920-093225.html?itemId=/content/journals/10.1146/annurev-control-042920-093225&mimeType=html&fmt=ahah

Literature Cited

  1. 1. 
    World Health Organ 2011. World report on disability 2011 Rep., World Health Organ. Geneva:
  2. 2. 
    Riek LD. 2017. Healthcare robotics. Commun. ACM 60:1168–78
    [Google Scholar]
  3. 3. 
    Moharana S, Panduro AE, Lee HR, Riek LD 2019. Robots for joy, robots for sorrow: community based robot design for dementia caregivers. 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI)458–67 Piscataway, NJ: IEEE
  4. 4. 
    Natl. Alliance Caregiv., AARP Public Policy Inst 2015. Caregiving in the U.S. Rep. Natl. Alliance Caregiv. and AARP Public Policy Inst. Washington, DC:
    [Google Scholar]
  5. 5. 
    World Health Organ 2015. World report on ageing and health Rep., World Health Organ. Geneva:
  6. 6. 
    Wada K, Shibata T, Musha T, Kimura S 2008. Robot therapy for elders affected by dementia. IEEE Eng. Med. Biol. Mag. 27:453–60
    [Google Scholar]
  7. 7. 
    Krebs H, Volpe B, Aisen M, Hogan N 2000. Increasing productivity and quality of care: robot-aided neuro-rehabilitation. J. Rehabil. Res. Dev. 37:639–52
    [Google Scholar]
  8. 8. 
    Semprini M, Laffranchi M, Sanguineti V, Avanzino L, De Icco R et al. 2018. Technological approaches for neurorehabilitation: from robotic devices to brain stimulation and beyond. Front. Neurol. 9:212
    [Google Scholar]
  9. 9. 
    Donati AR, Shokur S, Morya E, Campos DS, Moioli RC et al. 2016. Long-term training with a brain-machine interface-based gait protocol induces partial neurological recovery in paraplegic patients. Sci. Rep. 6:30383
    [Google Scholar]
  10. 10. 
    Pineau J, Montemerlo M, Pollack M, Roy N, Thrun S 2003. Towards robotic assistants in nursing homes: challenges and results. Robot. Auton. Syst. 42:271–81
    [Google Scholar]
  11. 11. 
    Pino O, Palestra G, Trevino R, De Carolis B. 2020. The humanoid robot NAO as trainer in a memory program for elderly people with mild cognitive impairment. Int. J. Soc. Robot. 12:21–33
    [Google Scholar]
  12. 12. 
    Riek LD 2016. Robotics technology in mental health care. Artificial Intelligence in Behavioral and Mental Health Care DD Luxton 185–203 London: Academic
    [Google Scholar]
  13. 13. 
    Stefano M, Patrizia P, Mario A, Ferlini G, Rizzello R, Rosati G 2014. Robotic upper limb rehabilitation after acute stroke by NeReBot: evaluation of treatment costs. BioMed Res. Int. 2014:265634
    [Google Scholar]
  14. 14. 
    Guan C, Bouzida A, Oncy-Avila R, Moharana S, Riek LD 2021. Taking an (embodied) cue from community health: designing dementia caregiver support technology to advance health equity. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems655 New York: ACM
  15. 15. 
    Tapus A, Ţăpuş C, Matarić MJ. 2009. The use of socially assistive robots in the design of intelligent cognitive therapies for people with dementia. 2009 IEEE International Conference on Rehabilitation Robotics924–29 Piscataway, NJ: IEEE
  16. 16. 
    Graf B, Hans M, Schraft RD 2004. Care-O-bot II—development of a next generation robotic home assistant. Auton. Robots 16:193–205
    [Google Scholar]
  17. 17. 
    Kittmann R, Fröhlich T, Schäfer J, Reiser U, Weißhardt F, Haug A 2015. Let me introduce myself: I am Care-O-bot 4, a gentleman robot. Mensch und Computer 2015 Tagungsband S Diefenbach, N Henze, M Pielot 223–32 Stuttgart, Ger: Oldenbourg
    [Google Scholar]
  18. 18. 
    Asprino L, Gangemi A, Nuzzolese AG, Presutti V, Recupero DR, Russo A 2019. Ontology-based knowledge management for comprehensive geriatric assessment and reminiscence therapy on social robots. Data Science for Healthcare: Methodologies and Applications S Consoli, DR Recupero, M Petković 173–93 Cham, Switz: Springer
    [Google Scholar]
  19. 19. 
    Kubota A, Peterson EI, Rajendren V, Kress-Gazit H, Riek LD. 2020. JESSIE: synthesizing social robot behaviors for personalized neurorehabilitation and beyond. Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction121–30 New York: ACM
  20. 20. 
    Clabaugh C, Matarić MJ. 2018. Robots for the people, by the people: personalizing human-machine interaction. Sci. Robot. 3:eaat7451
    [Google Scholar]
  21. 21. 
    Moyle W, Jones CJ, Murfield JE, Thalib L, Beattie ER et al. 2017. Use of a robotic seal as a therapeutic tool to improve dementia symptoms: a cluster-randomized controlled trial. J. Am. Med. Dir. Assoc. 18:766–73
    [Google Scholar]
  22. 22. 
    Geva N, Uzefovsky F, Levy-Tzedek S. 2020. Touching the social robot PARO reduces pain perception and salivary oxytocin levels. Sci. Rep. 10:9814
    [Google Scholar]
  23. 23. 
    Tapus A, Ţăpuş C, Matarić M. 2009. The role of physical embodiment of a therapist robot for individuals with cognitive impairments. RO-MAN 2009: The 18th IEEE International Symposium on Robot and Human Interactive Communication103–7 Piscataway, NJ: IEEE
  24. 24. 
    Cruz-Sandoval D, Morales-Tellez A, Sandoval EB, Favela J. 2020. A social robot as therapy facilitator in interventions to deal with dementia-related behavioral symptoms. Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction161–69 New York: ACM
  25. 25. 
    Barman A, Chatterjee A, Bhide R 2016. Cognitive impairment and rehabilitation strategies after traumatic brain injury. Indian J. Psychol. Med. 38:172–81
    [Google Scholar]
  26. 26. 
    Huckans M, Hutson L, Twamley E, Jak A, Kaye J, Storzbach D 2013. Efficacy of cognitive rehabilitation therapies for mild cognitive impairment (MCI) in older adults: working toward a theoretical model and evidence-based interventions. Neuropsychol. Rev. 23:63–80
    [Google Scholar]
  27. 27. 
    Lopresti EF, Mihailidis A, Kirsch N 2004. Assistive technology for cognitive rehabilitation: state of the art. Neuropsychol. Rehabil. 14:5–39
    [Google Scholar]
  28. 28. 
    Chan J, Nejat G 2012. Social intelligence for a robot engaging people in cognitive training activities. Int. J. Adv. Robot. Syst. 9:113
    [Google Scholar]
  29. 29. 
    Tsiakas K, Abujelala M, Makedon F. 2018. Task engagement as personalization feedback for socially-assistive robots and cognitive training. Technologies 6:49
    [Google Scholar]
  30. 30. 
    Woodworth B, Ferrari F, Zosa TE, Riek LD 2018. Preference learning in assistive robotics: observational repeated inverse reinforcement learning. Proceedings of the 3rd Machine Learning for Healthcare Conference F Doshi-Velez, J Fackler, K Jung, D Kale, R Ranganath, et al. 420–39 Proc. Mach. Learn. Res. 85. N.p.: PMLR
  31. 31. 
    Rossi S, Ferland F, Tapus A. 2017. User profiling and behavioral adaptation for HRI: a survey. Pattern Recognit. Lett. 99:3–12
    [Google Scholar]
  32. 32. 
    Wang L, Rau PLP, Evers V, Robinson BK, Hinds P 2010. When in Rome: the role of culture & context in adherence to robot recommendations. 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI)359–66 Piscataway, NJ: IEEE
  33. 33. 
    Tapus A, Ţăpuş C, Matarić MJ. 2008. User—robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy. Intell. Serv. Robot. 1:169
    [Google Scholar]
  34. 34. 
    World Health Organ 2017. Global action plan on the public health response to dementia 2017–25 Rep., World Health Organ. Geneva:
  35. 35. 
    Teng E, Tassniyom K, Lu PH 2012. Reduced quality-of-life ratings in mild cognitive impairment: analyses of subject and informant responses. Am. J. Geriatr. Psychiatry 20:1016–25
    [Google Scholar]
  36. 36. 
    Petersen RC. 2004. Mild cognitive impairment as a diagnostic entity. J. Intern. Med. 256:183–94
    [Google Scholar]
  37. 37. 
    Bruscoli M, Lovestone S. 2004. Is MCI really just early dementia? A systematic review of conversion studies. Int. Psychogeriatr. 16:129–40
    [Google Scholar]
  38. 38. 
    Arnáiz E, Almkvist O 2003. Neuropsychological features of mild cognitive impairment and preclinical Alzheimer's disease. Acta Neurol. Scand. 107:34–41
    [Google Scholar]
  39. 39. 
    Fleming JM, Shum D, Strong J, Lightbody S. 2005. Prospective memory rehabilitation for adults with traumatic brain injury: a compensatory training programme. Brain Injury 19:1–10
    [Google Scholar]
  40. 40. 
    Bahar-Fuchs A, Clare L, Woods B 2013. Cognitive training and cognitive rehabilitation for persons with mild to moderate dementia of the Alzheimer's or vascular type: a review. Alzheimer's Res. Ther. 5:35
    [Google Scholar]
  41. 41. 
    Garand L, Dew MA, Eazor LR, DeKosky ST, Reynolds CF 2005. Caregiving burden and psychiatric morbidity in spouses of persons with mild cognitive impairment. Int. J. Geriatr. Psychiatry 20:512–22
    [Google Scholar]
  42. 42. 
    Dixon E, Lazar A. 2020. Approach matters: linking practitioner approaches to technology design for people with dementia. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems New York: ACM. https://doi.org/10.1145/3313831.3376432
  43. 43. 
    Poulos CJ, Bayer A, Beaupre L, Clare L, Poulos RG et al. 2017. A comprehensive approach to reablement in dementia. Alzheimer's Dement. Transl. Res. Clin. Interv. 3:450–58
    [Google Scholar]
  44. 44. 
    Clare L, van Paasschen J, Evans SJ, Parkinson C, Woods RT, Linden DE. 2009. Goal-oriented cognitive rehabilitation for an individual with mild cognitive impairment: behavioural and neuroimaging outcomes. Neurocase 15:318–31
    [Google Scholar]
  45. 45. 
    Lehmann H, Iacono I, Robins B, Marti P, Dautenhahn K 2011. Make it move’: playing cause and effect games with a robot companion for children with cognitive disabilities. Proceedings of the 29th Annual European Conference on Cognitive Ergonomics105–12 New York: ACM
  46. 46. 
    Robins B, Dautenhahn K, Te Boekhorst R, Billard A 2005. Robotic assistants in therapy and education of children with autism: Can a small humanoid robot help encourage social interaction skills?. Univers. Access Inf. Soc. 4:105–20
    [Google Scholar]
  47. 47. 
    Scassellati B 2007. How social robots will help us to diagnose, treat, and understand autism. Robotics Research: Results of the 12th International Symposium ISRR S Thrun, R Brooks, H Durrant-Whyte 552–63 Berlin: Springer
  48. 48. 
    Kim ES, Berkovits LD, Bernier EP, Leyzberg D, Shic F et al. 2013. Social robots as embedded reinforcers of social behavior in children with autism. J. Autism Dev. Disord. 43:1038–49
    [Google Scholar]
  49. 49. 
    Cabibihan JJ, Javed H, Ang M, Aljunied SM 2013. Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. Int. J. Soc. Robot. 5:593–618
    [Google Scholar]
  50. 50. 
    Ujike S, Yasuhara Y, Osaka K, Sato M, Catangui E et al. 2019. Encounter of Pepper-CPGE for the elderly and patients with schizophrenia: an innovative strategy to improve patient's recreation, rehabilitation, and communication. J. Med. Investig. 66:50–53
    [Google Scholar]
  51. 51. 
    Raffard S, Bortolon C, Khoramshahi M, Salesse RN, Burca M et al. 2016. Humanoid robots versus humans: How is emotional valence of facial expressions recognized by individuals with schizophrenia? An exploratory study. Schizophr. Res. 176:506–13
    [Google Scholar]
  52. 52. 
    Giannopulu I. 2013. Multimodal cognitive nonverbal and verbal interactions: the neurorehabilitation of autistic children via mobile toy robots. IARIA Int. J. Adv. Life Sci. 5:214–22
    [Google Scholar]
  53. 53. 
    Costa S, Santos C, Soares F, Ferreira M, Moreira F. 2010. Promoting interaction amongst autistic adolescents using robots. 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology3856–59 Piscataway, NJ: IEEE
  54. 54. 
    Eriksson J, Matarić MJ, Winstein CJ. 2005. Hands-off assistive robotics for post-stroke arm rehabilitation. 9th International Conference on Rehabilitation Robotics, 200521–24 Piscataway, NJ: IEEE
  55. 55. 
    Deng E, Mutlu B, Matarić MJ. 2019. Embodiment in socially interactive robots. Found. Trends Robot. 7:251–356
    [Google Scholar]
  56. 56. 
    Kidd CD, Breazeal C. 2008. Robots at home: understanding long-term human-robot interaction. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems3230–35 Piscataway, NJ: IEEE
  57. 57. 
    Moyle W, Jones C, Cooke M, O'Dwyer S, Sung B, Drummond S 2014. Connecting the person with dementia and family: a feasibility study of a telepresence robot. BMC Geriatr 14:7
    [Google Scholar]
  58. 58. 
    Gross HM, Schroeter C, Mueller S, Volkhardt M, Einhorn E et al. 2011. Progress in developing a socially assistive mobile home robot companion for the elderly with mild cognitive impairment. 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems2430–37 Piscataway, NJ: IEEE
  59. 59. 
    Tamura T, Yonemitsu S, Itoh A, Oikawa D, Kawakami A et al. 2004. Is an entertainment robot useful in the care of elderly people with severe dementia?. J. Gerontol. A 59:M83–85
    [Google Scholar]
  60. 60. 
    Yamazaki R, Kochi M, Zhu W, Kase H 2018. A pilot study of robot reminiscence in dementia care. Int. J. Med. Health Biomed. Bioeng. Pharm. Eng. 12:253–57
    [Google Scholar]
  61. 61. 
    Libin A, Cohen-Mansfield J. 2004. Therapeutic robocat for nursing home residents with dementia: preliminary inquiry. Am. J. Alzheimer's Dis. Other Dement. 19:111–16
    [Google Scholar]
  62. 62. 
    Chu MT, Khosla R, Khaksar SMS, Nguyen K. 2017. Service innovation through social robot engagement to improve dementia care quality. Assist. Technol. 29:8–18
    [Google Scholar]
  63. 63. 
    Tao C, Han R, Huang J, Wang X, Ma L 2015. Development and experiment study of an intelligent walking-aid robot. Int. J. Model. Identif. Control 24:216–23
    [Google Scholar]
  64. 64. 
    Chang WL, Šabanovic S, Huber L 2013. Use of seal-like robot PARO in sensory group therapy for older adults with dementia. 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI)101–2 Piscataway, NJ: IEEE
  65. 65. 
    Šabanović S, Bennett CC, Chang WL, Huber L 2013. PARO robot affects diverse interaction modalities in group sensory therapy for older adults with dementia. 2013 IEEE 13th International Conference on Rehabilitation Robotics Piscataway, NJ: IEEE. https://doi.org/10.1109/ICORR.2013.6650427
  66. 66. 
    Woods B, O'Philbin L, Farrell EM, Spector AE, Orrell M. 2018. Reminiscence therapy for dementia. Cochrane Database Syst. Rev. 3:CD001120
    [Google Scholar]
  67. 67. 
    Wobbrock JO, Kane SK, Gajos KZ, Harada S, Froehlich J. 2011. Ability-based design: concept, principles and examples. ACM Trans. Access. Comput. 3:9
    [Google Scholar]
  68. 68. 
    Morris MR. 2020. AI and accessibility. Commun. ACM 63:635–37
    [Google Scholar]
  69. 69. 
    Whittaker M, Alper M, Bennett CL, Hendren S, Kaziunas L et al. 2019. Disability, bias, and AI. Rep., AI Now Inst. N.Y. Univ. New York:
    [Google Scholar]
  70. 70. 
    Holthe T, Halvorsrud L, Karterud D, Hoel KA, Lund A. 2018. Usability and acceptability of technology for community-dwelling older adults with mild cognitive impairment and dementia: a systematic literature review. Clin. Interv. Aging 13:863–86
    [Google Scholar]
  71. 71. 
    Small GW, Rabins PV, Barry PP, Buckholtz NS, DeKosky ST et al. 1997. Diagnosis and treatment of Alzheimer disease and related disorders: consensus statement of the American Association for Geriatric Psychiatry, the Alzheimer's Association, and the American Geriatrics Society. JAMA 278:1363–71
    [Google Scholar]
  72. 72. 
    Rocca WA, Boyd CM, Grossardt BR, Bobo WV, Rutten LJF et al. 2014. Prevalence of multimorbidity in a geographically defined American population: patterns by age, sex, and race/ethnicity. Mayo Clin. Proc. 89:1336–49
    [Google Scholar]
  73. 73. 
    Heerink M. 2011. How elderly users of a socially interactive robot experience adaptiveness, adaptability and user control. 2011 IEEE 12th International Symposium on Computational Intelligence and Informatics (CINTI)79–84 Piscataway, NJ: IEEE
  74. 74. 
    Martins GS, Al Tair H, Santos L, Dias J 2019. αPOMDP: POMDP-based user-adaptive decision-making for social robots. Pattern Recognit. Lett. 118:94–103
    [Google Scholar]
  75. 75. 
    Charlton JI. 2000. Nothing About Us Without Us: Disability Oppression and Empowerment Berkeley: Univ. Calif. Press
  76. 76. 
    Lee HR, Riek LD. 2018. Reframing assistive robots to promote successful aging. ACM Trans. Human-Robot Interact. 7:11
    [Google Scholar]
  77. 77. 
    Van Wynsberghe A. 2013. Designing robots for care: care centered value-sensitive design. Sci. Eng. Ethics 19:407–33
    [Google Scholar]
  78. 78. 
    Castellano G, Aylett R, Dautenhahn K, Paiva A, McOwan PW, Ho S. 2008. Long-term affect sensitive and socially interactive companions. Paper presented at the 4th International Workshop on Human-Computer Conversation Oxford, UK:, Oct. 6–7
    [Google Scholar]
  79. 79. 
    Nahum-Shani I, Hekler EB, Spruijt-Metz D 2015. Building health behavior models to guide the development of just-in-time adaptive interventions: a pragmatic framework. Health Psychol 34:Suppl.1209–19
    [Google Scholar]
  80. 80. 
    Fong T, Nourbakhsh I, Dautenhahn K 2003. A survey of socially interactive robots. Robot. Auton. Syst. 42:143–66
    [Google Scholar]
  81. 81. 
    Szafir D, Mutlu B. 2012. Pay attention! Designing adaptive agents that monitor and improve user engagement. CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems11–20 New York: ACM
  82. 82. 
    Duffy BR. 2003. Anthropomorphism and the social robot. Robot. Auton. Syst. 42):177–90
    [Google Scholar]
  83. 83. 
    Martin JN, Nakayama TK. 2013. Intercultural Communication in Contexts New York: McGraw-Hill, 6th ed..
  84. 84. 
    Foukarakis M, Leonidis A, Antona M, Stephanidis C 2014. Combining finite state machine and decision-making tools for adaptable robot behavior. International Conference on Universal Access in Human-Computer Interaction: Aging and Assistive Environments C Stephanidis, M Antona 625–35 Cham, Switz: Springer
  85. 85. 
    Meng Q, Wu W. 2008. Artificial emotional model based on finite state machine. J. Central South Univ. Technol. 15:694–99
    [Google Scholar]
  86. 86. 
    Lee MH, Siewiorek DP, Smailagic A, Bernardino A, Bermúdez i Badia S. 2020. An exploratory study on techniques for quantitative assessment of stroke rehabilitation exercises. Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization303–7 New York: ACM
  87. 87. 
    Liu C, Conn K, Sarkar N, Stone W. 2008. Online affect detection and robot behavior adaptation for intervention of children with autism. IEEE Trans. Robot. 24:883–96
    [Google Scholar]
  88. 88. 
    Ritschel H, André E. 2017. Real-time robot personality adaptation based on reinforcement learning and social signals. Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction265–66 New York: ACM
  89. 89. 
    Lockerd A, Breazeal C. 2004. Tutelage and socially guided robot learning. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 43475–80 Piscataway, NJ: IEEE
  90. 90. 
    Prommer T, Holzapfel H, Waibel A. 2006. Rapid simulation-driven reinforcement learning of multimodal dialog strategies in human-robot interaction. INTERSPEECH 2006: Ninth International Conference on Spoken Language Processing1918–21 Baixas, Fr.: Int. Speech Commun. Assoc.
  91. 91. 
    Krsmanovic F, Spencer C, Jurafsky D, Ng AY 2006. Have we met? MDP based speaker ID for robot dialogue. INTERSPEECH 2006: Ninth International Conference on Spoken Language Processing461–64 Baixas, Fr.: Int. Speech Commun. Assoc.
  92. 92. 
    Malfaz M, Castro-González Á, Barber R, Salichs MA. 2011. A biologically inspired architecture for an autonomous and social robot. IEEE Trans. Auton. Ment. Dev. 3:232–46
    [Google Scholar]
  93. 93. 
    Ferreira E, Lefevre F. 2015. Reinforcement-learning based dialogue system for human–robot interactions with socially-inspired rewards. Comput. Speech Lang. 34:256–74
    [Google Scholar]
  94. 94. 
    Karami AB, Sehaba K, Encelle B. 2016. Adaptive artificial companions learning from users' feedback. Adapt. Behav. 24:69–86
    [Google Scholar]
  95. 95. 
    Chen H, Park HW, Breazeal C. 2020. Teaching and learning with children: impact of reciprocal peer learning with a social robot on children's learning and emotive engagement. Comput. Educ. 150:103836
    [Google Scholar]
  96. 96. 
    Park HW, Grover I, Spaulding S, Gomez L, Breazeal C. 2019. A model-free affective reinforcement learning approach to personalization of an autonomous social robot companion for early literacy education. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence687–94 Palo Alto, CA: AAAI Press
  97. 97. 
    Gao AY, Barendregt W, Castellano G. 2017. Personalised human-robot co-adaptation in instructional settings using reinforcement learning. Paper presented at the Workshop on Persuasive Embodied Agents for Behavior Change. International Conference on Intelligent Virtual Agents Stockholm:, Aug. 27
    [Google Scholar]
  98. 98. 
    Biancardi B, Mancini M, Lerner P, Pelachaud C. 2019. Managing an agent's self-presentational strategies during an interaction. Front. Robot. AI 6:93
    [Google Scholar]
  99. 99. 
    Gordon G, Spaulding S, Westlund JK, Lee JJ, Plummer L et al. 2016. Affective personalization of a social robot tutor for children's second language skills. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence3951–57 Palo Alto, CA: AAAI Press
  100. 100. 
    Taha T, Miró JV, Dissanayake G. 2011. A POMDP framework for modelling human interaction with assistive robots. 2011 IEEE International Conference on Robotics and Automation544–49 Piscataway, NJ: IEEE
  101. 101. 
    Karami AB, Jeanpierre L, Mouaddib AI 2009. Partially observable Markov decision process for managing robot collaboration with human. 2009 21st IEEE International Conference on Tools with Artificial Intelligence518–21 Piscataway, NJ: IEEE
  102. 102. 
    Spaulding S, Breazeal C. 2017. Learning behavior policies for interactive educational play. Paper presented at the Workshop on Mathematical Models, Algorithms, and Human-Robot Interaction, Robotics: Science and Systems XIII Cambridge, MA: July 12–16
    [Google Scholar]
  103. 103. 
    Hoey J, von Bertoldi A, Poupart P, Mihailidis A 2007. Assisting persons with dementia during handwashing using a partially observable Markov decision process. Proceedings of the 5th International Conference on Computer Vision Systems Bielefeld, Ger.: Appl. Comput. Sci. Group, Bielefeld Univ. https://doi.org/10.2390/biecoll-icvs2007-89
  104. 104. 
    Hemminahaus J, Kopp S. 2017. Towards adaptive social behavior generation for assistive robots using reinforcement learning. 2017 12th ACM/IEEE International Conference on Human-Robot Interaction332–40 Piscataway, NJ: IEEE
  105. 105. 
    Belpaeme T, Baxter P, Read R, Wood R, Cuayáhuitl H et al. 2013. Multimodal child-robot interaction: building social bonds. J. Hum.-Robot Int. 1:33–53
    [Google Scholar]
  106. 106. 
    Mitsunaga N, Smith C, Kanda T, Ishiguro H, Hagita N. 2006. Robot behavior adaptation for human-robot interaction based on policy gradient reinforcement learning. J. Robot. Soc. Jpn. 24:820–29
    [Google Scholar]
  107. 107. 
    Chandramohan S, Geist M, Lefevre F, Pietquin O. 2011. User simulation in dialogue systems using inverse reinforcement learning. INTERSPEECH 2011: 12th Annual Conference of the International Speech Communication Association1025–28 Baixas, Fr.: Int. Speech Commun. Assoc.
  108. 108. 
    Boularias A, Chinaei HR, Chaib-draa B. 2010. Learning the reward model of dialogue pomdps from data. Paper presented at the Workshop on Machine Learning for Assistive Techniques, 24th Conference on Neural Information Processing Systems Vancouver, Can: Dec. 6–11
    [Google Scholar]
  109. 109. 
    Sugiyama H, Meguro T, Minami Y. 2012. Preference-learning based inverse reinforcement learning for dialog control. INTERSPEECH 2012: 13th Annual Conference of the International Speech Communication Association222–25 Baixas, Fr.: Int. Speech Commun. Assoc.
  110. 110. 
    Senft E, Baxter P, Kennedy J, Belpaeme T 2015. SPARC: Supervised Progressively Autonomous Robot Competencies. International Conference on Social Robotics A Tapus, E André, JC Martin, F Ferland, M Ammi 603–12 Cham, Switz: Springer
  111. 111. 
    Dermouche S, Pelachaud C. 2019. Generative model of agent's behaviors in human-agent interaction. 2019 International Conference on Multimodal Interaction375–84 New York: ACM
  112. 112. 
    Yan H, Ang MH, Poo AN. 2014. A survey on perception methods for human–robot interaction in social robots. Int. J. Soc. Robot. 6:85–119
    [Google Scholar]
  113. 113. 
    Pantic M, Pentland A, Nijholt A, Huang TS 2007. Human computing and machine understanding of human behavior: a survey. Artifical Intelligence for Human Computing TS Huang, A Nijholt, M Pantic, A Pentland 44–71 Berlin: Springer
    [Google Scholar]
  114. 114. 
    Bohren J, Cousins S. 2010. The SMACH high-level executive. IEEE Robot. Autom. Mag. 17:418–20
    [Google Scholar]
  115. 115. 
    Sutton RS, Barto AG. 1998. Reinforcement Learning: An Introduction Cambridge, MA: MIT Press
  116. 116. 
    Wang Z, Singh MK, Zhang C, Riek LD, Chaudhuri K. 2020. Stochastic multi-player bandit learning from player-dependent feedback. Paper presented at the Workshop on Real World Experiment Design and Active Learning. 37th International Conference on Machine Learning, virtual July 18
    [Google Scholar]
  117. 117. 
    Auer P, Cesa-Bianchi N, Freund Y, Schapire RE. 2002. The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32:48–77
    [Google Scholar]
  118. 118. 
    Sutton RS, McAllester DA, Singh SP, Mansour Y 2000. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems 12 S Solla, T Leen, KM Müller 1057–63 Cambridge, MA: MIT Press
    [Google Scholar]
  119. 119. 
    Smallwood RD, Sondik EJ. 1973. The optimal control of partially observable Markov processes over a finite horizon. Oper. Res. 21:1071–88
    [Google Scholar]
  120. 120. 
    Dietterich TG. 2000. Hierarchical reinforcement learning with the MAXQ value function decomposition. J. Artif. Intell. Res. 13:227–303
    [Google Scholar]
  121. 121. 
    Janecek A, Gansterer W, Demel M, Ecker G 2008. On the relationship between feature selection and classification accuracy. Proceedings of the Workshop on New Challenges for Feature Selection in Data Mining and Knowledge Discovery at ECML/PKDD 2008 Y Saeys, H Liu, I Inza, L Wehenkel, Y Van de Pee 90–105 Proc. Mach. Learn. Res. 4. N.p.: PMLR
  122. 122. 
    Gardner MW, Dorling S. 1998. Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences. Atmos. Environ. 32:2627–36
    [Google Scholar]
  123. 123. 
    Hochreiter S, Schmidhuber J. 1997. Long short-term memory. Neural Comput 9:1735–80
    [Google Scholar]
  124. 124. 
    O'Connor MF, Riek LD. 2015. Detecting social context: a method for social event classification using naturalistic multimodal data. 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) Piscataway, NJ: IEEE. https://doi.org/10.1109/FG.2015.7284843
  125. 125. 
    Nigam A, Riek LD. 2015. Social context perception for mobile robots. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)3621–27 Piscataway, NJ: IEEE
  126. 126. 
    Dautenhahn K, Nehaniv CL, Walters ML, Robins B, Kose-Bagci H et al. 2009. Kaspar – a minimally expressive humanoid robot for human–robot interaction research. Appl. Bionics Biomech. 6:369–97
    [Google Scholar]
  127. 127. 
    Sung JY, Guo L, Grinter RE, Christensen HI 2007.. “ My Roomba is Rambo”: intimate home appliances. UbiComp 2007: Ubiquitous Computing J Krumm, GD Abowd, A Seneviratne, T Strang 145–62 Berlin: Springer
    [Google Scholar]
  128. 128. 
    Forlizzi J, DiSalvo C. 2006. Service robots in the domestic environment: a study of the Roomba vacuum in the home. Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction258–65 New York: ACM
  129. 129. 
    Carpenter J. 2016. Culture and Human-Robot Interaction in Militarized Spaces: A War Story London: Routledge
  130. 130. 
    Riek L, Howard D. 2014. A code of ethics for the human-robot interaction profession Paper presented at We Robot Coral Gables, FL: Apr. 4–5
  131. 131. 
    Kubota A, Pourebadi M, Banh S, Kim S, Riek LD 2021. Somebody that I used to know: the risks of personalizing robots for dementia care. Paper presented at We Robot, Coral Gables, FL, Sept. 23–25
  132. 132. 
    Wang Z, Zhang C, Singh MK, Riek L, Chaudhuri K 2021. Multitask bandit learning through heterogeneous feedback aggregation. Proceedings of the 24th International Conference on Artificial Intelligence and Statistics A Banerjee, K Fukumizu 1531–39 Proc. Mach. Learn. Res. 130. N.p. PMLR
  133. 133. 
    Auer P, Cesa-Bianchi N, Fischer P. 2002. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47:235–56
    [Google Scholar]
/content/journals/10.1146/annurev-control-042920-093225
Loading
/content/journals/10.1146/annurev-control-042920-093225
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error