Deutsch Intern
Media Informatics

Social Robots and Multimodal Storytelling

Storytelling is an important and long-established human cultural technique. Fields of application include the transfer of knowledge between generations or single persons, communication of social norms, values or warning of danger, storytelling in health and medicine, or entertainment.

Socially Interactive Agents (SIAs), with either a virtual or robotic embodiment, bear great potential as storytellers as they can enhance the storytelling experience though multiple channels. Our research addresses SIAs in storytelling scenarios from two sides: In controlled studies, we investigate how different multimodal behaviours, e.g. emotional facial display, affect the perception of human users. Based on finite state machines, we investigate how the SIAs’ behaviours can dynamically react to the interaction of the user, taking the current state of the story into account.    

Videos

Follow the white robot

Involved researchers

Prof. Dr. Birgit Lugrin

Sophia Steinhäußer (M.Sc.)

Related publications

2022[ to top ]
  • Effects of Colored LEDs in Robotic Storytelling on Storytelling Experience and Robot Perception. Steinhaeusser, Sophia C.; Lugrin, Birgit. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI2022). IEEE Press, 2022.
  • Designing Social Robots’ Speech in the Hotel Context-A Series of Online Studies. Steinhaeusser, Sophia C.; Lein, Martina; Donnermann, Melissa; Lugrin, Birgit. In 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 163–170. IEEE, 2022.
2021[ to top ]
  • The emotional robotic storyteller: On the influence of affect congruency on narrative transportation, robot perception, and persuasion. Appel, Markus; Lugrin, Birgit; Kühle, Mayla; Heindl, Corinna. In Computers in Human Behavior, p. 106749. Elsevier, 2021.
  • Comparing a Robotic Storyteller versus Audio Book with Integration of Sound Effects and Background Music. Steinhaeusser, Sophia C.; Schaper, Philipp; Lugrin, Birgit. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2021), C. Bethel, A. Paiva, E. Broadbent, D. Feil-Seifer, D. Szafir (eds.). ACM, 2021.
  • Anthropomorphize me! - Effects of Robot Gender on Listeners’ Perception of the Social Robot NAO in a Storytelling Use Case. Steinhaeusser, Sophia C.; Schaper, Philipp; Akuffo, Ohenewa Bediako; Friedrich, Paula; Ön, Jülide; Lugrin, Birgit. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2021), C. Bethel, A. Paiva, E. Broadbent, D. Feil-Seifer, D. Szafir (eds.). ACM, 2021.
  • Your New Friend NAO vs. Robot No. 783 - Effects of Personal or Impersonal Framing in a Robotic Storytelling Use Case. Steinhaeusser, Sophia C.; Gabel, Juliane J.; Lugrin, Birgit. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2021), C. Bethel, A. Paiva, E. Broadbent, D. Feil-Seifer, D. Szafir (eds.). ACM, 2021.
2019[ to top ]
  • Modeling and Evaluating Emotion, Contextual Head Movement and Voices for a Social Robot Storyteller. Striepe, Hendrik; Donnermann, Melissa; Lein, Martina; Lugrin, Birgit. In International Journal of Social Robotics. 2019.
  • Towards Narrative-Driven Atmosphere for Virtual Classroom. Lugrin, Jean-Luc; Latoschik, Marc Erich; Glémarec, Yann; Bosser, Anne-Gween; Chollet, Mathieu; Lugrin, Birgit. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–6. 2019.
2018[ to top ]
  • Follow the White Robot - A Role-Playing Game with a Robot Game Master. Fischbach, Martin; Brandt, Michael; Zimmerer, Chris; Lugrin, Jean-Luc; Latoschik, Marc Erich; Lugrin, Birgit. In 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2018), of AAMAS \u002718, pp. 1812–1814. ACM, Stockholm, Sweden, 2018.
2017[ to top ]
  • There Once Was a Robot Storyteller: Measuring the Effects of Emotion and Non-verbal Behaviour. Striepe, Hendrik; Lugrin, Birgit. In 9th International Conference on Social Robotics (ICSR 2017), of LNCS. Springer, 2017.
2014[ to top ]
  • Designing User-Character Dialog in Interactive Narratives: An Exploratory Experiment. Endrass, Birgit; Klimmt, Christoph; Mehlmann, Gregor; André, Elisabeth; Roth, Christian. In IEEE Transactions on Computational Intelligence and AI in Games, 6(2), pp. 166–173. 2014.
  • {Full Body Interaction with Virtual Characters in an Interactive Storytelling Scenario}. Kistler, Felix; Endrass, Birgit; André, Elisabeth. In 14th International Conference on Intelligent Virtual Agents (IVA 2014), Vol. 8637 of LNCS, T. W. Bickmore, S. Marsella, C. L. Sidner (eds.), pp. 236–239. Springer, 2014.
  • A Pictorial Interaction Language for Children to Communicate with Cultural Virtual Characters. Endrass, Birgit; Hall, Lynne E.; Hume, Colette; Tazzyman, Sarah; André, Elisabeth. In 16th International Conference on Human-Computer Interaction. Advanced Interaction Modalities and Techniques, Vol. 8511 of LNCS, M. Kurosu (ed.), pp. 532–543. Springer, 2014.
2011[ to top ]
  • Modeling Parallel State Charts for Multithreaded Multimodal Dialogues. Mehlmann, Gregor; Endrass, Birgit; André, Elisabeth. In 13th International Conference on Multimodal Interfaces (ICMI 2011), H. Bourlard, T. S. Huang, E. Vidal, D. Gatica-Perez, L.-P. Morency, N. Sebe (eds.), pp. 385–392. ACM, 2011.
  • Exploration of User Reactions to Different Dialog-Based Interaction Styles. Endrass, Birgit; Klimmt, Christoph; Mehlmann, Gregor; André, Elisabeth; Roth, Christian. In 4th International Conference on Interactive Digital Storytelling (ICIDS 2011), Vol. 7069 of LNCS, M. Si, D. Thue, E. André, J. C. Lester, J. Tanenbaum, V. Zammitto (eds.), pp. 243–248. Springer, 2011.
2009[ to top ]
  • What Would You Do in Their Shoes? Experiencing Different Perspectives in an Interactive Drama for Multiple Users. Endrass, Birgit; Boegler, Michael; Bee, Nikolaus; André, Elisabeth. In 2nd International Conference on Interactive Digital Storytelling (ICIDS 2009), Vol. 5915 of LNCS, I. Iurgel, N. Zagalo, P. Petta (eds.), pp. 258–268. Springer, 2009.
2007[ to top ]
  • Integrating the user in the social group dynamics of agents. Rehm, Matthias; Endrass, Birgit; Wissner, Michael. In Social Intelligence Design (SID). 2007.