Publications

• Reflected Reality: A Mixed Reality Knowledge Representation for Context-aware Systems Eckstein, Benjamin; Niebling, Florian; Lugrin, Birgit in 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games) (2019).

  accepted for publication
  • [ BibTeX ]
  • [ EndNote ]
  @conference{eckstein2019reflected, author = {Eckstein, Benjamin and Niebling, Florian and Lugrin, Birgit}, booktitle = {11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games)}, keywords = {mirrorworld}, note = {accepted for publication}, title = {Reflected Reality: A Mixed Reality Knowledge Representation for Context-aware Systems}, year = 2019 }

  %0 Generic %1 eckstein2019reflected %A Eckstein, Benjamin %A Niebling, Florian %A Lugrin, Birgit %B 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games) %D 2019 %T Reflected Reality: A Mixed Reality Knowledge Representation for Context-aware Systems
• Smart substitutional reality: Integrating the smart home into virtual reality Eckstein, Benjamin; Krapp, Eva; Elsässer, Anne; Lugrin, Birgit in Entertainment Computing (2019). 31 100306.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  Recent advances in VR technology allow users to consume immersive content within their own living room. Substitutional Reality (SR) promises to enhance this experience by integrating the physical environment into the simulation. We propose a novel approach, called Smart Substitutional Reality (SSR), that extends the passive haptics of SR with the interactive functionality of a smart home environment. SSR is meant to serve as a foundation for serious games. In this paper, we describe the concept of SSR alongside the implementation of a prototype in our smart lab. We designed multiple virtual environments with a varying degree of mismatch compared to the real world, and added selected objects to induce additional haptic and thermal stimuli to increase immersion. In two user studies we investigate their impact on spatial presence and intrinsic motivation which are especially important for serious games. Results suggest that spatial presence is maintained with higher mismatch, and increased by inducing additional stimuli. Intrinsic motivation is increased in both cases.

  @article{ECKSTEIN2019100306, abstract = {Recent advances in VR technology allow users to consume immersive content within their own living room. Substitutional Reality (SR) promises to enhance this experience by integrating the physical environment into the simulation. We propose a novel approach, called Smart Substitutional Reality (SSR), that extends the passive haptics of SR with the interactive functionality of a smart home environment. SSR is meant to serve as a foundation for serious games. In this paper, we describe the concept of SSR alongside the implementation of a prototype in our smart lab. We designed multiple virtual environments with a varying degree of mismatch compared to the real world, and added selected objects to induce additional haptic and thermal stimuli to increase immersion. In two user studies we investigate their impact on spatial presence and intrinsic motivation which are especially important for serious games. Results suggest that spatial presence is maintained with higher mismatch, and increased by inducing additional stimuli. Intrinsic motivation is increased in both cases.}, author = {Eckstein, Benjamin and Krapp, Eva and Elsässer, Anne and Lugrin, Birgit}, journal = {Entertainment Computing}, keywords = {Exteroceptive}, pages = 100306, title = {Smart substitutional reality: Integrating the smart home into virtual reality}, volume = 31, year = 2019 }

  %0 Journal Article %1 ECKSTEIN2019100306 %A Eckstein, Benjamin %A Krapp, Eva %A Elsässer, Anne %A Lugrin, Birgit %D 2019 %J Entertainment Computing %P 100306 %R https://doi.org/10.1016/j.entcom.2019.100306 %T Smart substitutional reality: Integrating the smart home into virtual reality %U http://www.sciencedirect.com/science/article/pii/S1875952119300151 %V 31 %X Recent advances in VR technology allow users to consume immersive content within their own living room. Substitutional Reality (SR) promises to enhance this experience by integrating the physical environment into the simulation. We propose a novel approach, called Smart Substitutional Reality (SSR), that extends the passive haptics of SR with the interactive functionality of a smart home environment. SSR is meant to serve as a foundation for serious games. In this paper, we describe the concept of SSR alongside the implementation of a prototype in our smart lab. We designed multiple virtual environments with a varying degree of mismatch compared to the real world, and added selected objects to induce additional haptic and thermal stimuli to increase immersion. In two user studies we investigate their impact on spatial presence and intrinsic motivation which are especially important for serious games. Results suggest that spatial presence is maintained with higher mismatch, and increased by inducing additional stimuli. Intrinsic motivation is increased in both cases.

• Adapted Foreigner-directed Communication towards Virtual Agents Lugrin, Birgit; Eckstein, Benjamin; Bergmann, Kirsten; Heindl, Corinna in International Conference on Intelligent Virtual Agents (2018).

  in press
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  People tend to adapt to their conversation partners. In mixed-cultural settings, with a native and a non-native speaker, adaptation can manifest itself in the usage of simplified language or increased usage of non-verbal scaffolding to foster understanding. In this contribution, we address the question whether the phenomenon of AFC (adapted, foreigner-directed communication) is also shown towards a virtual agent. We therefore implemented a demonstrator with a local and a foreign agent that interact with human users in a direction giving scenario. A user study revealed that participants behaved differently towards the two agents and adapted both, their verbal and their non-verbal behaviour. We are thus confident, that our demonstrator is well suited to systematically study AFC.

  @inproceedings{lugrin2018adapted, abstract = {People tend to adapt to their conversation partners. In mixed-cultural settings, with a native and a non-native speaker, adaptation can manifest itself in the usage of simplified language or increased usage of non-verbal scaffolding to foster understanding. In this contribution, we address the question whether the phenomenon of AFC (adapted, foreigner-directed communication) is also shown towards a virtual agent. We therefore implemented a demonstrator with a local and a foreign agent that interact with human users in a direction giving scenario. A user study revealed that participants behaved differently towards the two agents and adapted both, their verbal and their non-verbal behaviour. We are thus confident, that our demonstrator is well suited to systematically study AFC.}, author = {Lugrin, Birgit and Eckstein, Benjamin and Bergmann, Kirsten and Heindl, Corinna}, booktitle = {International Conference on Intelligent Virtual Agents}, keywords = {myown}, note = {in press}, organization = {Springer}, title = {Adapted Foreigner-directed Communication towards Virtual Agents}, year = 2018 }

  %0 Conference Paper %1 lugrin2018adapted %A Lugrin, Birgit %A Eckstein, Benjamin %A Bergmann, Kirsten %A Heindl, Corinna %B International Conference on Intelligent Virtual Agents %D 2018 %T Adapted Foreigner-directed Communication towards Virtual Agents %U https://www.youtube.com/watch?v=tuyGgCD_RbE %X People tend to adapt to their conversation partners. In mixed-cultural settings, with a native and a non-native speaker, adaptation can manifest itself in the usage of simplified language or increased usage of non-verbal scaffolding to foster understanding. In this contribution, we address the question whether the phenomenon of AFC (adapted, foreigner-directed communication) is also shown towards a virtual agent. We therefore implemented a demonstrator with a local and a foreign agent that interact with human users in a direction giving scenario. A user study revealed that participants behaved differently towards the two agents and adapted both, their verbal and their non-verbal behaviour. We are thus confident, that our demonstrator is well suited to systematically study AFC.
• Towards Serious Games and Applications in Smart Substitutional Reality Eckstein, Benjamin; Krapp, Eva; Lugrin, Birgit in 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games) (2018). 25–32.

  in press
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  Substitutional Reality (SR), the integration of the physical environment into Virtual Reality (VR), is a novel approach to facilitate and intensify the use of home-based VR systems. We propose to extend the passive haptics of SR with the interactive functionality of a smart home environment. This concept of smart SR is meant as a foundation for serious games and applications. In this paper, we describe the concept behind smart SR as well as the prototype in our lab environment. We created multiple virtual environments with a varying degree of mismatch regarding the real world. We present a user study where we examined the influence of these environments on the perceived sense of presence and motivation of users. Our findings showed that presence was high in all conditions while motivation increased with the level of mismatch. This provides us with a promising basis for further research.

  @inproceedings{eckstein2018towards, abstract = {Substitutional Reality (SR), the integration of the physical environment into Virtual Reality (VR), is a novel approach to facilitate and intensify the use of home-based VR systems. We propose to extend the passive haptics of SR with the interactive functionality of a smart home environment. This concept of smart SR is meant as a foundation for serious games and applications. In this paper, we describe the concept behind smart SR as well as the prototype in our lab environment. We created multiple virtual environments with a varying degree of mismatch regarding the real world. We present a user study where we examined the influence of these environments on the perceived sense of presence and motivation of users. Our findings showed that presence was high in all conditions while motivation increased with the level of mismatch. This provides us with a promising basis for further research.}, author = {Eckstein, Benjamin and Krapp, Eva and Lugrin, Birgit}, booktitle = {2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games)}, keywords = {myown}, month = {Sept}, note = {in press}, pages = {25-32}, title = {Towards Serious Games and Applications in Smart Substitutional Reality}, year = 2018 }

  %0 Conference Paper %1 eckstein2018towards %A Eckstein, Benjamin %A Krapp, Eva %A Lugrin, Birgit %B 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games) %D 2018 %P 25-32 %T Towards Serious Games and Applications in Smart Substitutional Reality %U https://www.researchgate.net/publication/327720238_Towards_Serious_Games_and_Applications_in_Smart_Substitutional_Reality %X Substitutional Reality (SR), the integration of the physical environment into Virtual Reality (VR), is a novel approach to facilitate and intensify the use of home-based VR systems. We propose to extend the passive haptics of SR with the interactive functionality of a smart home environment. This concept of smart SR is meant as a foundation for serious games and applications. In this paper, we describe the concept behind smart SR as well as the prototype in our lab environment. We created multiple virtual environments with a varying degree of mismatch regarding the real world. We present a user study where we examined the influence of these environments on the perceived sense of presence and motivation of users. Our findings showed that presence was high in all conditions while motivation increased with the level of mismatch. This provides us with a promising basis for further research.

• Augmented Reasoning in the Mirror World Eckstein, Benjamin; Lugrin, Birgit in Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology, VRST ’16 (2016). 313–314.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  In order to enable a social agent to behave in a believable and realistic way, it needs a wide range of information in the form of both low-level value-based data as well as high-level semantic knowledge. In this work we propose a system that puts a virtual reality layer between the real world and an agent's knowledge representation. This mirror world allows the agent to use its abstract representation of the environment and inferred events as an additional source of knowledge when reasoning about the real world. Additionally, users and developers can use the mirror world, with its visualized data and highlighting of the agent's reasoning, for further understanding of the agent's behavior, debugging and testing, or the simulation of additional sensor input.

  @inproceedings{Eckstein:2016:ARM:2993369.2996316, abstract = {In order to enable a social agent to behave in a believable and realistic way, it needs a wide range of information in the form of both low-level value-based data as well as high-level semantic knowledge. In this work we propose a system that puts a virtual reality layer between the real world and an agent's knowledge representation. This mirror world allows the agent to use its abstract representation of the environment and inferred events as an additional source of knowledge when reasoning about the real world. Additionally, users and developers can use the mirror world, with its visualized data and highlighting of the agent's reasoning, for further understanding of the agent's behavior, debugging and testing, or the simulation of additional sensor input.}, address = {New York, NY, USA}, author = {Eckstein, Benjamin and Lugrin, Birgit}, booktitle = {Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology}, keywords = {mirrorworld}, pages = {313–314}, publisher = {ACM}, series = {VRST '16}, title = {Augmented Reasoning in the Mirror World}, year = 2016 }

  %0 Conference Paper %1 Eckstein:2016:ARM:2993369.2996316 %A Eckstein, Benjamin %A Lugrin, Birgit %B Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology %C New York, NY, USA %D 2016 %I ACM %P 313--314 %R 10.1145/2993369.2996316 %T Augmented Reasoning in the Mirror World %U http://doi.acm.org/10.1145/2993369.2996316 %X In order to enable a social agent to behave in a believable and realistic way, it needs a wide range of information in the form of both low-level value-based data as well as high-level semantic knowledge. In this work we propose a system that puts a virtual reality layer between the real world and an agent's knowledge representation. This mirror world allows the agent to use its abstract representation of the environment and inferred events as an additional source of knowledge when reasoning about the real world. Additionally, users and developers can use the mirror world, with its visualized data and highlighting of the agent's reasoning, for further understanding of the agent's behavior, debugging and testing, or the simulation of additional sensor input. %@ 978-1-4503-4491-3
• Dynamic Context Integration through Modularized Ontologies and Semantic Blueprints Eckstein, Benjamin; Lugrin, Birgit in International Workshop on Human-Agent Interaction Design and Models (HAIDM 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Some recent approaches in context-aware systems deploy on-tologies to benefit from their expressive power and extensive availability within the World Wide Web. However, wide-ranging ontologies tend to grow into large and complicated constructs, thereby making them difficult to maintain or reason on. In this work, we present a lightweight system design that combines the advantages of minimal, distributed or modularized ontologies with the computational power of a state-of-the-art real-time interactive system. Therefore, we introduce a simple data-structure called blueprints, that describes various reasoning operations to allow the dynamic integration of domain-specific knowledge for time-critical tasks, e.g. in multi-agent systems. Following this concept, we formulate four major use cases which are described through exemplary problems and proposed solutions. The presented design aims at porta-bility and adaptability, while maintaining real-time capability.

  @conference{eckstein2016dynamic, abstract = {Some recent approaches in context-aware systems deploy on-tologies to benefit from their expressive power and extensive availability within the World Wide Web. However, wide-ranging ontologies tend to grow into large and complicated constructs, thereby making them difficult to maintain or reason on. In this work, we present a lightweight system design that combines the advantages of minimal, distributed or modularized ontologies with the computational power of a state-of-the-art real-time interactive system. Therefore, we introduce a simple data-structure called blueprints, that describes various reasoning operations to allow the dynamic integration of domain-specific knowledge for time-critical tasks, e.g. in multi-agent systems. Following this concept, we formulate four major use cases which are described through exemplary problems and proposed solutions. The presented design aims at porta-bility and adaptability, while maintaining real-time capability.}, author = {Eckstein, Benjamin and Lugrin, Birgit}, booktitle = {International Workshop on Human-Agent Interaction Design and Models (HAIDM 2016)}, keywords = {blueprints}, title = {Dynamic Context Integration through Modularized Ontologies and Semantic Blueprints}, year = 2016 }

  %0 Generic %1 eckstein2016dynamic %A Eckstein, Benjamin %A Lugrin, Birgit %B International Workshop on Human-Agent Interaction Design and Models (HAIDM 2016) %D 2016 %T Dynamic Context Integration through Modularized Ontologies and Semantic Blueprints %X Some recent approaches in context-aware systems deploy on-tologies to benefit from their expressive power and extensive availability within the World Wide Web. However, wide-ranging ontologies tend to grow into large and complicated constructs, thereby making them difficult to maintain or reason on. In this work, we present a lightweight system design that combines the advantages of minimal, distributed or modularized ontologies with the computational power of a state-of-the-art real-time interactive system. Therefore, we introduce a simple data-structure called blueprints, that describes various reasoning operations to allow the dynamic integration of domain-specific knowledge for time-critical tasks, e.g. in multi-agent systems. Following this concept, we formulate four major use cases which are described through exemplary problems and proposed solutions. The presented design aims at porta-bility and adaptability, while maintaining real-time capability.

• PEARS - Physics Extension And Representation through Semantics Eckstein, B.; Lugrin, Jean-Luc; Wiebusch, D.; Latoschik, M.E. in IEEE Transactions on Computational Intelligence and AI in Games (2015). 8(2) 178–189.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  Today’s physics engines mainly simulate classical mechanics and rigid body dynamics, with some late advances also capable of simulating massive particle systems and some approximations of fluid dynamics. An accurate numerical simulation of complex non-mechanical processes in real-time is beyond the state-of-the-art in the respective fields. This article illustrates an alternative approach to a purely numerical solution. It uses a semantic representation of physical properties and processes as well as a reasoning engine to model cause and effect between objects, based on their material properties. Classical collision detection is combined with semantic rules to model various physical processes, e.g., in the areas of thermodynamics, electrodynamics, and fluid dynamics as well as chemical processes. Each process is broken down into fine-grained sub-processes capable of approximating continuous transitions with discretized state changes. Our system applies these high-level state descriptions to low-level value changes, which are directly mapped to a graphical representation of the scene. We demonstrate our framework’s ability to support multiple complex, causally connected physical and chemical processes by simulating a Goldberg machine. Our performance benchmarks validate its scalability and potential application for entertainment or edutainment purposes.

  @article{7347376, abstract = {Today’s physics engines mainly simulate classical mechanics and rigid body dynamics, with some late advances also capable of simulating massive particle systems and some approximations of fluid dynamics. An accurate numerical simulation of complex non-mechanical processes in real-time is beyond the state-of-the-art in the respective fields. This article illustrates an alternative approach to a purely numerical solution. It uses a semantic representation of physical properties and processes as well as a reasoning engine to model cause and effect between objects, based on their material properties. Classical collision detection is combined with semantic rules to model various physical processes, e.g., in the areas of thermodynamics, electrodynamics, and fluid dynamics as well as chemical processes. Each process is broken down into fine-grained sub-processes capable of approximating continuous transitions with discretized state changes. Our system applies these high-level state descriptions to low-level value changes, which are directly mapped to a graphical representation of the scene. We demonstrate our framework’s ability to support multiple complex, causally connected physical and chemical processes by simulating a Goldberg machine. Our performance benchmarks validate its scalability and potential application for entertainment or edutainment purposes.}, author = {Eckstein, B. and Lugrin, Jean-Luc and Wiebusch, D. and Latoschik, M.E.}, journal = {IEEE Transactions on Computational Intelligence and AI in Games}, keywords = {myown}, month = {June}, number = 2, pages = {178-189}, title = {PEARS - Physics Extension And Representation through Semantics}, volume = 8, year = 2015 }

  %0 Journal Article %1 7347376 %A Eckstein, B. %A Lugrin, Jean-Luc %A Wiebusch, D. %A Latoschik, M.E. %D 2015 %J IEEE Transactions on Computational Intelligence and AI in Games %N 2 %P 178-189 %R 10.1109/TCIAIG.2015.2505404 %T PEARS - Physics Extension And Representation through Semantics %U http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7347376 %V 8 %X Today’s physics engines mainly simulate classical mechanics and rigid body dynamics, with some late advances also capable of simulating massive particle systems and some approximations of fluid dynamics. An accurate numerical simulation of complex non-mechanical processes in real-time is beyond the state-of-the-art in the respective fields. This article illustrates an alternative approach to a purely numerical solution. It uses a semantic representation of physical properties and processes as well as a reasoning engine to model cause and effect between objects, based on their material properties. Classical collision detection is combined with semantic rules to model various physical processes, e.g., in the areas of thermodynamics, electrodynamics, and fluid dynamics as well as chemical processes. Each process is broken down into fine-grained sub-processes capable of approximating continuous transitions with discretized state changes. Our system applies these high-level state descriptions to low-level value changes, which are directly mapped to a graphical representation of the scene. We demonstrate our framework’s ability to support multiple complex, causally connected physical and chemical processes by simulating a Goldberg machine. Our performance benchmarks validate its scalability and potential application for entertainment or edutainment purposes.

• Towards Learning Error-Driven Transformations for Information Extraction Eckstein, Benjamin; Kluegl, Peter; Puppe, Frank in Working Notes of the LWA 2011 - Learning, Knowledge, Adaptation (2011).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{eckstein2011towards, author = {Eckstein, Benjamin and Kluegl, Peter and Puppe, Frank}, booktitle = {Working Notes of the LWA 2011 - Learning, Knowledge, Adaptation}, keywords = {extraction}, title = {Towards Learning Error-Driven Transformations for Information Extraction}, year = 2011 }

  %0 Conference Paper %1 eckstein2011towards %A Eckstein, Benjamin %A Kluegl, Peter %A Puppe, Frank %B Working Notes of the LWA 2011 - Learning, Knowledge, Adaptation %D 2011 %T Towards Learning Error-Driven Transformations for Information Extraction