Systematically tackling the social acceptance of XR systems requires following a rigorous methodology of modelling, development, assessment, and in-lab and real-life validation. The SERMAS Methodology helps XR engineers who, assisted by security analysts and social scientists, intend to develop next-generation XR systems that their human users can accept.
#1
Define the SERMAS
Methodology
The SERMAS XR Agent combines hardware, software, and algorithmic modules to implement an XR model and create personalised XR systems. It is also for non-specialised users.
#2
Develop the SERMAS XR Agent
The XR Agent accesses visual and language information and communicates with users using verbal and non-verbal signals. The SERMAS team will evaluate the model's ability to generate appropriate responses using automatic and human-based metrics.
#3
Improve open natural language generation
The SERMAS XR Agent is context-aware, using its sensing suite to gather information on the environment and nearby users to determine appropriate actions and gestures.
#4
Context awareness and integration of structured knowledge
The SERMAS XR Agent interacts with users through human-to-human communication, utilising known effective communication mechanisms. The goal is to create a frictionless interaction between the user and the XR Agent.
#5
Frictionless interaction
The SERMAS XR Agent recognises and generates pointing gestures to refer to spatially-located entities. Gesture recognition relies on machine learning applied to gesture sensing and geometrical techniques. These gestures serve as an additional means of interaction within a broader scope that includes multiple verbal and nonverbal communication channels.
#6
Augmented gesture-based communication skills
The Toolkit combines the SERMAS XR Agent and the methods and tools developed in the previous objectives. It also integrates the technologies and pilots from the third-party innovators joining SERMAS following our open calls. The Toolkit provides XR engineers with examples and an integrated front-end to maximise the social acceptance of their systems.
#8
Release the SERMAS Toolkit
The pilots consolidate the value and enable the wider use and adoption of the SERMAS Methodology and Toolkit, advancing and establishing the social acceptance of the XR systems.
#9
Demonstrate the SERMAS Proof-of-Concept and establish social acceptance of the pilots