WHAT DO WE DO?

To achieve this, we will:

1. Use the SERMAS Toolkit to simplify the creation and management of socially acceptable XR systems. Our scientific and technological methodology combines interdisciplinary, multisectoral, and pilot-driven approaches, to help innovators design, develop, deploy, and manage their XR systems.

2. Test the SERMAS Toolkit with real-world examples in different industries, transferring the project results to industrial practice. The developers of XR applications will actively work with our team to make this possible.

3. Enable innovators to cut down the time-to-market of their XR systems, by leveraging the SERMAS Toolkit to improve social acceptance of their solutions, thereby enhancing the competitiveness of the vendors.

4. Develop a sustainability plan to position and extend the use of the SERMAS toolkit in the next XR Systems.

We Have Nine Objectives:

#1 Define the SERMAS Methodology

Systematically tackling the social acceptance of XR systems requires following a rigorous methodology of modelling, development, assessment, and in-lab and real-life validation. The SERMAS Methodology helps XR engineers who, assisted by security analysts and social scientists, intend to develop next-generation XR systems that their human users can accept.

#2 Develop the SERMAS XR Agent

The SERMAS XR Agent combines hardware, software, and algorithmic modules to implement an XR model and create personalised XR systems. It is also for non-specialised users.

#3 Improve open natural language generation

The XR Agent accesses visual and language information and communicates with users using verbal and non-verbal signals.

The SERMAS team will evaluate the model's ability to generate appropriate responses using automatic and human-based metrics.

#4 Context awareness and integration of structured knowledge

The SERMAS XR Agent is context-aware, using its sensing suite to gather information on the environment and nearby users to determine appropriate actions and gestures.

#5 Frictionless interaction

The SERMAS XR Agent interacts with users through human-to-human communication, utilising known effective communication mechanisms. The goal is to create a frictionless interaction between the user and the XR Agent.

#6 Augmented gesture-based communication skills

The SERMAS XR Agent recognises and generates pointing gestures to refer to spatially-located entities. Gesture recognition relies on machine learning applied to gesture sensing and geometrical techniques. These gestures serve as an additional means of interaction within a broader scope that includes multiple verbal and nonverbal communication channels.

#7 Security assessment

Social acceptance of an XR system ultimately relies on the interaction’s security (including privacy and trustworthiness). Establishing security requires treating the XR system as a socio-technical framework in which technology and humans exchange messages and data.

#8 Release the SERMAS Toolkit

The Toolkit combines the SERMAS XR Agent and the methods and tools developed in the previous objectives. It also integrates the technologies and pilots from the third-party innovators joining SERMAS following our open calls. The Toolkit provides XR engineers with examples and an integrated front-end to maximise the social acceptance of their systems.

#9 Demonstrate the SERMAS Proof-of-Concept and establish social acceptance of the pilots

The pilots consolidate the value and enable the wider use and adoption of the SERMAS Methodology and Toolkit, advancing and establishing the social acceptance of the XR systems.