Socially Embodied Artificial Intelligence

Today´s interactions with conversational interfaces are still mainly utilitarian and transactional because autonomous social decision-making is still unattainable in most embodied agents, especially in complex environments where multiple participants need to rely on their social skills to better perceive and make decisions.

To enable more social and relational types of interactions and applications, this project introduces a Socially Embodied Artificial Intelligence (SE.

AI) framework that connects multimodal social perception with multimodal behavior generation in complex social environments.

The framework includes a data collection platform that uses teleoperation and mixed reality technologies to enable the creation of robust multimodal, multiparty social datasets, that provide a real-time simulation of task actions, gestures, and other social behaviors.

Leveraging data extracted from this platform, novel models for social decision-making (i.e., believable and contingent task, verbal, and non-verbal behavior) for embodied agents will be created using multimodal fusion and deep machine learning techniques.

This framework is a paradigm shift compared to how robots learn social behavior today, paving the way for the next generation of socially autonomous embodied agents to be used in a wide range of applications, from healthcare assistants to personalized tutors.