AURA.I. AUgmented Reality utilizing Artificial. Intelligence.

Purpose and goal:

In collaboration with Beebox Studios, University West, and IIT Madras; the objectives of this project are to design a Software Development Kit (SDK) that can be used in 3D engines to support Extended Reality (XR) applications in utilizing Large Language Model (LLM) Artificial Intelligence (AI) to provide more personalized and more specific instructions to the user, using both the user´s profile and the environment. The project will also use this SDK to create an official prototype application to assist inspection tasks at GKN in an industrial setting.

Expected results and effects:

The expected result is to produce a marketable toolkit that can be integrated into 3D engines to produce applications for XR devices that can utilize AI to provide instructions. Published results will be released into the development of an LLM model integrated with XR systems. Alongside research for both the large language model and XR hardware integration, results of the user study for the toolkit integrated into an application for inspection processes at GKN will be made public.

Approach and implementation:

Our project is divided into 5 main tasks. Task 1 is to do research into Large Language Models with IIT Madras and University West, task 2 is to do research into integrating XR hardware inputs and outputs into AI with GKN and University West, task 3 to develop an AI toolkit with Beebox, task 4 is to use the toolkit with a Augmented Reality application in an industrial setting Beebox and IIT Madras, and task 5 is to create a usability study on the application with University West.