Optimising Camera Planning for Aesthetic Cinematographic Production

Scholarship Program: MyWorld provides state-of-the-art studios which encompass almost every aspect of entertainment research, from virtual production stage to a viewing theatre with built in sensors for audience understanding. With more than 30 collaborators (including Oscar-award winning production houses) and many historic partnerships (producers with large stakes in the visual entertainment space), this project is well supported.

Research Context and Potential Impacts:

Cinematography concerns the art of motion-picture filmmaking, thus intelligent cinematography (IC) encompasses the technology (predominantly Artificial Intelligence (AI)) which aids and improves this art-form.

Historically, we have only really seen improvements in camera technology as a result of cinematographic endeavours, thus IC is well a founded and proven research domain

The research field is niche and highly specialised - understanding the cinematographic language, audience psychology and variations in artistic style, as well as how AI (notably 3-D video and image capturing techniques) fits into this, is necessary. Unfortunately, there are few researchers with these skill sets who are actively engaging in open-source research.

Large pioneering enterprises (Disney, Netflix, Amazon Prime Video, etc.) "gatekeep" a significant amount of high-quality information, thus there is also potential to contribute to the lacking amount of open-source information (supported by the MyWorld infrastructure).

Filmmaking is expensive and time-consuming, thus practical research and development can lead to substantial reductions in cost and energy (ILM's Stagecraft is a perfect example of cutting set-design related costs). However, few publications address the uses of their systems within the context of real production, as it is costly to verify research results in actual practice. With the MyWorld scholarship, this level of verification is easier to attain with the resources and network.

Primary Objectives:

Reduce the cost of aesthetic cinematic production by automating camera-planning tasks (minimising necessary equipment, optimising distribution and orientation) relative to a dynamic 3-D set. Capture a 3-D environment and simulate the key sequences in scene, tailored for AI tasking. Provide this in an accessible manner to be used during pre-production (using generally accepted software and kits)

Stetch Objectives:

Generate and optimise sequences of shots, transitions and cuts to provide a variety of suggestions for how to shoot given some target characteristic (e.g. stylised sequences, like a Tarantino-styled climactic event) - useful to maintain coherency between scenes which embed similar characteristics Novelties:

This method looks at improving two aspects which are substantially lacking in current research:

Placing more attention on audience psychology and understanding during the development and verification stages of the project

Utilising state-of-the-art infrastructure (MyWorld) to ensure the quality of research is appropriate for in-field practice (for development and verification)

EPSRC Alignment: This project falls within the EPSRC Artificial Intelligence Technologies research area. More technically, it concerns video and image processing techniques for 3-D environment capture and simulation and well as AI for multi-objective optimisation . Companies and Collaborators: TBD - meetings set up for late 2022/early 2023