Angel Chang (Simon Fraser University, Vancouver, and Canada CIFAR AI Chair): “Connecting 3D and Language”

Abstract

People communicate about objects, scenes, and spatial relations in the real world using natural language. In this talk, I will give an overview of recent work that explores how to endow computational systems with the ability to connect natural language and 3D representations. Concretely, I will summarize projects that develop neural models for localizing and describing objects in 3D scenes using natural language, generating 3D content from text, and other recent work exploring tasks that require connections between natural language and 3D representations.

Manolis Savva (Simon Fraser University, Vancouver, and Canada Research Chair in Computer Graphics): “Connecting 3D and Language”

Abstract

3D simulators are increasingly being used to develop and evaluate “embodied AI” (agents perceiving and acting in realistic virtual environments). Much of the prior work in this space has treated simulators as “black boxes” within which learning algorithms are to be deployed. However, the system characteristics of the simulation platforms themselves and the datasets that are used with these platforms both greatly impact the feasibility and the outcomes of experiments involving simulation. In this talk, I will describe recent projects that outline emerging challenges and opportunities in the development of 3D simulation for embodied AI.