UTMIST Talk Series | ML for 3D Perception: Differentiating and Embedding Imaging Systems for Deep Learning

Image of Seminar Speaker

UTMIST is thrilled to have Wenzheng Chen, a 4th-year Ph.D. student, with us to talk about his work intersecting computational photography with deep learning on March 8th!

3D perception is one of the fundamental problems in computer vision, with numerous applications ranging from AR/VR to industrial automation. Various 3D imaging systems, together with AI based methods, are proposed to address this problem. However, existing techniques are always restricted in different aspects. For example, AI based methods require 3D training data, which is very expensive to acquire. Stereo matching, relying on correspondence, would easily fail on a white wall. Active 3D imaging systems (i.e. Structured Light or Time of Flight) infer depth by actively sending and receiving light signals, which would be trapped in complex materials (light propagation path is hard to compute) or low SNR (receiving too weak signals in dark environment).

In this talk, Wenzheng Chen will introduce several projects from NVIDIA Toronto AI Lab, which try to overcome these limitations. The key idea is to differentiate imaging systems and incorporate them with deep learning. Turning imaging systems to be differentiable is a prerequisite to put them in SGD optimization, while the learning systems, built on top of imaging systems, could inherently utilize the specific geometric or physical rules inside the imaging systems, resulting in much better performance than traditional black box neural networks. He demonstrates that such a combination brings huge benefits.

Event Details

Agenda

Presentation on Differentiating and Embedding Imaging Systems (5:00 - 5:50PM)

Q&A with Wenzheng Chen (5:50 - 6:00PM)

About the Presenter

Wenzheng Chen is a 4-th Ph.D. student at University of Toronto, supervised by Prof. Sanja Fidler and Prof. Kyros Kutulakos. He is also a research scientist at NVIDIA. His research mainly focuses on the intersection of computational photography and deep learning, with special interest in differentiating various imaging systems and embedding them in deep learning. If you would like to know more about his work, you can check out his website here.