3 Computer Vision News Neural radiance fields (NeRF) excel at reconstructing new viewpoints in 3D scenes using multiple 2D images. With a series of images captured by a smartphone, NeRF methods can generate a complete 360-degree rendering around an object of interest. However, there are certain limitations associated with this approach, including that the photos must be taken in pristine conditions , without any distracting elements such as moving pedestrians, clouds, or shadows. If any of these distractors or transient objects are present, NeRF renderings may exhibit artifacts . In this work, Sara attempts to prepare clean renderings as if the images were captured under ideal conditions, even when using images taken in the wild with different transient objects in the scene. “ Previous works tried to segment the pedestrian out or train a separate model for these moving objects, ” she tells us. “ They could only handle a specific set of transients. However, we approach it as an optimization task and say anything that’s not constant in the scene is an outlier in your optimization because it will have a really high loss. Our model can handle hundreds of transients of different types in the scene, and it will remove all of them! They don’t need to all be in the same category. ” In earlier works, transient objects typically consisted of common elements like the camera’s shadow or pedestrians on the street. Handling a specific model for these transients was sufficient for most datasets. However, as NeRF gained Sara Sabour is a Research Scientist at Google DeepMind and a PhD student at the University of Toronto. Her paper was accepted by CVPR 2023 as a highlight presentation. It proposes a new method for NeRF training to remove outliers from a scene, which is simple to incorporate into modern NeRF frameworks. She spoke to us ahead of her poster presentation. RobustNeRF: Ignoring Distractors with Robust Losses RobustNeRF: Ignoring Distractors with Robust Losses
RkJQdWJsaXNoZXIy NTc3NzU=