ICCV Daily 2021 - Thursday

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis was published at ECCV last year and Jon tells us even as they were finishing that paper, they knew there were still several things on their list to solve. One of those was a critical problem NeRF had with aliasing. “ NeRF works by casting rays from the camera out into the world and then sampling discrete points along that ray and pushing them through a neural network, ” he explains. “ This is generally a good idea, but it has a problem. You are shooting rays that are infinitely small. They are perfect rays, they do not have any thickness to them, and they fly through the air. Then you sample individual points along those rays. Those points are completely singular points. They have no size associated with them. ” Like a pixel? “ A pixel isn’t really a point; a pixel is a box , ” Jon responds. “ A good way to think of a pixel is that the value of a pixel is the integral of everything that hit that pixel. This is a critical thing for people working in graphics and image processing to consider. You can’t treat pixels like point estimates, you have to treat them like integrals over a little box to avoid aliasing and jaggies – bad looking images. In NeRF, we didn’t do that, we treated the pixel like a little point. We shot this little ray out that had no thickness and didn’t tell the ray it came from a small box, then when we would sample points along the ray, we would treat those points as if they were infinitely small. Maybe there’s something inside the pixel, but the ray Jon Barron is a staff research scientist in the Perception team at Google Research. His work, which he describes as essentially a bug fix for NeRF, has been accepted as an oral presentation, and received an Honorable Mention in the Paper Awards on Tuesday. Big congratulations to Jon! He speaks to us ahead of his live Q&A session today. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields 4 DAILY ICCV Thursday Oral Presentation