CVPR Daily - Tuesday

“ All these NeRF papers assume there are hundreds of images, but you don’t want to have to take hundreds of images of an object, ” Fabian points out. “ You take a few, and then this problem becomes severe because the poses you get are usually bad, and you have both problems simultaneously. That’s why this is an important research direction for real applications! ” Prune agrees: “ For us, it was still interesting. Even though this joint pose-NeRF refinement didn’t work for the sparse scenario, I started from this and worked my way up by adding new constraints. ” For the constraints, Prune took inspiration from multi-view problems and bundle adjustments , which are well explored in classical computer vision and geometry , and set about integrating them into the NeRF framework. SPARF can train in a much shorter window, but the output is still far from the rendering quality that could be achieved with dense views. Prune is keen to point out that although they have made great strides, there is still a road ahead. “ There’s still a lot of work towards getting perfect rendering from only two or three views, ” she tells us. “ We can do this joint pose-NeRF refinement much better than before, but we rely on point correspondences between the different views. The problem is getting these point correspondences is itself a research problem. It would be great if there were a way to train or refine everything together. For example, if you’re training the NeRF and refining the poses based on the correspondences, could you also update the correspondences based on the NeRF and the poses? Like a system where everything can get better all at once. We haven’t reached that point yet! ” Fabian continues: “ It’s a chicken and egg problem. COLMAP uses correspondences to get the poses, but you need the poses to get the correspondences. We need a tool that 5 DAILY CVPR Tuesday SPARF - NeRF from Sparse and Noisy Poses