Computer Vision News - May 2023

6 CVPR Highlight Paper: SPARF For the constraints, Prune took inspiration from multi-view problems and bundle adjustments , which are well explored in classical computer vision and geometry , and set about integrating them into the NeRF framework. SPARF can train in a much shorter window, but the output is still far from the rendering quality that could be achieved with dense views. Prune is keen to point out that although they have made great strides, there is still a road ahead. “ There’s still a lot of work towards getting perfect rendering from only two or three views, ” she tells us. “ We can do this joint pose-NeRF refinement much better than before,butwerelyonpointcorrespondences between the different views. The problem is getting these point correspondences is itself a research problem. It would be great if there were a way to train or refine everything together. For example, if you’re training the NeRF and refining the poses based on the correspondences, could you also update the correspondences based on the NeRF and the poses? Like a system where everything can get better all at once. We haven’t reached that point yet! ” Fabian continues: “ It’s a chicken and egg problem. COLMAP uses correspondences to get the poses, but you need the poses to get the correspondences. We need a tool that does it all at the same time. ” Could this be the seed for their next paper or perhaps open the door for other researchers? “ That would be pretty nice! ” Prune responds. “ We did a couple of experiments in that direction, and it’s turning out more challenging than we hoped, but it would be a very nice output and future direction. ” hundreds of images, but you don’t want to have to take hundreds of images of an object, ” Fabian points out. “ You take a few, and then this problem becomes severe because the poses you get are usually bad, andyouhavebothproblems simultaneously. That’s why this is an important research direction for real applications! ” Prune agrees: “ For us, it was still interesting. Eventhoughthis jointpose-NeRF refinement didn’t work for the sparse scenario, I started from this and worked my way up by adding new constraints. ”