Computer Vision News - March 2023

6 AI Research a first step to more complicated robotics algorithms. She submitted the work to ICRA 2023 , a robotics conference, with these goals in mind. “ Something I want to achieve for my thesis and PhD is to collect assets from the real world , ” she tells us. “ If we have a NeRF of a cup on a table, I will see the complete 360 degrees of the cup, but not the underneath. If I make a model from that one NeRF, it will always miss the bottom part. Although there are ways to find this bottom part, through inpainting and generative models, they’re not 100% accurate. ” To solve this, Lily came up with the idea of making another NeRF of the cup from a different angle and registering it with the original NeRF to obtain complete information about the object. Having completed the registration step, she is now working on combining the two NeRFs to extract the final model . Thinking about a medical scenario, is it possible to envision a scenario where it does not change with different lighting and is not v i ew- dependent , whereas color is completely view- dependent. She extracted a new field called a surface field from NeRF, which can be performed as a complete post- processing step without altering the NeRF pipeline . Surface fields give the probability that each point in a space is located on the surface of an object. This approach offers a clean domain with zero or one values describing the scene’s geometry. “ Now that we had the domain to compare the two NeRFs against, we could sample points in the two NeRFs near the object we wanted to register and then compare the pointwise surface values, ” Lily explains. “ To perform the classic optimization loop for rigid transformation, very similar to ICP, we compare the surface value of the sample pairs and optimize their rotation and translation to make these match iteratively. ” nerf2nerf is a high-level tool that could be used in different applications – bundle adjustment, SLAM, or structure from motion, which are hot topics in robotics right now. As 3D vision is likely to replace 2D vision in robotics, people are finding novel ways to incorporate NeRFs or other neural field domains into 3D vision for robotics . Lily hopes to see registration being used as

RkJQdWJsaXNoZXIy NTc3NzU=