CVPR Daily - Tuesday

Yoni Kasten is a Computer Vision PhD Student at Weizmann Institute of Science in Rehovot, Israel. He speaks to us about his paper, which explores a unique method for solving the problem of structure-from-motion. This paper is co-authored with Amnon Geifman, Meirav Galun and Ronen Basri. Structure-from-motion is a classic problem and a very important one for automatic 3D modelling using 2D images. Yoni claims his method is the first one that solves the projective structure-from-motion problem directly from a set of fundamental matrices, where the fundamental matrix defines the relationship between any two images of the same scene. The method produces one consistent structure of the scene and the cameras, from all the available fundamental matrices . To do this, it uses a big matrix called a multi-view fundamental matrix which contains as blocks all the fundamental matrices between pairs of cameras. The paper defines and proves sufficient conditions on the n-view fundamental matrix that makes this matrix consistent with a set of cameras. After optimising for such consistent n-view fundamental matrix from the measurement pairwise fundamental matrices, the method can then extract all the cameras in one step directly from the fundamental matrices. A challenge in doing this is that the fundamental matrices that come from the measurements are noisy and there are also missing entries that have to be completed somehow. Yoni explains the optimization process: “ We actually had to optimise for rank constraints on the matrices, which is very challenging because optimizing with rank constraint is non-convex optimization . We had to somehow perform the optimization with the constraints and still get good enough results. We solved this using a successful approach called ADMM . We also had to build a graph of all the cameras and extract a triplet cover of the graph. This way we could complete all the missing entries and generate consistent cameras .” Global Projective SFM Using Algebraic Constraints on Multi-View Fundamental Matrices 16 DAILY CVPR Tuesday Presentation