Computer Vision News - March 2019

by moving around and taking many images; in this way, we gave many 2D 3D correspondences to the algorithm, as if you were taking images of a big chessboard. It's actually good to use more than one image in any case for the optimization algorithm, even if most of the field of view is covered by one image. We even had to replace the chessboard since the first one was printed over a slightly bending material, bending the straight lines with it, which contributes to the error. In addition, for the extrinsic calibration part (the calibration between the two cameras), you want the camera to see exactly the same 3D points to correspond and then in some of the images we saw that there was a slight movement due to humans holding physically the chessboard in their hands. This slight difference can have a huge influence on the error, because it is actually not the same point that you are showing. The conditions for stereo calibration require that you use a grid with straight lines, with no movement between the two pictures. This solution per se is not very difficult: it requires organization and precision in order to make it work well at the first time, localizing exact 3D point images to give as input to the navigation system . The second challenge consisted in automatically extracting the same feature points from both images. To that purpose, we used deep learning neural network, which we trained to accurately localize these points. Of course, this requires a lot of data, which must be collected without delaying the Proof Of Concept’s schedule. Trying to triangulate image points that do not correspond exactly will cause an error. That’s why a great accuracy is required, even after the training of the network, hence some algorithms were used to enhance this accuracy of points correspondence. For instance, if the point of interest is on some plane seen by both cameras, you may want to get the homography matrix – a matrix that relates the transformation between two planes. Using the homography matrix, you can transform one image point to its exact corresponding point in the second image. How do you get the Homography? Again, there are several traditional computer vision algorithms. One way is using SIFT – matching similar feature points between the two images, after the neural network has identified the area of interest. It must be noted that SIFT might not work very well when the orientation of the cameras is too different; in addition, it may fail for pattern-like repeating structures, because the algorithm can mistakenly match similar wrong feature points. In that case another approach can be taken, which is called a dense stereo matching . In this method we find the depth map of our area of interest. When you have a depth map, you can better match each point with its correspondent point on the second image. With enough corresponding points, you can use algorithms like RANSAC to get the best fitting plane equation for the inliers point correspondences, and the plane equation will give you the homography. There are many challenges in computer vision projects like this and it is key that you work with experts like RSIP Vision’s engineers to solve them in the optimal way and reach your goals. Project 17 Computer Vision News A project by RSIP Vision Take us along for your next Deep Learning project! Request a call here