Here, you can play around with the GUI a bit to get familiar with the perception pipeline (we will code everything and will not use the GUI, but it is good to get familiar with it). The first command is use_armtag_tuner_gui:=true, which will open up a Graphical User Interface (GUI) that will allow you to figure out where the position of the arm is with respect to the camera. In the Armtag tuner GUI, you can see a field named ’num samples’. This can vary from 1 to 10, with 10 indicating the highest level of accuracy. When you press the ’snap pose’ button, the system will capture the specified quantity of images as defined in the ’num samples’ setting. The AprilTag algorithm will then run on each of the images, probably producing slightly different poses for the AprilTag’s location relative to the camera. Finally, the program behind the GUI will average all those poses to hopefully obtain the most accurate position. As you will see in the GUI message the snapped pose represents the transform from the ’camera_optical_frame’ frame to the ’px_100/base_link’ frame. Try to verify this by manually measuring the portion of the camera w.r.t the base. At this point, you should see a pointcloud version of your tabletop with the objects on it. The second GUI applies some filters on the image in a way that all the objects are clear in our image. This GUI obtains the raw point cloud from our depth camera and applies several filters to it in a manner that makes the objects (clusters) visible. This GUI contains several sections for filtering the point cloud. The description for each of these sections is provided in the GUI windows, and you can also see this guide for how to go about doing this. In general, this GUI will employ a mathematical model to identify the plane on which the objects are positioned. Then, it applies a Radius Outlier Removal filter to omit ’noisy’ points in the point cloud. To understand how to create a perception pipeline, you can follow the instructions in this link. Now, let’s go ahead and implement our own code for cluster detection using the camera and picking and placing by solving the numerical inverse kinematics of the arm at the desired poses detected by the camera. 9 Computer Vision News Vision-aided Screw Theory-based…
RkJQdWJsaXNoZXIy NTc3NzU=