Computer Vision News - June 2023

18 Women in Computer Vision trying to understand distances by querying batches of thousands of points and trying to adapt the way the robot would move so that it does not collide and effectively completes a task. This combines ideas both from geometric deep learning and reinforcement learning, so that the robot can learn based on what it perceives how to adapt. Doyouevergetfrustrated with the robot and say, I am done with that? I do not want to work with robots anymore! [ Georgia laughs ] Yes, this is an everyday experience for us! Basically, you start working with the robot, you set up everything, and then suddenly, when you have everything very nicely done in simulation, you try to do it on the real robot, and actually, it does not work. Other things that happen are that the robot may overheat and then stop working, or you program things in a very good situation, and then suddenly your camera needs to be calibrated again, and then you have to start all over until you find out what is going on. You can look at it and you can see that it was working, but what is happening? Then you say, oh no, I have to calibrate this camera again, and hours have passed. So, yes, frustration is a part of our job! [ She laughs ] But I am sure that on the other side, there are many things that you have taught the robot to do that we cannot do. What is missing for us to be able to do it? We need many things. First of all, we need to be able to perceive well the components of the scene, such that we know which parts hold enough for them, which can be graspable, and for which task they can be used. Then we need this higher level of reasoning, so the robot will be able to think of the sub-tasks that you have to do to do these more complicated, easy-for-us things to do. The other part is that you need better coordination in the body of the robot. For example, for us, it is very intuitive to open the door of the fridge, and while we are opening it, we are also adapting our body while this door is being manipulated, but this is very challenging to do with a robot without the robot hitting the door of the fridge, hitting itself, colliding, and so on. We have a lot of progress to do there. For this, wehave looked intoneural representations, so geometric representations. We are Katrin Binner