Computer Vision News - June 2023

19 Georgia Chalvatzaki That is more the part of reinforcement learning and how the robot can figure things out that go beyond what I would have expected. For example, one of the first works that I did here in Darmstadt when I arrived was on reinforcement learning. We had a robotic hand that we wanted to learn on the robot. We were not doing simulation at all; we were training the robotic hand itself. We wanted it to learn to open valves. We had a hand with four fingers, and the robot found out that it can do this very effectively using only two out of the four fingers. It did not lose energy to move the other two fingers because it figured out that “ I can do it very well by just using my two fingers! ” This is connected to what I said before. This amazed me. Yeah, it was very, very interesting for me to see. You have chosen to work and have a career outside of your own country. I ask the question because I visited Greece There are things definitely that, by reinforcement learning, for example, we can see robots finding solutions that could be beyond what I could have taught the robot by kinesthetic teaching. I could have taken the armof the robot and tried toput it in a specific location, and then if I just allow the robot to explore its own capabilities in some sense because robots and humans do not have exactly the same structure in our bodies, and every robot is completely different, through reinforcement learning, we can see robots discovering their abilities into learning how to do things so that they can complete a task without doing exactly what I would have done. As long as they can deliver the thing. For me, it is okay if they do it in a completely different fashion than I would do it, but of course, they have to be successful. How you can motivate the robot to learn is actually one complexity of reinforcement learning. How do you define the reward? How do you define what is good behavior and what is bad behavior so that the robot is able to experience things and learn through this experience? What was your most impressive eureka moment? Ah, eureka! Yes! [ Georgia laughs ] This is a very, very interesting question. I would say that my first experience with reinforcement learning was the one, trying to understand the interplay between thinking ahead and being very reactive. Claus Voelker