Computer Vision News - January 2023

39 Claudia Dettorre algorithms, therefore, should be considering all these aspects when aiming to autonomously execute surgical tasks. I worked with three main applications of the pick and place task (Figure 1): pick-up of a circular needle prior to suture, pick and place of a Pneumatically Attachable Flexible (PAF) rail for intraoperative Ultrasound (US) scanning and retraction for intraoperative organ repositioning using the PAF rail. I developed a calibration pipeline embedded in a control algorithm that allows precision movement of the robot, capable of grasping a few millimetre-thick needle [1] . I used learning from demonstration approaches to plan the pick and place of the PAF rail inside a surgeon-in-the-loop control algorithm for execute us scanning in case of partial nephrectomy [2] . I also worked on planning in dynamically changing environments for organ retraction using gradient-basedmethods totriggerasmootherobject repositioningphaseduring intraoperative procedures [3] . This procedure becomes extremely useful when surgeons need to access areas of the patient’s abdomen that are not visible because covered by other organs or tissues. As last, to improve scene understanding, I developed a simulation environment where multiple tasks can be learned based on the surgical scene using Reinforcement Learning and then transferred to the real robot (Figure 2) [4] . My experiments proved thatautomationof thepick and place task of different surgical objects is possible. The robot was successfully able to autonomously pick up a suturing needle, position a surgical device for intraoperative ultrasound scanning and manipulate soft tissue for intraoperative organ retraction. Figure 2: representation of the MDP control loop on the top, and on the bottom how the model has been applied to solving surgical tasks.