Computer Vision News - November 2022

59 Metrics Reloaded students and two master’s students in my group. We’re pursuing work on everything to do with human-machine interaction, including active learning approaches, explainable AI, and detecting failures in systems . When I’m not supervising my group, my hands-on work is mostly the Metrics Reloaded project. ” The team plans to submit two papers to Nature Methods shortly. One is about understanding and analyzing the pitfalls of metrics. The second is the recommendation framework, where users are guided through the process of selecting and applying appropriate validation metrics while being made aware of the potential pitfalls. It includes a user-friendly online tool, providing a point of access to explore the weaknesses and strengths of the most common metrics. Project coordinator Lena Maier-Hein believes the work to be essential to understanding the limited translation of machine learning methods into clinical practice. “ Thiswork is just thestart, ”Paul teases. “ This online framework maps your problem to recommended metrics, but there’s room for much more. It could be the central point for education and transparency about metrics and their limitations. If you’re interested in a specific metric, we have a library where you can click on it and get all the information, see the connections, and generate tables with the correlations between metrics. I imagine interactive educational tools where you can create simulations with data sets to shift around and see how the metrics behave . We want to be the central hub for metric understanding in the community. That is the long-term vision. ” convince people that they may have been doing something wrong. ” Ultimately, the team wants to be able to map every personal problem to a set of recommended metrics. Their rules need to catch every use case. They are looking to tackle all scales of classification at once, including semantic segmentation, instance segmentation, object detection, and image-level classification , and aim to be agnostic to application domains, incorporating histopathologic images, radiological images, and microscopic images . They see similarities and synergies between them all. One can imagine that telling people where they have been going wrong is not without its own pitfalls. Did the team face any opposition? “ Yes,constantly! ”Paul reveals.“ I’mcurrently part of parallel email discussions fighting about the best metric for certain use cases. It’s not always clear. We could have made it easier for ourselves, but we defined a very broad scope and ambitious setting. It makes sense to address everything at once, but that also takes more time and effort. ” Although Metrics Reloaded occupies a large part of their time, Paul and Minu have other roles. Minu is a physician by background, having studied medicine, but is now working at the intersection of her department’s technical and medical sides. “ Metrics is my main occupation timewise, but inour department,we’re trying to create long-lasting structural efforts in surgical data science , and I’m heavily involved in that too, ” she tells us. Paul adds: “ Now that I’m an independent research group leader, I supervise four PhD