MIDL Vision 2022

The Metrics Reloaded framework is based on the novel concept of a problem fingerprint - a structured representation of the given problem. The fingerprint not only captures the domain interest, such as the particular importance of structure centers, boundaries or volumes, but also properties of the target structures (e.g. small size; high variability of sizes), the data set (e.g. class imbalance) and the algorithm output. Based on the fingerprint, the framework guides users through the process of selecting and applying a set of appropriate validation metrics while making them aware of potential pitfalls related to individual choices. An instantiation of the framework for several use cases from the field of biological and medical image analysis demonstrates the general applicability of the suggested approach. To improve the user experience , the team is currently developing the Metrics Reloaded toolkit, which captures all peculiarities of their framework in a user-centric manner . 17 Annika Reinke VISION MIDL There are various pitfalls related to the choice of metrics. For example, the Dice Similarity Coefficient (DSC) does not treat over- and undersegmentation equally.