39 Reproducibility Computer Vision News During the tutorial, the participants were asked to comment on the reproducibility of various fake MICCAI-like papers that we invented. They were guided by the MICCAI checklist, which covers several elements: Models and Algorithms, Datasets, Code and Experimental results. As there is no absolute answer when analyzing reproducibility, this led to rich discussions and a few conclusions. First and foremost, it is not easy to understand what is expected in terms of reproducibility. Should the authors aim for exact reproducibility or rather conceptual reproducibility? Should the items of the reproducibility checklist to focus on be adapted to the type of paper? For example, a paper presenting a new method may need to particularly focus on the “Experimental results” section to adequately demonstrate the improvements reached compared with the state-of-the-art. The second main point of discussion was the role of the reviewers. Should they comment on the reproducibility of the paper in general, verify the consistency between the checklist and the paper, or both? Finally, the use of the checklist in the decisionmaking process was unclear to the participants, who were both authors and/or reviewers, which may impede the growth of reproducible research within the MICCAI community. We hope that the participants of the tutorial now have a better understanding of the concept of reproducibility and how to implement it in practice. We are also calling on the community to help clarify the points raised during our discussions for the future MICCAI conferences! BEST OF MICCAI 2023
RkJQdWJsaXNoZXIy NTc3NzU=