Computer Vision News - December 2022

51 PI-CAI: Prostate Cancer AI Aswewrapup, Anindooffers us awonderful insight into the process of preparing for this challenge, which harks back to the old saying: if at first you don’t succeed, try, try again. “ Before I started as a PhD here, I was a master’s student, and there was another PhD working on this topic who was using part of the dataset we use now, ” he recalls. “ He found a lot of issues with that dataset that we fixed. Then whilst working, I found many more issues that were never spotted, and we fixed those. I was thinking, okay, fine, we now have perfect data. Then Joeran started his PhD, and he found even more issues, and I thought, oh my, this never ends! Then we fixed those. Great. Once we publicly released the dataset, the community found even more issues we had to fix! But that’s just how it is. Our expertise individually is limited, but when you release it to the public, you seemore things, and you can collectively make a nice, strong, clean dataset . The work with data never ends. It’s just forever data, data, data! ” a challenge has a hidden test set on which you submit your algorithm, but for the final phase of PI-CAI, the top teams will be invited to provide their training algorithms - not the trained algorithm, the one that’s finished, but the scripts that made it based on the input dataset . “ We’ll train their algorithms on an extended dataset of 9,000 cases, of which not everything was publicly available, ” Joeran explains. “ This will show us how well their algorithms scale up to a new dataset. We’ll be able to leverage all data that is feasible. Also, because we’re collaborating with Amazon, we’ll train each of the top five teams’ algorithms 10 times, so 50 runs in total, to estimate the variance between the training runs, and we’ll do a proper statistical analysis on that. Participants are evaluated on how good their training algorithms are rather than on something that’s come out of their pipeline . Then these training algorithms can be used in future work as well. ”