Computer Vision News - December 2022

14 Best Student Paper IJCLR high accuracy on every performance metric, but the neural network would sometimes tell her the survival probability at time T+1 was much higher than at time T, which did not make sense from a human perspective. She needed a way to tell the neural network it was doing something she knew was wrong . “ The main action detection pipeline is very much rooted in computer vision, ” Eleonora explains. “ We use the 3D-RetinaNet model . I still believe we need a data- driven, computer vision-based approach to do the basic detections. Then you can bring in some reasoning. We included the requirements in the loss function, or we used the MaxSat solver to correct the predictions at post-processing time. It’s very much a combination of computer vision and reasoning. ” Does all this mean that the papers we were celebrating two or three years ago were built on an inaccurate basis? Not At first, she anticipated that applying the constraints in a way that minimally changed the neural network prediction would not negatively impact the performance. However, it turned out that adding these requirements naively can hurt performance, whereas learning to use them smartly helps teach the neural networkandcanimproveitsperformance . “ Until a few years ago, it was felt that it was enough to have very high accuracy to avoid these problems, ” she points out. “ At conferences, people told me that if we have 99.9% accuracy, then neural networks won’t make these stupid mistakes. But then neural networks started to be applied to more complex problems and safety- critical scenarios where even a single error can make a big difference . ” The basic idea behind this work grew from Eleonora’s master’s thesis when she was working on healthcare data and predicting survival curves. She achieved Percentage of predictions that violate at least one requirement when varying the decision threshold. For each label, neural networks return a value in [0,1], and a label is predicted if the corresponding output is greater that the decision threshold (often such decision threshold is set equal to 0.5)