Computer Vision News - October 2022

38 Poster Presentation this look reasonable to you?’ to inform our decision process.” This process is not unique to machine learning but used inmany fields that feature causality and counterfactuals, such as econometrics and epidemiology. It is always important to be careful and strict about any assumptions, how they are made, and how they are implemented. Results could be wrong, but theoretical constraints and expert knowledge can help mitigate this. Did anything go wrong? Or did they see any surprising results? “We were curious to see how the model would perform under extreme conditions, so we did produce some videos where, in real life, the patient would be dead!” Thanos laughs nervously. “The video itself was able in terms of how the heart would react if the ejection fraction were, for example, 10%. However, a human being cannot handle a 10% ejection fraction, but that’s a completely different story.” and we knew how to make a network that would evaluate the ejection fraction in the counterfactual video, so we could backpropagate the loss through this expert network to enforce the ejection fraction.” On the theoretical side, the problem was that, by definition, the counterfactual was an alternative world they did not have access to but wanted to contemplate. Even figuring out a video would look a certain way, they had no way to tell if it was true. They had to apply some stringent conditions to allow them to make such claims. “Those conditions are essentially called identifiability conditions,” Thanos explains. “The question here is, can we identify the correct counterfactual from the data we observe? There are an enormous number of potential counterfactuals. In our case, because the echocardiograms and the videos were difficult to identify, we used a combination of theoretical constraints, plus we went to cardiologists and said, ‘Does This shows 3 triplets of echocardiograms: the input (confounder) and both outputs (factual and counterfactual). It demonstrates that the anatomy is preserved while the ejection faction changes (the amplitude of the movement is different). BES MI