25 Marilyn Keller Computer Vision News Computer Vision News The limit of medical imaging is that they only show static bodies. So to learn where bones are inside posed bodies, she then used motion capture datasets and biomechanical models to create a paired dataset of people in motion with the skeleton inside. From these data, she built SKEL, a body model that given a pose and a shape parameter outputs a posed body and the corresponding skeleton. Contrary to common body models used in Computer Vision, like SMPL, SKEL has fewer and more anatomical degrees of freedom like the arm and leg flexion and the forearm supination. After working on bones, Marilyn turned to the challenge of predicting soft tissue layers, in particular subcutaneous adipose tissue (the fat layer under the skin), given the external body shape. We started by annotating MRI scans with the different tissues. Then, learning from these MRI required leveraging a statistical body model (SMPL) and an implicit representation of the different tissues, i.e. representing each tissue inside the body by an occupancy function defined on R3. The resulting method, which was called HIT, works as follows. Given a 3D body and a location x ∈ R3, the point x is warped to the average template body, and a Multi-Layer Perceptron is trained to predict the tissue at this location (adipose tissue, lean tissue like muscles and organs, and bones). Marilyn released the implementation code of each project and you can learn more on her website.
RkJQdWJsaXNoZXIy NTc3NzU=