Applications of deformable object reconstruction can be found in several fields: one of those is clinics, where monitoring the state of an organ by CT, angiography or x-ray can provide important prognostic insight before invasive procedure are engaged. Furthermore, once a candidate surface has been reconstructed to represent the target organ, a virtual model can be used for real-time navigation, where catheters or other equipment is tracked and its 3D position is overlaid on the reconstructed model. Physicians can then navigate in an otherwise occluded environment inside the body, increasing operation accuracy and possibly preventing injuries caused by equipment manipulation.
Three-dimensional reconstruction of an object
The three-dimensional reconstruction of a deformable object includes two main parts. The first involves tracking, feature matching or the establishment of temporal link between successive points on the surface. The second part includes surface representation of the target object. The trajectories tracked can be used for learning the temporal relationship between the set of points, which produces a mathematical representation of their motion. The importance of learning these trajectories lies in the ability to put them into use for motion predictor-correctors systems such as the Kalman filters. Doing so, we can pose restriction on the global motion (envelope) deformation of the object. This dynamic point cloud is then used for surface fitting. The surface representing the object should be flexible to take many forms, but not violate too strongly the deformation restrictions. Surface matching smooths out irregularities in the point cloud and enables a clearer view of the object.
The model to be fitted to the object is oftentimes learned from a template database, which is constructed offline. Using an object database, multiple templates can be created, enabling us to learn the shape space. Furthermore, the template database allows us to extract features related to object’s appearance and its deformation constraints. This knowledge can be put to use when reconstructing an object in an occluded environment, where, as is many times the case, we have to isolate the target object from its surrounding.
The technology leading to surface reconstruction of rigid objects with predictable shape from a sequence of images is quite advanced. However, this is not the same for deformable objects. The three dimensional reconstruction of an object demands a careful construction of the deformation shape space and an accurate mathematical model to represent its motion. Additional steps involving feature extraction and image pre-processing are essential to assure correct surface representation. These challenges are yet to be solved with an off-the-shelf technique and therefore require custom development.
RSIP Vision specializes in constructing tailor-made algorithmic solutions for image processing, computer vision and machine learning tasks. We have developed the expertise to handle reconstruction challenges from data acquisition to 3D representation. You will find in RSIP Vision’s project page much more information about our customized solutions, including our work and research in 3D reconstruction.
