Deep learning based 3-dimensional liver motion estimation using 2-dimensional ultrasound images

Purpose: Radiofrequency ablation therapy support system aims to display the true tumor position by tracking the tumor during the treatment. This sys-tem utilizes 2-dimensional ultrasound images. As a major cause of the tracking error, the tumor moves in the direction which is perpendicular to the scan plane of the ultra-sound image. Methods: To cope with this, in the proposed method, the six-axis movement amount is estimated from two 2-dimensional ultrasound images by a convolution neural network. In addition, we tried to improve the processing speed by using multitask learning. The liver moves differently from other tissues in the ultra-sound image. In particular, in order to improve the accuracy, a heatmap as a result of segmentation estimation of the liver region was utilized for estimating the amount of movement. By this method, we could estimate the amount of movement in the direction which is perpendicular to the ultrasound image plane so as to track the organ motion more precisely and robustly. We conducted an experiment to confirm the estimating performance of the amount of probe movement relative to the liver phantom, and evaluated estimation accuracy and measured processing speed. Results: The number of the used image sets was 9,506 for the learning and 1,337 for the evaluation. Conclusion: The experimental results show the effectiveness of our novel method. This article does not contain any studies with human participants or animals performed by any of the authors.