• 2023 (Vol.37)
  • 1990 (Vol.4)
  • 1989 (Vol.3)
  • 1988 (Vol.2)
  • 1987 (Vol.1)

A convolutional neural network, robust to label noise for glioma segmentation

© 2020 T. N. Saparov, A. I. Kurmukov, B. N. Shirokih, S. V. Zolotova, A. V. Golovanov, M. G. Belyaev, A. V. Dalechina

Institute for Information Transmission Problems of the Russian Academy of Sciences (Kharkevich Institute), 127994 Moscow, Bol’shoy Karetny Pereulok, 19, Russian
Moscow Institute of Physics and Technology, 141701 Dolgoprudny, Moscow Region, Institutskiy Pereulok, 9, Russian
Higher School of Economics – National Research University, 101000 Moscow, Myasnitskaya Ulitsa, 20, Russian
Skolkovo Institute of Science and Technology, 143026 Moscow, Bol’shoy Bul’var, 30, Russian
Federal State Autonomous Institution “N.N. Burdenko National Medical Research Center of Neurosurgery” of the Ministry of Health of the Russian Federation, 125047 Moscow, 4-Ya Tverskaya-Yamskaya Ulitsa, 16, Russian
Moscow Gamma Knife Center, Joint Stock Company “Neurosurgery Business Center”, 125047 Moscow, 1-Y Tverskoy-Yamskoy Pereulok, 13/5, Russian

Received 05 Jun 2020

Medical image segmentation is one of the most important tasks in radiotherapy. Recently, deep neural networks have achieved great success in solving a wide variety of medical image segmentation tasks using standardized or/and hardly preprocessed data. However, with typical clinical data, the situation drastically changes because of data complexity. For example, in radiotherapy tasks, data are much more variable in sizes and pixel intensities depending on a particular MRI machine. This heterogeneity leads to high intra- and interrater variability in tumor delineation. Finally, tumor contours might not coincide with pathological regions on the MRI image due to complex delineation procedures. Thus, the annotations in the existing datasets are relatively noisy. We propose a robust learning procedure based on the modern Convolutional Neural Network architecture. Our extension uses a separate module with learnable parameters, which outputs weights for the weighted cross-entropy. In our experiments, we achieved an impressive 38% boost in the Dice score using the data with high variability and target noise.

Key words: medical image segmentation, MRI, deep learning, high grade glioma

DOI: 10.31857/S0235009220040071

Cite: Saparov T. N., Kurmukov A. I., Shirokih B. N., Zolotova S. V., Golovanov A. V., Belyaev M. G., Dalechina A. V. Ustoichivaya k shumu v razmetke svertochnaya neironnaya set v zadache segmentatsii gliom na mrt izobrazheniyakh [A convolutional neural network, robust to label noise for glioma segmentation]. Sensornye sistemy [Sensory systems]. 2020. V. 34(4). P. 329–339 (in Russian). doi: 10.31857/S0235009220040071


  • Algan G., Ulusoy I. Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey. https://arxiv.org/pdf/1912.05170.pdf (accessed 01.06.2020)
  • Bakas S., Reyes M., Jakab A., Bauer S., Rempfler M. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. https://arxiv.org/pdf/1811.02629.pdf (accessed 05.11.2018)
  • Chen C., Qin C., Qui H., Tarroni G., Duan J. Deep learning for cardiac image segmentation: A review. Frontiers in Cardiovascular Medicine. 2020. V. 7. P. 25. https://doi.org/10.3389/fcvm.2020.00025
  • Dvorak P., Menze B. Structured prediction with convolutional neural networks for multimodal brain tumor segmentation. Proceeding of the multimodal brain tumor image segmentation challenge. 2015. P. 13–24.
  • Frénay B., Verleysen M. Classification in the presence of label noise: a survey. IEEE transactions on neural networks and learning systems. 2013. V. 25(5). P. 845–869. https://doi.org/10.1109/tnnls.2013.2292894
  • García-Lorenzo D., Francis S., Narayanan S., Arnold D.L., Collins D.L. Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging. Medical image analysis. 2013. V. 17(1). P. 1–18. https://doi.org/10.1016/j.media.2012.09.004
  • Growcott S., Dembrey T., Patel R., Eaton D., Cameron A. Inter-Observer Variability in Target Volume Delineations of Benign and Metastatic Brain Tumours for Stereotactic Radiosurgery: Results of a National Quality Assurance Programme. Clinical Oncology. 2020. V. 32 (1). P. 13–25. https://doi.org/10.1016/j.clon.2019.06.015
  • Hesamian M.H., Jia W., He X., Kennedy P. Deep learning techniques for medical image segmentation: Achievements and challenges. Journal of digital imaging. 2019. V. 32 (4). P. 582–596. https://doi.org/10.1007/s10278-019-00227-x
  • Karimi D., Dou H., Warfield S.K., Gholipour A. Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. https://arxiv.org/pdf/1912.02911.pdf (accessed 01.06.2020)
  • Kelly C.J., Karthikesalingam A., Suleyman M., Corrado G., King D. Key challenges for delivering clinical impact with artificial intelligence. BMC medicine. 2019. V. 17 (1). P. 195. https://doi.org/10.1186/s12916-019-1426-2
  • Kickingereder P., Isensee F., Tursunova I., Petersen J., Neuberger U. Automated quantitative tumour response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study. The Lancet Oncology. 2019. V. 20 (5). P. 728–740. https://doi.org/10.1016/s1470-2045(19)30098-1
  • Kim S., Bae W.C., Masuda K., Chung C.B., Hwang D. Fine-grain segmentation of the intervertebral discs from MR spine images using deep convolutional neural networks: BSU-Net. Applied Sciences. 2018. V. 8 (9). P.1656. https://doi.org/10.3390/app8091656
  • Lustberg T., van Soest J., Jochems A., Deist T., van Wijk Y. Big Data in radiation therapy: challenges and opportunities. The British journal of radiology. 2017. V. 90 (1069). 20160689. https://doi.org/10.1259/bjr.20160689
  • Mirikharaji Z., Yan Y., Hamarneh G. Learning to segment skin lesions from noisy annotations. Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data. Cham. Springer. 2019. P. 207–215. https://doi.org/10.1007/978-3-030-33391-1_24
  • Nie D., Gao Y., Wang L., Shen D. Asdnet: Attention based semi-supervised deep networks for medical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham. Springer. 2018. P. 370–378. https://doi.org/10.1007/978-3-030-00937-3_43
  • Otsu N. A threshold selection method from gray-level histograms. IEEE transactions on systems, man, and cybernetics. 1979. V. 9 (1). P. 62–66. https://doi.org/10.1109/tsmc.1979.4310076
  • Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computerassisted intervention. Cham. Springer. 2015. P. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
  • Sahiner B., Pezeshk A., Hadjiiski L.M., Wang X., Drukker K. Deep learning in medical imaging and radiation therapy. Medical physics. 2019. V. 46 (1). P. e1–e36. https://doi.org/10.1002/mp.13264
  • Sandström H., Jokura H., Chun C., Toma-Dasu I. Multiinstitutional study of the variability in target delineation for six targets commonly treated with radiosurgery. Acta Oncologica. 2018. V. 57 (11). P. 1515–1520. https://doi.org/10.1080/0284186x.2018.1473636
  • Tajbakhsh N., Jeyaseelan L., Li Q., Chiang J.N., Wu Z. Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Medical Image Analysis. 2020. 101693. https://doi.org/10.1016/j.media.2020.101693
  • Tanaka D., Ikami D., Yamasaki T., Aizawa K. Joint optimization framework for learning with noisy labels. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. P. 5552–5560. https://doi.org/10.1109/CVPR.2018.00582
  • Vinod S.K., Jameson M.G., Min M., Holloway L.C. Uncertainties in volume delineation in radiation oncology: a systematic review and recommendations for future studies. Radiotherapy and Oncology. 2016. V. 121 (2). P. 169–179. https://doi.org/10.1016/j.radonc.2016.09.009
  • Wang C., Zhu X., Hong J.C., Zheng D. Artificial intelligence in radiotherapy treatment planning: present and future. Technology in cancer research & treatment. 2019. V. 18. https://doi.org/10.1177/1533033819873922
  • Xue C., Dou Q., Shi X., Chen H., Heng P.A. Robust learning at noisy labeled medical images: applied to skin lesion classification. IEEE 16th International Symposium on Biomedical Imaging. IEEE. 2019. P. 1280–1283. https://doi.org/10.1109/isbi.2019.8759203
  • Zeng Q., Karimi D., Pang E.H.T., Mohammed S., Schneider C. Liver Segmentation in Magnetic Resonance Imaging via Mean Shape Fitting with Fully Convolutional Neural Networks. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham. Springer. 2019. P. 246–254. https://doi.org/10.1007/978-3-030-32245-8_28
  • Zhang C., Bengio S., Hardt M., Recht B., Vinyals O. Understanding deep learning requires rethinking generalization. 5th International Conference on Learning Representations, Conference Track Proceedings. Toulon. OpenReview.net. 2017a.
  • Zhang Y., Yang L., Chen J., Fredericksen M., Hughes D.P. Deep adversarial networks for biomedical image segmentation utilizing unannotated images. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham. Springer. 2017b. P. 408–416. https://doi.org/10.1007/978-3-319-66179-7_47
  • Zhu H., Shi J., Wu J. Pick-and-learn: Automatic quality evaluation for noisy-labeled image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham. Springer. 2019. P. 576–584. https://doi.org/10.1007/978-3-030-32226-7_64
  • Zhao F., Li M., Kong L., Zhang G., Yu J. Delineation of radiation therapy target volumes for patients with postoperative glioblastoma: A review. Onco Targets Ther. 2016. V. 9. P. 3197–3204. https://doi.org/10.2147/ott.s104241