Volume 35 #2

Contents

1. Neurophysiological support of motor control in “Hybrid” positions. Literature review
2. Functional investigations of the primary auditory cortex in the cat
3. Analysis of individual strategies in categorical visual search
4. Effect of auditory LTP-like stimulation on the processing of sounds
5. Gradient algorithm for raindrop detection on a sequence of images
6. Local path finding in autonomous car state lattice based on multi-criteria optimization

Autonomous vehicles use cameras as one of the primary sources of information about the environment. Weather conditions and other external factors, such as raindrops, snow, mud, and others, can lead to various image artifacts. Such artifacts significantly degrade the quality and reliability of the obtained visual data and can lead to accidents in case they are not detected in time. Artifacts detection algorithms should meet high requirements: be able to work in real-time, as well as work with limited computing and power resources. In this paper, we propose a new algorithm for detecting raindrops on a camera lens, based on averaging the gradient maps of an image sequence. To test the algorithm, a set of frame sequences, taken by a camera fixed on the car while driving, was collected. Three types of image sequences were collected: with real raindrops, without raindrops, and with artificially generated drops. The problem of detecting raindrops was considered as a problem of binary classification of image sequences. So, we use AUC-ROC (area under the receiver operating characteristic curve) as a quality metric. The results of testing the algorithm show that it reliably detects raindrops, both artificial and real. Moreover, the proposed algorithm, in comparison with the existing one based on pixel-wise cross-correlation (Einecke et al., 2014), showed a higher quality of sequence classification and image processing speed. Thus, the algorithm based on the gradient map provides high-quality raindrops detection on a sequence of images and its computational efficiency allows using it as a self-checking procedure in autonomous visual systems.

Key words: gradient map, raindrop detection, artifact detection, autonomous vehicle, image sequence, selfchecking procedure, autonomous visual systems, image artifacts

DOI: 10.31857/S0235009221020049

References:

• Kunina I.A., Panfilova E.I., Povolotskiy M.A. Detektirovaniye peshekhodnykh perekhodov na izobrazheniyakh dorogi na osnove metoda dinamicheskogo vyravnivaniya vremennykh ryadov [Zebra-crossing detection on road images using dynamic time warping]. Proc. Instit. systems analysis RAS. 2018. V. 68. № S1. P. 23–31(in Russian). https://doi.org/10.14357/20790279180503
• Kurnikov P.A., Sholomov D.L., Panchenko A.V. Sistema opredeleniya tumannykh dorozhnykh stsen, osnovannaya na ansamble klassifikatorov [The system for foggy road scenes detection based on the ensembles of classifiers]. J. Inform. Technol. Comput. Systems. 2018. V. 2. № 2. P. 70–77 (in Russian). https://doi.org/10.14357/20718632180206
• Minaev E.Y., Nikonorov A.V. Detektirovaniye i raspoznavaniye obyektov v sistemakh avtomobilnoy bezopasnosti na osnove fraktalnogo analiza [Object detection and recognition in the driver assistance system based on the fractal analysis]. Computer optics. 2012. V. 36. № 1. P. 124–130 (in Russian).
• Tsaruk V.B. Vydelenie iskazhenii, vnosimykh atmosfernymi osadkami na videoizobrazheniya [Weather effects detection in video]. Aktual’nye problemy aviatsii i kosmonavtiki. 2018. V. 2. № 4 (14). P. 176–178 (in Russian).
• Akkala V., Parikh P., Mahesh B.S., Deshmukh A.S., Medasani S. Lens adhering contaminant detection using spatio-temporal blur. Intern. Conf. Signal Proc. Commun (SPCOM). 2016. P. 1–5. https://doi.org/10.1109/SPCOM.2016.7746664
• Bae I., Bang S., Kim W. Convolutional neural network for raindrop detection. 16th Intern. Conf. Ubiquitous Robots (UR). 2019. P. 102–105. https://doi.org/10.1109/URAI.2019.8768613.
• Cord A., Aubert D. Towards rain detection through use of in-vehicle multipurpose cameras. IEEE Intelligent Vehicles Sympos. (IV). 2011. P. 833–838. https://doi.org/10.1109/IVS.2011.5940484
• Einecke N., Gandhi H., Deigmöller J. Detection of camera artifacts from camera images. 17th Intern. IEEE Conf. Intelligent Transportation Systems (ITSC). 2014. P. 603–610. https://doi.org/10.1109/ITSC.2014.6957756.
• Gladkov A., Gladilin S., Ershov E. Comparison of stereo matching algorithms for obstacle detection and collision avoidance. Eleventh Intern. Conf. Machine Vision (ICMV). 2019. P. 92. https://doi.org/10.1117/12.2523116
• Gu J., Belhumeur P., Nayar S., Ramamoorthi R. Removing image artifacts due to dirty camera lenses and thin occluders. ACM Transactions on Graphics. 2009. V. 28. P. 1–10. https://doi.org/10.1145/1618452.1618490
• Hirohashi Y., Narioka K., Suganuma M., Liu X., Tamatsu Y. and Okatani T. Removal of image obstacles for vehicle-mounted surrounding monitoring cameras by real-time video inpainting. IEEE/CVF Conf. Computer Visi. Pattern Recogn. Workshops (CVPRW). 2020. P. 857–866. https://doi.org/10.1109/CVPRW50498.2020.00115
• Hu L., Chen L., Cheng J. Gray spot detection in surveillance video using convolutional neural network. 13th IEEE Conf. Industrial Electron. Applicat. (ICIEA). 2018. P. 2806–2810. https://doi.org/10.1109/ICIEA.2018.8398187
• Ivanov A., Yudin D. Visibility loss detection for video camera using deep convolutional neural networks. Proc. Third Intern. Sci. Conf. “Intelligent Information Technologies for Industry” (IITI’18). 2019. V. 1. P. 434–443. https://doi.org/10.1007/978-3-030-01818-4_43
• Korobov N., Shipitko O., Konovalenko I., Grigoryev A., Chukalina M. SWaP-C Based Comparison of Onboard Computers for Unmanned Vehicles. Proc. 14th Intern. Conf. Electromechanics and Robotics “Zavalishin’s Readings”. Springer, Singapore. 2020. P. 573–583.
• McDonnell M.J. Box-filtering techniques. Computer Graphics and Image Processing. 1981. V. 17. P. 65–70. https://doi.org/10.1016/S0146-664X(81)80009-3
• Nashashibi F., Charette R., Lia A. Detection of unfocused raindrops on a windscreen using low level image processing. Intern. Conf. Control, Automat. Robot. Vision: 2010. P. 1410–1415. https://doi.org/10.1109/ICARCV.2010.5707398
• Qian R., Tan T., Yang W. Attentive generative adversarial network for raindrop removal from a single image. IEEE/CVF Conf. Comp. Vis. Pattern Recogn. 2018. P. 2482–2491. https://doi.org/10.1109/CVPR.2018.00263
• Roser M., Geiger A. Video-based raindrop detection for improved image registration. IEEE 12th Intern. Conf. Computer Vision Workshops. 2009. P. 570–577. https://doi.org/10.1109/ICCVW.2009.5457650
• Sobel I., Feldman G. A 3 × 3 isotropic gradient operator for image processing. Pattern Classification and Scene Analysis. 1973. P. 271–272.
• You S., Tan R.T., Kawakami R. Adherent raindrop modeling, detection and removal. IEEE Transact. Pattern Analys. Machine Intellig. 2016. V. 38. № 9. P. 1721–1733. https://doi.org/10.1109/TPAMI.2015.2491937