• 2024 (Vol.38)
  • 1990 (Vol.4)
  • 1989 (Vol.3)
  • 1988 (Vol.2)
  • 1987 (Vol.1)

Volume 36 #4

Contents

  1. L. N. Mogilev’s contribution to the research of the spatial vision mechanisms (to the 100th anniversary of the birth)
  2. Рrocessing and memorizing non-verbal non-meaning material when the exposure time is limited
  3. Оtoacoustic emission testing in 21-day dry immersion as prospects for intracranial pressure monitoring
  4. Modification of the method of detecting and describing keypoints SIFT for optical-to-SAR image registration

Modification of the method of detecting and describing keypoints SIFT for optical-to-SAR image registration

© 2022 V. V. Volkov

Moscow Institute of Physics and Technology, 141701 Dolgoprudny, Institutskiy per., 9, Russia
Institute for Information Transmission Problems (Kharkevich Institute) RAS, 127051 Moscow, Bolshoy Karetny per., 19, Russia

Received 28 Apr 2022

Image registration is the alignment (i.e. finding a general coordinate system) of two or more images of the same scene. Complex case of this task is the multimodal image registration, for example, optical-to-SAR. The need for such registration appeared in image fusion and object detection. One of the ways for image registration is to detect keypoints, calculate descriptor for each keypoint and form keypoints pairs by comparing descriptors. Further, the geometric transformation between two images is calculated by geometric model. In this paper modifications for the algorithm of detection SIFT keypoints was evaluated for optical-to-SAR image registration. Additionally, in this paper a modification of the SIFT descriptor was proposed and compared with SIFT descriptor. Image distortions is considered to be only shifts. All studies were carried out on a published dataset of 100 aligned pairs of optical-SAR images.

Key words: image registration, repeatability of keypoints, description of keypoints

DOI: 10.31857/S0235009222040060

Cite: Volkov V. V. Modifitsirovanie metoda poiska i deskribirovaniya ustoichivykh tochek sift dlya sopostavleniya opticheskikh i radiolokatsionnykh izobrazhenii [Modification of the method of detecting and describing keypoints sift for optical-to-sar image registration]. Sensornye sistemy [Sensory systems]. 2022. V. 36(4). P. 349–365 (in Russian). doi: 10.31857/S0235009222040060

References:

  • Volkov V.V., Shvets E.A. Dataset and Method for Evaluating Optical-to-Sar Image Registration Algorithms Based on Keypoints. Informacionnye tekhnologii i vychislitel’nye sistemy [Journal of Information Technologies and Computing Systems]. 2021. V. 2. P. 44–57. https://doi.org/10.14357/20718632210205 (in Russian).
  • Sidorchuk D.S., Volkov V.V. Fusion of radar, visible and thermal imagery with account for differences in brightness and chromaticity perception. Sensornye systemy [Sensory systems]. 2018. V. 32 (1). P. 14–18. https://doi.org/10.7868/S0235009218010031 (in Russian).
  • Tropin D.V., Shemiakina J.A., Konovalenko I.A., Faradjev I.A. Localization of planar objects on the images with complex structure of projective distortion. Informacionnye processy [Information processes]. 2019. V. 19 (2). P. 208–229 (in Russian).
  • Chekanov M.O., Shipitko O.S., Ershov E.I. 1-point RANSAC for axial rotation angle estimation by tomographic projections. Sensornye sistemy [Sensory systems]. 2020. V. 34 (1). P. 72–86. https://doi.org/10.31857/S0235009220010060 (in Russian).
  • Shemiakina J.A. Ispol’zovanie tochek i pryamyh dlya vychisleniya proektivnogo preobrazovaniya po dvum izobrazheniyam ploskogo ob"ekta [Using points and straight lines to calculate a projective transformation from two images of a flat object]. Informacionnye tekhnologii i vychislitel’nye sistemy [Journal of Information Technologies and Computing Systems]. 2017. V. 3. P. 79–91 (in Russian).
  • Bay H., Ess A., Tuytelaars T., Van Gool L. Speeded-up robust features (SURF). Computer vision and image understanding. 2008. V. 110. № 3. P. 346–359. https://doi.org/10.1016/j.cviu.2007.09.014
  • Chen M., Habib A., He H., Zhu Q., Zhang W. Robust feature matching method for SAR and optical images by using Gaussian-gamma-shaped bi-windows-based descriptor and geometric constraint. Remote Sensing. 2017. V. 9. № 9. P. 882. https://doi.org/10.3390/rs9090882
  • Copernicus Open Access Hub, Terms and Conditions. URL: https://scihub.copernicus.eu/twiki/do/view/ SciHubWebPortal/TermsConditions (дата обращения 18.04.2022).
  • Copernicus Open Access Hub. URL: https://scihub.copernicus.eu/ (дата обращения 18.04.2022).
  • Errico A., Angelino C.V., Cicala L., Persechino G., Ferrara C., Lega M., Vallario A., Parente C., Masi G., Gaetano R., Scarpa G. Detection of environmental hazards through the feature-based fusion of optical and SAR data: A case study in southern Italy. International Journal of Remote Sensing. 2015. V. 36. № 13. P. 3345–3367. https://doi.org/10.1080/01431161.2015.1054960
  • Fan B., Huo C., Pan C., Kong Q. Registration of optical and SAR satellite images by exploring the spatial relationship of the improved SIFT. IEEE Geoscience and Remote Sensing Letters. 2012. V. 10. № 4. P. 657–661. https://doi.org/10.1109/LGRS.2012.2216500
  • Fan J., Wu Y., Li M., Liang W., Cao Y. SAR and optical image registration using nonlinear diffusion and phase congruency structural descriptor. IEEE Transactions on Geoscience and Remote Sensing. 2018. V. 56. № 9. P. 5368–5379. https://doi.org/10.1109/TGRS.2018.2815523
  • Fischler M.A., Bolles R.C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM. 1981. V. 24. № 6. P. 381–395. https://doi.org/10.1145/358669.358692
  • Gong M., Zhao S., Jiao L., Tian D., Wang S. A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Transactions on Geoscience and Remote Sensing. 2013. V. 52. № 7. P. 4328–4338. https://doi.org/10.1109/TGRS.2013.2281391
  • Hamdi I., Tounsi Y., Benjelloun M., Nassim A. Evaluation of the change in synthetic aperture radar imaging using transfer learning and residual network. Computer Optics. 2021. V. 45 (4). P. 600–607. https://doi.org/10.18287/2412-6179-CO-814
  • Kunina I., Panfilova E., Gladkov A. Matching of SAR and optical images by independent referencing to vector map. In Eleventh International Conference on Machine Vision (ICMV 2018). 2019. V. 11041. https://doi.org/10.1117/12.2523132
  • Leutenegger S., Chli M., Siegwart R.Y. BRISK: Binary robust invariant scalable keypoints. In 2011 International conference on computer vision. 2011. P. 2548–2555. https://doi.org/10.1109/ICCV.2011.6126542
  • Li J., Hu Q., Ai M. RIFT: Multi-modal image matching based on radiation-variation insensitive feature transform. IEEE Transactions on Image Processing. 2019. V. 29. P. 3296–3310. https://doi.org/10.1109/TIP.2019.2959244
  • Lowe D.G. Distinctive image features from scale-invariant keypoints. International journal of computer vision. 2004. V. 60. № 2. P. 91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  • Ma W., Wen Z., Wu Y., Jiao L., Gong M., Zheng Y., Liu L. Remote sensing image registration with modified SIFT and enhanced feature matching. IEEE Geoscience and Remote Sensing Letters. 2016. V. 14. № 1. P. 3–7. https://doi.org/10.1109/LGRS.2016.2600858
  • Matas J., Chum O., Urban M., Pajdla T. Robust wide-baseline stereo from maximally stable extremal regions. Image and vision computing. 2004. V. 22. № 10. P. 761–767. https://doi.org/10.1016/j.imavis.2004.02.006
  • OpenCV: Main page. URL: https://docs.opencv.org/master/index.html (дата обращения 18.04.2022).
  • Paul S., Pati U.C. Automatic optical-to-SAR image registration using a structural descriptor. IET Image Processing. 2019. V. 14. № 1. P. 62–73. https://doi.org/10.1049/iet-ipr.2019.0389
  • Paul S., Pati U.C. Optical-to-SAR image registration using modified distinctive order based self-similarity operator. In 2018 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS). 2018. P. 1–5. https://doi.org/10.1109/SCEECS.2018.8546950
  • Plank S., Twele A., Martinis S. Landslide mapping in vegetated areas using change detection based on optical and polarimetric SAR data. Remote Sensing. 2016. V. 8. № 4. P. 307. https://doi.org/10.3390/rs8040307
  • Rosten E., Porter R., Drummond T. Faster and better: A machine learning approach to corner detection. IEEE transactions on pattern analysis and machine intelligence. 2008. V. 32. № 1. P. 105–119. https://doi.org/10.1109/TPAMI.2008.275
  • Schmid C., Mohr R., Bauckhage C. Evaluation of interest point detectors. International Journal of computer vision. 2000. V. 37. № 2. P. 151–172. https://doi.org/10.1023/A:1008199403446
  • Shi W., Su F., Wang R., Fan J. A visual circle based image registration algorithm for optical and SAR imagery. In 2012 IEEE International Geoscience and Remote Sensing Symposium. 2012. P. 2109–2112. https://doi.org/10.1109/IGARSS.2012.6351089
  • Skoryukina N., Faradjev I., Bulatov K., Arlazarov V.V. Impact of geometrical restrictions in RANSAC sampling on the ID document classification. In Twelfth International Conference on Machine Vision (ICMV 2019). 2020. V. 11433. https://doi.org/10.1117/12.2559306
  • Suri S., Reinartz P. Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. IEEE Transactions on Geoscience and Remote Sensing. 2009. V. 48. № 2. P. 939–949. https://doi.org/10.1109/TGRS.2009.2034842
  • Tropin D.V., Nikolaev D.P., Slugin D.G. The method of image alignment based on sharpness maximization. In Eleventh International Conference on Machine Vision (ICMV 2018). 2019. V. 11041. https://doi.org/10.1117/12.2522903
  • Wang H., Wang C., Li P., Chen Z., Cheng M., Luo L., Liu Y. Optical-to-SAR Image Registration Based On Gaussian Mixture Model. ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2012. V. 39. P. 179–183.
  • Wang L., Sun M., Liu J., Cao L., Ma G. A Robust Algorithm Based on Phase Congruency for Optical and SAR Image Registration in Suburban Areas. Remote Sensing. 2020. V. 12. № 20. P. 3339. https://doi.org/10.3390/rs12203339
  • Xiang Y., Wang F., You H. OS-SIFT: A robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas. IEEE Transactions on Geoscience and Remote Sensing. 2018. V. 56. № 6. P. 3078–3090. https://doi.org/10.1109/TGRS.2018.2790483
  • Xie Z., Liu J., Liu C., Zuo Y., Chen X. Optical and SAR Image Registration Using Complexity Analysis and Binary Descriptor in Suburban Areas. IEEE Geoscience and Remote Sensing Letters. 2021. V. 19. P. 1–5. https://doi.org/10.1109/LGRS.2021.3071870
  • Xiong X., Xu Q., Jin G., Zhang H., Gao X. Rank-Based Local Self-Similarity Descriptor for Optical-to-SAR Image Matching. IEEE Geoscience and Remote Sensing Letters. 2019. V. 17. № 10. P. 1742–1746. https://doi.org/10.1109/LGRS.2019.2955153
  • Xu C., Sui H., Li H., Liu J. An automatic optical and SAR image registration method with iterative level set segmentation and SIFT. International Journal of Remote Sensing. 2015. V. 36. № 15. P. 3997–4017. https://doi.org/10.1080/01431161.2015.1070321
  • Ye S.P., Chen C.X., Nedzved A., Jiang J. Building detection by local region features in SAR images. Computer Optics. 2020. V. 44. № 6. P. 944–950. https://doi.org/10.18287/2412-6179-CO-703
  • Yu H., Yang W., Liu Y. Coarse-to-fine accurate registration for airborne Sar images using SAR-FAST and DSPLATCH. Progress In Electromagnetics Research. 2018. V. 163. P. 89–106. https://doi.org/10.2528/PIER18070801
  • Yu Q., Ni D., Jiang Y., Yan Y., An J., Sun T. Universal SAR and optical image registration via a novel SIFT framework based on nonlinear diffusion and a polar spatialfrequency descriptor. ISPRS Journal of Photogrammetry and Remote Sensing. 2021. V. 171. P. 1–17. https://doi.org/10.1016/j.isprsjprs.2020.10.019