• 1990 (Vol.4)
  • 1989 (Vol.3)
  • 1988 (Vol.2)
  • 1987 (Vol.1)

Volume 38 #3

Contents

  1. AUDITORY SYSTEM OF BLOOD-SUCKING MOSQUITOES (DIPTERA, CULICIDAE)
  2. DEVELOPMENT OF IMAGE PRE-PROCESSING METHODS FOR SOFTWARE COMPENSATION OF ANOMAL REFRACTION OF THE OBSERVER’S EYES
  3. PERCEIVED TRAJECTORIES OF CYCLIC SOUND MOVEMENT
  4. ABOUT THE POSSIBILITY OF USING FIXATION MICROSACCADES TO IMPROVE A QUALITY OF VISIBLE IMAGES IN THE FOVEAL ZONE
  5. METHODOLOGICAL RECOMMENDATIONS FOR THE CREATION OF SENSOR MEASUREMENT SYSTEMS FOR RESPIRATORY RATE MONITORING BASED ON PHOTOPLETHYSMOGRAPHIC SIGNAL PROCESSING

DEVELOPMENT OF IMAGE PRE-PROCESSING METHODS FOR SOFTWARE COMPENSATION OF ANOMAL REFRACTION OF THE OBSERVER’S EYES

© 2024 N. B. Alkzir, M. S. Yarykina, D. P. Nikolaev, I. P. Nikolaev

HSE University, 20 Myasnitskaya Ulitsa, Moscow, 101000, Russia
Institute for Information Transmission Problems, RAS, Bolshoy Karetny per., 19, build. 1, Moscow, 127051, Russia
Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences. Prospekt 60 let Oktyabrya, Moscow, 117312, Russia
Smart Engines Service LLC, 117312, Moscow, Russia

Received 20 Apr 2024

In recent decades, the practice of demonstrating various static and video images to users using digital, processorcontrolled, most often self-luminous devices (computer monitors, smartphone and tablet screens, etc.) has spurred the development of various methods to improve the perception of such images by means of computerized image preprocessing. This also applies to methods of preprocessing images shown to users with various refractive anomalies of the eye(s) (e.g., myopia or astigmatism) in situations where they are not armed with glasses or other corrective devices. Over the past 20+ years, researchers have published dozens of papers on this task, referred to as the precompensation task. In our opinion, the time has come to reflect on the development of scientific thought in this direction and to highlight the most important milestones in realizing the problems on the way to achieving “ideal” precompensation and in approaches to their successful solution. This is the focus of the first part of this review. In the second part, we focus on the current state of research in the stated area, highlight the problems not solved so far, and try to catch the trends of further development of image precompensation methods, paying maximum attention to neural network approaches.

Key words: Image precompensation, Wiener filtering, refractive error, tone mapping, neural network, image deconvolution

DOI: 10.31857/S0235009224030027  EDN: BSFLPC

Cite: Alkzir N. B., Yarykina M. S., Nikolaev D. P., Nikolaev I. P. Razvitie metodov predvaritelnoi obrabotki izobrazhenii dlya programmnoi kompensatsii anomalii refraktsii glaz nablyudatelya [Development of image pre-processing methods for software compensation of anomal refraction of the observer’s eyes]. Sensornye sistemy [Sensory systems]. 2024. V. 38(3). P. 31–50 (in Russian). doi: 10.31857/S0235009224030027

References:

  • Yablokov M. G., Machekhin V. A., Doga A. V., Kolotov M. G., Vartapetov S. K., Larichev A. V., Iroshnikov N. G. Results of wavefront studies on the first Russia aberrometer “Multispot-250”. Oftal’mokhirurgiya. 2005. V. 2. P. 4–8. (In Russian).
  • Agarwal C., Khobahi S., Bose A., Soltanalian M., Schonfeld D. Deep-URL: A model-aware approach to blind deconvolution based on deep unfolded Richardson-Lucy network. 2020 IEEE international conference on image processing (ICIP). 2020. P. 3299 3303. DOI: 10.1109/ICIP40778.2020.9190825
  • Alkzir N., Nikolaev I., Nikolaev D. SCA-2023: A two-part dataset for benchmarking the methods of image precompensation for users with refractive errors. ECMS. 2023. P. 298–305. DOI: 10.7148/2023-0298
  • Alkzir N. B., Nikolaev I. P., Nikolaev D. P. Search for image quality metrics suitable for assessing images specially precompensated for users with refractive errors. Sixteenth international conference on machine vision (ICMV 2023). 2024. V. 13072. P. 230–238. DOI: 10.1117/12.3023509
  • Alonso M. Jr, Barreto A., Cremades J. G., Jacko J. A., Adjouadi M. Image pre-compensation to facilitate computer access for users with refractive errors. Behaviour Information Technology. 2005a. 24(3). P. 161–173. DOI: 10.1080/01449290412331327456
  • Alonso M. Jr, Barreto A., Jacko J. A., Adjouadi M., Choudhury M. Improving computer interaction for users with visual acuity deficiencies through inverse point spread function processing. Proceedings IEEE southeastcon. 2005b. P. 421–427. DOI: 10.1109/SECON.2005.1423281
  • Alonso M. Jr, Barreto A. B. Pre-compensation for high-order aberrations of the human eye using on-screen image deconvolution. Proceedings of the 25th annual international conference of the IEEE engineering in medicine and biology society (IEEE cat. no. 03ch37439). 2003. V. 1. P. 556–559. DOI: 10.1109/IEMBS.2003.1279804
  • Alonso M. Jr, Barreto A., Adjouadi M. Digital image inverse filtering for improving visual acuity for computer users with visual aberrations. Inverse Problems in Science and Engineering. 2008. V. 16(8). P. 957–966. DOI: 10.1080/17415970802082823
  • Campbell C. E. Matrix method to find a new set of Zernike coefficients from an original set when the aperture radius is changed. JOSA A. 2003. V. 20(2). P. 209–217. DOI: 10.1364/JOSAA.20.000209
  • Cascarano P., Sebastiani A., Comes M. C., Franchini G., Porta F. Combining weighted total variation and deep image prior for natural and medical image restoration via admm. I2021 21st international conference on computational science and its applications (ICCSA). 2021. P. 39–46. DOI: 10.1109/ICCSA54496.2021.00016
  • Chaganova O., Grigoryev A., Nikolaev D., Nikolaev I. Applied aspects of modern non-blind image deconvolution methods. Компьютерная оптика. 2024. V. 48(4). DOI: 10.18287/2412-6179-CO-1409
  • Dong J., Roth S., Schiele B. Deep wiener deconvolution: Wiener meets deep learning for image deblurring. Advances in Neural Information Processing Systems. 2020. V. 33. P. 1048–1059.
  • Fernández E. J. Adaptive optics for visual simulation. International Scholarly Research Notices. 2012. V. 2012(1). P. 104870. DOI: 10.5402/2012/104870
  • Fine E. M., Peli E. Enhancement of text for the visually impaired. JOSA A. 1995. V. 12(7). P. 1439–1447. DOI: 10.1364/JOSAA.12.001439
  • Gong D., Zhang Z., Shi Q., van den Hengel A., Shen C., Zhang Y. Learning deep gradient descent optimization for image deconvolution. IEEE transactions on neural networks and learning systems. 2020. V. 31 (12). P. 5468–5482. DOI: 10.1109/TNNLS.2020.2968289
  • Goodman J. W. Introduction to Fourier optics. McGraw-Hill, 1968.
  • Güzel A. H., Beyazian J., Chakravarthula P., AKS, it K. Chromacorrect: prescription correction in virtual reality headsets through perceptual guidance. Biomedical Optics Express. 2023. V. 14(5). P. 2166–2180. DOI: 10.1364/BOE.485776
  • Holden B. A., Fricke T. R., Wilson D. A., Jong M., Naidoo K. S., Sankaridurg P., Wong T. Y., Naduvilath T. J., Resnikoff S. Global prevalence of myopia and high myopia and temporal trends from 2000 through 2050. Ophthalmology. 2016. V. 123(5). P. 1036–1042. DOI: 10.1016/j.ophtha.2016.01.006
  • Huang F.-C., Lanman D., Barsky B. A., Raskar R. Correcting for optical aberrations using multilayer displays. ACM transactions on graphics (TOG). 2012a. V. 31(6). P. 1–12. DOI: 10.1145/2366145.2366204
  • Huang J., Barreto A., Adjouadi M. Dynamic image pre-compensation for computer access by individuals with ocular aberrations. 2012 annual international conference of the IEEE engineering in medicine and biology society. 2012c. P. 3320–3323. DOI: 10.1109/EMBC.2012.6346675
  • Huang J., Barreto A., Adjouadi M. Evaluation of dynamic image pre-compensation forcomputer users with severe refractive error. Proceedings of the 14th international ACM SIGACCESS conference on Сomputers and accessibility. 2012b. P. 175–182. DOI: 10.1145/2384916.238494
  • Huang J., Barreto A., Alonso M. Jr, Adjouadi M. Vision correction for computer users based on image pre-compensation with changing pupil size. 2011 annual international conference of the IEEE engineering in medicine and biology society. 2011. P. 4868–4871. DOI: 10.1109/IEMBS.2011.6091206
  • Huang J., Barreto A., Alonso M. Jr, Adjouadi M. Contrast improvement in pre-compensation of ocular aberrations for computer users. Biomedical sciences instrumentation. 2012d. V. 48. P. 179–186. DOI: 10.1007/978-1-4614-3535-8_32
  • Ji Y., Ye J., Bing Kang S., Yu J. Image pre-compensation: Balancing contrast and ringing. Proceedings of the IEEE conference on computer vision and pattern recognition. 2014. P. 3350–3357. DOI: 10.1109/CVPR.2014.428
  • Jumbo O. E., Asfour S., Sayed A. M., Abdel-Mottaleb M. Correcting higher order aberrations using image processing. IEEE Transactions on Image Processing. 2021. V. 30. P. 2276–2287. DOI: 10.1109/TIP.2021.3051499
  • Krishnan D., Fergus R. Fast image deconvolution using hyper-Laplacian priors. Advances in neural information processing Systems. 2009. V. 22. https://proceedings.neurips.cc/paper/2009/file/3dd48ab31d016ffcbf3314df2b3cb9ce-Paper.pdf DOI:10.1145/1531326.1531402
  • Lawton T. B. Improved word recognition for observers with age- related maculopathies using compensation filters. Clinical Vision Sciences. 1988. V. 3(2, 19). P. 125–135. https://ntrs.nasa.gov/citations/19890037275
  • Lawton T. B. Improved reading performance using individualized compensation filters for observers with losses in central vision. Ophthalmology. 1989. V. 96(1). P. 115–126. DOI: 10.1016/S0161-6420(89)32935-6
  • Lawton T. B. Image enhancement filters significantly improve reading performance for low vision observers. Ophthalmic and Physiological Optics. 1992. V. 12(2). P. 193–200. DOI: 10.1111/j.1475-1313.1992.tb00289.x
  • Lucy L. B. An iterative technique for the rectification of observed distributions. Astronomical Journal. 1974. V. 79. P. 745. DOI: 10.1086/111605
  • Mohammadpour S., Mehridehnavi A., Rabbani H., Lakshminarayanan V. A pre-compensation algorithm for different optical aberrations using an enhanced wiener filter and edge tapering. 2012 11th international conference on information science, signal processing and their applications (ISSPA). 2012. P. 935–939. DOI: 10.1109/ISSPA.2012.6310689
  • Montalto C., Garcia-Dorado I., Aliaga D., Oliveira M. M., Meng F. A total variation approach for customizing imagery to improve visual acuity. ACM Transactions on Graphics (TOG). 2015. V. 34(3). P. 1–16. DOI: 10.1145/2717307
  • Mou C., Wang Q., Zhang J. Deep generalized unfolding networks for image restoration. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. P. 17399–17410. DOI: 10.48550/arXiv.2204.13348
  • Nan Y., Ji H. Deep learning for handling kernel/model uncertainty in image deconvolution. Proceedings of the IEEE/ CVF conference on computer vision and pattern recognition. 2020. P. 2388– 2397. DOI: 10.1109/CVPR42600.2020.00246
  • Peli E., Goldstein R. B., Young G. M., Trempe C. L., Buzney S. M. Image enhancement for the visually impaired. simulations and experimental results. Investigative ophthalmology visual science. 1991. V.32 (8). P. 2337–2350. DOI: 10.1109/NEBC.1989.36690
  • Peli E., Lee E., Trempe C. L., Buzney S. Image enhancement for the visually impaired: the effects of enhancement on face recognition. JOSA A. 1994. V. 11(7). P. 1929–1939. DOI: 10.1364/JOSAA.11.001929
  • Peli E., Peli T. Image enhancement for the visually impaired. Optical engineering. 1984. V. 23(1). P. 47–51. DOI: 10.1117/12.7973251
  • Peli T., Lim J. S. Adaptive filtering for image enhancement. Optical Engineering. 1982. V. 21(1). P. 108–112. DOI: 10.1117/12.7972868
  • Richardson W. H. Bayesian-based iterative method of image restoration. JOSA. 1972. V. 62(1). P. 55–59. DOI: 10.1364/JOSA.62.000055
  • Schuler C. J., Christopher Burger H., Harmeling S., Scholkopf B. A machine learning approach for non-blind image deconvolution. Proceedings of the IEEE conference on computer vision and pattern recognition. 2013. P. 1067–1074. DOI: 10.1109/CVPR.2013.142
  • Tanaka H., Kawano H. Image correction for improving visual acuity using Zernike-based vision simulation. 2021 20th international symposium on communications and information technologies (ISCIT). 2021. P. 32–36. DOI: 10.1109/ISCIT52804.2021.9590607
  • Thibos L. N. Formation and sampling of the retinal image. Seeing. Academic Press, 2000. P. 1–54. DOI: 10.1016/B978-012443760-9/50003-9
  • Vogel C. R., Oman M. E. Fast total variation-based image reconstruction. International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. A merican Society of Mechanical Engineers, 1995. V. 97669. P. 1009–1015. DOI: 10.1115/DETC1995-0672
  • Wiener N. Extrapolation, interpolation, and smoothing of stationary time series: With engineering applications. The MIT Press, 1949. DOI: 10.7551/mitpress/2946.001.0001
  • Xu F., Li D. Software based visual aberration correction for hmds. 2018 IEEE conference on virtual reality and 3d user interfaces (VR). 2018. P. 246–250. DOI: 10.1109/VR.2018.8447557
  • Xu L., Ren J. S., Liu C., Jia J. Deep convolutional neural network for image deconvolution. Advances in neural information processing systems. 2014. V. 27.
  • Ye J., Ji Y., Zhou M., Kang S. B., Yu J. Content aware image pre-compensation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018. V. 41(7). P. 1545–1558. DOI: 10.1109/TPAMI.2018.2839115
  • Zhang K., Gool L. V., Timofte R. Deep unfolding network for image super-resolution. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. P. 3217–3226. DOI: 10.1109/CVPR42600.2020.00328
  • Zhang L., Nayar S. Projection defocus analysis for scene capture and image display. ACM siggraph 2006 papers. 2006. P. 907– 915. DOI: 10.1145/1179352.1141974