Feature Based Image Registration Using ORB and CNN for Remote Sensing Images

Laukikkumar K Patel; Manish I Patel

doi:10.17485/IJST/v16i42.1782

Article

Feature Based Image Registration Using ORB and CNN for Remote Sensing Images

VIEWS 315
PDF 121

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v16i42.1782

Year: 2023, Volume: 16, Issue: 42, Pages: 3803-3813

Original Article

Feature Based Image Registration Using ORB and CNN for Remote Sensing Images

Laukikkumar K Patel^1,2*, Manish I Patel³

¹Ph.D. Scholar, Sankalchand Patel University, Visnagar, 384315, Gujarat, India
²Lecturer, Department of Electronics and Communication Engineering, Government Polytechnic, Palanpur, Gujarat, India
³Assistant Professor, Department of Electronics and Communication Engineering, Institute of Technology, Nirma University, Ahmedabad, Gujarat, India

*Corresponding Author
Email: [email protected]

Received Date:18 July 2023, Accepted Date:03 October 2023, Published Date:13 November 2023

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Objectives: To improve image registration by reducing the estimation error of the rotation transformation parameter under illumination change effect in remote sensing images using Oriented Fast and Rotated Brief (ORB) and Convolutional Neural Network (CNN). Also, to reduce computational complexity that can be increased due to use of CNN. Methods: The image registration process aligns two or more images geometrically and a novel feature based approaches for image registration is proposed here, where ORB and CNN are used to estimate rotation transformation parameter under illumination change effects in remote sensing images with novelty in generation of feature descriptor. The results of the proposed approach are compared with an approach which uses only ORB for both feature detection and descriptor generation. Findings: In the proposed approach, convolutional features from modified CNN are used in a novel manner with ORB descriptor to generate the final fusion descriptor. Also, this novel approach reduces the computational complexity by limiting the descriptor size that can be increased due to CNN. Here, three different combinations of CNN layers are provided for the generation of descriptor with ORB and this approach is also tested with transfer learning concept and other than remote sensing image, which shows improved results for taken cases. The results of novel approach show that the estimation of rotation transformation parameter and image registration is improved, and the estimation error is reduced to 0.1% to 0.9% for taken cases. Novelty: Novelty is provided in the generation of descriptor by fusion of CNN (modified visual geometry group (VGG19)) features with descriptor from ORB for reduction in estimation error, limiting descriptor size to reduce computational complexity, and improvement in image registration of remote sensing images. It is also tested for transfer learning case and other than remote sensing image, where improved results are also seen.

Keywords: Image registration, CNN, Remote sensing, VGG19, Oriented fast and rotated brief

References

Zitova B, Flusser J. Image registration methods: a survey. Image and vision computing. 2003;21(11):977–1000. Available from: https://doi.org/10.1016/S0262-8856(03)00137-9
Patel MI, Thakar VK, Shah SK. Image Registration of Satellite Images with Varying Illumination Level Using HOG Descriptor Based SURF. Procedia Computer Science. 2016;93:382–388. Available from: https://doi.org/10.1016/j.procs.2016.07.224
Joshi K, Patel MI. Recent advances in local feature detector and descriptor: a literature survey. International Journal of Multimedia Information Retrieval. 2020;9(4):231–247. Available from: https://doi.org/10.1007/s13735-020-00200-3
Lowe DG. Distinctive image features from scale-invariant keypoints. International journal of computer vision. 2004;60:91–110. Available from: https://doi.org/10.1023/B:VISI.0000029664.99615.94
Rublee E, Rabaud V, Konolige K, Bradski G. ORB: An efficient alternative to SIFT or SURF. In: 2011 International Conference on Computer Vision. Barcelona, Spain, 06-13 November 2011. IEEE. p. 2564–2571.
Ihmeida M, Wei H. Image Registration Techniques and Applications: Comparative Study on Remote Sensing Imagery. In: 2021 14th International Conference on Developments in eSystems Engineering (DeSE). Sharjah, United Arab Emirates, 07-10 December 2021. IEEE. p. 142–148.
Li H, Yu Y, Du W. Research on rapid image registration based on feature point detection. In: Eighth Symposium on Novel Photoelectronic Detection Technology and Applications. Kunming, China. SPIE. 12169:964–969.
Shang MS. An image registration method based on SIFT and ORB. In: 2nd Conference on High Performance Computing and Communication Engineering (HPCCE 2022). Harbin, China. SPIE. 12605:366–373.
Alnagdawi MA, Hashim SZM. ORB-PC feature-based image registration. In: 2019 IEEE International Conference on Signal and Image Processing Applications (ICSIPA). Kuala Lumpur, Malaysia, 17-19 September 2019. IEEE. p. 111–115.
Karami E, Prasad S, Shehata M. Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images. 2017. Available from: https://doi.org/10.48550/arXiv.1710.02726
Xie Y, Wang Q, Chang Y, Zhang X. Fast Target Recognition Based on Improved ORB Feature. Applied Sciences. 2022;12(2):1–14. Available from: https://doi.org/10.3390/app12020786
Pei H, Liu K, Zhao X, Yahya AA. Enhancing aphid detection framework based on ORB and convolutional neural networks. Scientific Reports. 2020;10(1):1–15. Available from: https://doi.org/10.1038/s41598-020-75721-2
Rani R, Kumar R, Singh AP. Implementation of ORB and Object Classification using KNN and SVM Classifiers. International Journal of Computer Sciences and Engineering. 2019;7(3):280–285. Available from: https://doi.org/10.26438/ijcse/v7i3.280285
Kuppala K, Banda S, Barige TR. An overview of deep learning methods for image registration with focus on feature-based approaches. International Journal of Image and Data Fusion. 2020;11(2):113–135. Available from: https://doi.org/10.1080/19479832.2019.1707720
Gupta A, Peng Y, Watson S, Yin H. Multitemporal Aerial Image Registration Using Semantic Features. In: International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2019: Intelligent Data Engineering and Automated Learning – IDEAL 2019, Lecture Notes in Computer Science . (pp. 78-86) Springer International Publishing. 2019.
Zou B, Li H, Zhang L. Self-Supervised SAR Image Registration With SAR-Superpoint and Transformation Aggregation. IEEE Transactions on Geoscience and Remote Sensing. 2022;61:1–15. Available from: https://ieeexplore.ieee.org/document/10002310
Luo X, Lai G, Wang X, Jin Y, He X, Xu W, et al. UAV Remote Sensing Image Automatic Registration Based on Deep Residual Features. Remote Sensing. 2021;13(18):1–15. Available from: https://doi.org/10.3390/rs13183605
Liu X, Han F, Ghazali KH, Mohamed II, Zhao Y. A review of Convolutional Neural Networks in Remote Sensing Image. In: Proceedings of the 2019 8th International Conference on Software and Computer Applications. (pp. 263-267) ACM. 2019.
Ye F, Su Y, Xiao H, Zhao X, MW. Remote Sensing Image Registration Using Convolutional Neural Network Features. IEEE Geoscience and Remote Sensing Letters. 2018;15(2):232–236. Available from: https://doi.org/10.1109/LGRS.2017.2781741
Patel LK, Patel MI. Feature based image registration using CNN features for satellite images having varying illumination level. International Journal of Advanced Technology and Engineering Exploration. 2023;10(101):440–457. Available from: https://doi.org/10.19101/IJATEE.2022.10100139
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014. Available from: https://doi.org/10.48550/arXiv.1409.1556
Zuliani M. Computational Methods for Automatic Image Registration. University of California thesis

Copyright

© 2023 Patel & Patel. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)