SciresolSciresolhttps://indjst.org/author-guidelinesIndian Journal of Science and Technology0974-564510.17485/IJST/v14i24.604Research Article Adaptive Sparsity through Hybrid Regularization for Effective Image Deblurring PoojaSpoojasydit@gmail.com1MallikarjunaswamySpruthvi.malli@gmail.com2SharmilaN3Research Scholar, Department of ECE, Research Scholar, Department of ECEBangalore, Karnataka, 560060IndiaAssociate Professor, Department of ECE, JSS Academy of Technical EducationBangalore, Karnataka, 560060IndiaAssistant Professor, Department of EEE, RNS Institute of TechnologyBangalore, Karnataka, 560060India142420512021Abstract
Objectives: To develop image deblurring algorithm to effectively recover the original image from the captured blurry and noisy image. Methods: A novel image deblurring algorithm using adaptive priors is proposed. The adaptive priors vary the sparsity induced based on factors such as the type of blur affecting the image, image region, statistical parameters regarding the blur kernel and/or statistical predictions on the blur kernel. The proposed algorithm using adaptive priors improves the quality of the deblurred result, when compared with various recent image deblurring algorithms. Finding: the work contains a case study with regard to certain standard parameters. It is observed that proposed method is better in terms of frequency response, Peak signal-to-noise ratio (PSNR) and Structural Similarity Index (SSIM) values in comparison with other priors. Novelty: the proposed priors lead to the most effective results for image deblurring using the Bayesian framework. The proposed method improves the performance by 30% in PSNR and 45% in SSIM values in dB with uniform kernel size 12 * 12 and improves performance by 30 % in PSNR and 32% SSIM with a standard deviation of 3.5. The proposed method enhances the frequency response of the real-time image restoration process. Applications:Some important applications include the restoration of medical images such as MRI images, CT images where the intensity of these radiations is maintained to avoid damage to human organs. Remote sensing images captured through drones at a specified time cannot be retaken and hence restoring such images is very important. Similarly restoring images from CCTV footage, astronomical images.
KeywordsBlind Image DeblurringMaximum a Posteriori EstimationImage priorsRegularizationStructural Similarity Index (SSIM) and Peak signal-to-noise ratio (PSNR)NoneIntroduction
In many image restoration and image deblurring algorithms, the image deblurring limitation assumes that there exists a linear shift invariant blur kernel caused due to the degradation process, also known as point spread function (PSF). If PSF is not known, the limitation is termed as blind image deconvolution. In the recent times, blind image deconvolution techniques capable of handling a large motion blur have been designed. The idea of a space invariant blur cannot be generalized. For example, if the camera shakes when a picture is taken, a space-variant blurring occurs due to non-negligible depth variation. The recorded picture is blurred version of real image representing the actual scene in most of the imaging applications. Blurring is caused due to various reasons like optical aberrations, atmospheric distortions, averaging on pixel site on sensor, motion of objects/persons in the scene and motion of camera. Image restoration aims in recovery of real image from a single or a set of recorded images. The problem of restoration is ill-posed and requires techniques of regularization to effectively restore the image.
Broad Classification of Image Deblurring Techniques
Figure 1 shows the broad classification of image deblurring techniques, with no knowledge of the point spread function the recovery of the original image constitutes the blind deconvolution. The main objective of the Image Restoration (IR) technique is the recreation or the retrieval of the original sharp image through the perception of the degradation affecting the blurred image. Natural images caught under uncontrolled environmental conditions undergo constant degradation leading to unique image degradation parameters because of flaws in the capturing and imaging cycles. These kinds of degradation can be either variant with respect to space or either invariant with respect to space. The current research work centers more towards invariant blur functions or degradations which are space invariant. The degradation affecting any image might be characterized into two significant classes i.e., Blur and Noise. Images might be blurred because of environmental disturbance, defocusing of the focal point, deviation in an optical framework, relative movement among the imaging framework and the original scene. An automatic image deblurring is a goal of extraordinary practical interest to the upgrade the images in digital photographs, captured through digital cameras or camcorders for numerous applications such as in stargazing, tomography, remote sensing, biomedical imaging, movement following applications, and so on. Image deblurring techniques can be broadly classified into two types: blind and non-blind image deconvolution. In case of non-blind strategies data regarding the blurring channel is known. Blind Deconvolution is the recovery of the original image with unknown blur channel which leads to issues relating to unknown parameters, for example when the captured image is assessed from the perceptions of degradation, the specific data regarding the degradation and also noise isn't accessible. Blind deconvolution issue is exceptionally trying as it is difficult to derive the original sharp image and the degradation parameters from just a captured picture. The strategy which is explained in this work addresses image deblurring techniques of the second kind. In most practical situations, the reactive parameters of a blur, additionally known as Point Spread Function (PSF), isn't known with acceptable accuracy. Non-blind deblurring algorithms are extremely sensitive to mismatches among PSF utilized in the algorithm and a true or valid PSF, helpless information on the blurring PSF typically tends to the lesser effects of deblurring. Through Blind Image Deblurring (BID) the true or the valid PSF is estimated hence the probability of mismatches caused due to the variations in the degradation parameters utilized by the algorithm and the valid PSF is reduced. The main issue in solving the blind deconvolution problem is its illposedness. The problem is illposed due to the lack of uniqueness and steadiness of the solution caused due to infinite numbers of arrangements (real picture + blurring channel) that are viable for the degraded image. The outline of BID techniques is referred from 1. The unique quality of the scene or the uniqueness of the captured image can be used to reduce the ill posedness of the problem. Various such techniques have been proposed, for example in2 a unique filter is applied on the deblurred images to recover the parts which are in focus and another image is used to recover the portions which are not in focus. Vahid Saffari et al.3 introduces the novel nonparametric regression technique for the deblurring of blurred and noisy images which depends on the idea of the LPA (Local Polynomial Approximation) of an image with the consideration of Intersecting Confidence Intervals (ICI) which is adapted to characterize a versatile changing parameter i.e., sizes of window in an LPA assessor. Mean Curvature (MC) Regularization is used to enhance the quality of deblurred images. This regularization method has been less popular compared to the more general image regularization models due to the complexity in solving the MC mathematical model which comprises of higher order derivatives. Faisal Fairag et al. 4 propose a two-level method to solve this problem where the mathematical model is split into two steps, the first step forms a nonlinear system including the higher order derivatives and the second step is a large problem with low order derivatives. The authors show that this MC regularization model results in effective image deblurring. Xiao Yuan et al. 5 propose the use elastic-net regularization as a rank prior. They use salient edge structures estimated from intermediate images along with the elastic net regularization technique which acts as a prior to guide the algorithm to estimate the true blur kernel. Image Deblurring focused on specific objects can use their unique features as priors to improve the deblurring result. One such example is the use of license plate features as a prior in deblurring images of license plates from surveillance cameras at traffic signals. Chenping et al. propose an image deblurring model which uses the features of the license plate as a prior in an Lp norm based sparse regularization model. Since the deblurring is focused only on the license plate, the features of the license plates act as a vital prior in deblurring the degraded license plate images. to gives a way to consolidate earlier information in investigation. Chenping Zhao et al.6 propose an image prior based on weighted L1-norm regularization. They claim the L1-norm is better suited to remove noise in smooth regions compared to the L2- norm, which is more suited for noise removal in the edges of an image. Figure 2 shows the image deblurring process using Bayesian methods. Figure 2(a) Shows the data analysis procedure stages for non-blind image blurring. Figure 2(b) shows kernel estimation for blind image deblurring and Figure 2(C) Shows the image deblurring based image to image intensity restoration regression process.
Shows the image deburring process using Bayesian analysis process. It is divided into three steps as shown in figure .2(a), 2(b) and 2(c).
Numerous image processing applications have evolved in various fields in recent times. J. Anger, G. Facciolo et al. developed a graph structure to analyse the constraints caused due to delays and describe the objectives related to cost in an embedded system 7. Zhenhua Xu,. Propose a two-tone image prior which converts the latent image in its intermediate stage into a piecewise constant image comprising of only two grey levels 8. Shao Wen-Ze et al. Propose social tracking method using deep learning to classify the event conduction 9. Fang Dong et al. Make use of the valence arousal dimension space of an image to represent its emotions by predicting the continuous probability distribution of the image 10. P. Blomgren, et al. Propose a microblog filter method designed using the distinguishing factors of the multiple views of the surroundings in the embedding method 11. A new concept was proposed to model images which is pertinent to various image processing application 12. Neural network is used to design a recognition classifier, which uses peripheral and partial Central auditory models. These neural processing models facilitate a speech recognizer comprising of an acoustic pre-processor in the front end to function effectively as a classifier 13. A. Chambolle, P.L. Lions et al. Use simultaneous textual and visual information to assess the relevance and similarities in images which are tagged by the users 14. Po-Wen Hsieh et al. use discriminative classifiers to recognise the core saliency of different objects in an image. Discriminative classifiers are detected or learnt by using a multistage training paradigm 15. In this paper, the true blur kernel and its factors are derived from a blurred image to recover the original image. The novel strategy implemented to derive the blur kernel. It depends on the particular distortion brought about by the distorting administrator in the Fourier range sufficiency of a picture. A format dependent on a few blurred pictures has been created for both kinds of Gaussian and uniform blurred effect. The range of Fourier amplitude in loud obscured picture is compared with the formats to distinguish the blur model. Major factors of the blur are additionally decided based on the Fourier range sufficiency threshold picture. Along with this assessed blur model and factors value, an obscuring channel PSF is built, which is utilized for getting the picture gauge (i.e., deblurred picture) from a blurred picture. Because of the poorly presented nature of IR, earlier information on common pictures can be utilized to minimize the IR issue.
The framework is executed and tests are led with a blur factors such as 7, 9 and 11 for the blur effects of Uniform and factors like 2.5, 2.9, 3.3 has been utilized for the blur effects of Gaussian. The PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity Index Measure) are the two kinds of exhibition estimates utilized here. (PSNR) and (SSIM). Through the outcomes acquired very well, it is noted that the proposed redundant and sparse priors (ASDS) generate results which are more accurate and aesthetically pleasing compared to the more recent methods.
Maximum A Posterior (MAP) estimation for the Image Deblurring Problem
In this section the basic image deblurring problem statement is first defined. The general Bayesian mathematical model and its use in the image deblurring model is described. And the gain in using the MAP method for the blind image deblurring problem is shown. The further part of the section relates the statistical characteristics of the most common blur models such as the Gaussian and the Uniform blur model. The most common image priors used in the MAP model is briefly explained.
The general mathematical expression for the image degradation and restoration model is given below
b=hf*a+w
Where b and a represents the degraded image and original sharp image respectively, the inevitable noise in the system is represented by was it is additive in nature it is mathematically represented through a summation. hf represent the PSF of blurring operation and * represents the convolution operation 16, 17, 18, 19.
Another alternative method of representing equation 1 will be carried out by depicting it through its spectral equivalent. Under the influence of Discrete Fourier Transforms the frequency domain equivalent of equation 1, is derived as below:
B=HfA+W
The capitals in the equation 2 represent the Fourier Transforms of a, b, hf, and w. In the practical case, the spectral representation is commonly adapted because of its accurate implementations of the restoration filters in a Fourier domain. In case of deblurring under the influence of Discrete Fourier Transforms the blur kernel estimated is responsible for noise generation. Prior information on natural images is utilized to minimize the random noise issue.20 Bayesian methodology gives the way to join prior information in data investigation. Bayesian investigation always evaluates over the circumstance of posterior probability that describes the level of one's certainty with the involvement of circumstances provided. The law of Bayes states that the probability of posterior is directly proportional to the result of probability and probability of prior. The probability incorporates the data which is provided through new information. Prior information communicates the level of certainty involving the circumstance previously and the information is taken to effectively estimate the unknown. Statistical analysis is conducted in order to increase the information involving the captured image. The current image deblurring problem statement can be described using the Bayes' law as:
Pda∣nd=pdnd|a|pd(a)pdnd
Posterior is represented as Pda∣nd because of its effects in the examination. It will be the conditional probability of a which gives a new information nd. The probability pd(a) is also known as prior because it addresses a condition of information before the trial. The amount of pd(nd|a|) is the probability, which communicates the probability of information d given a specific 'a'. The probability is typically received from a model for anticipating the information, givena, just like a noise in the probabilistic model. The term used in denominator p (d) might be viewed as unity just for standardization process. As standardization accessed from different parameters, Bayes' law is frequently composed as a proportionality, denoting the denominator21, 22.
Even though posterior probability explains the condition of certainty regarding possible image completely, it is required to choose the individual “image result” or reconstruction. A similar selection will be that image which improves the probability of a posterior that is well-known as maximum posteriori (MAP). From the known data b, the posterior of an image is described by Bayes law (3) under the influence of proportionality
pd(a|b)αpd(b|a)pd(b|a)pd(a)
In which pd(b|a) is the probability of a mentioned information and a is likelihood and pd(a) will be prior or earlier probability of an a. The posterior probability density function of a negative logarithm is stated as:
logpd(a∣b)=φ(a)=Λ(a)+Π(a)
In which the beginning term arise through a likelihood and next term through the prior probability.
Likelihood has been mentioned under the assumed probability density function in a measurement of fluctuations regarding their predicted value (without noise). The negative log will be just a half of chi-squared for the additive uncorrelated Gaussian noise assumptions
-logpd(b∣a)=Λ(a)=12χ2=12σw2b-ha2
Selection of likelihood function must depend on a practical statistical characteristic of the image, a statistical characteristic measurement of noise or a deterministic behavior of the blur kernel which is either known or predicted through statistical analysis. The restored image can be estimated from the equation (7) below:
A^=argmina12b-hd22+λπ(a)
Where a= restored image and λ= parameter of regularization.
2.1 Image Priors in MAP
The selection of the right prior is crucial for effective restoration of the original image. Five of the most common priors used in the MAP mathematical model are discussed in this section Tikhonov (L2), Sobolev, Total variational (TV), Sparse, Sparse and redundant prior 23, 24, 25, 26, 27.
2.1.1 Tikhonov and Sobolev Priors
The Tikhonov and the Sobolev priors are discussed in the same section due to their interdependence in image deblurring applications, as explained further in this section. The Tikhonov prior is more commonly called as L2 norm prior. It mainly depends on criteria that, the energy available in images are bounded, whereas the noise in the image is unbounded. The restored image can be derived from equation 13.
a^=argminx||b-hf*a||22+λx‖22Where∥a∥22=∑iai21/2
The solution to the above equation is computed in the frequency domain as given in equation 14. Due to the filtering operation which is diagonalized over the Fourier domain.
A^ω=B(ω)Hf^(ω)Hf^ω2+λ
In which A^ω, B(ω),Hf^(ω) represents the frequency domain equivalent of the restored image, captured blurred image and the blur function (PSF) respectively 28, 29, 30, 31, 32. As the L2 regularization will not conduct any denoising, in order to eliminate noise more frequencies in a blurred picture can be penalized by using Sobolev prior. The Sobolev is prior stated below:
πa=∑i||∇ai||2=∑wSω||A(ω)||2.
Where Sω=||ω||2
As this prior is applied in the Fourier domain, the solution to the deblurring with a Sobolev prior can be normally framed with the Fourier coefficients as given in equation 15:
A^ω=B(ω)Hf^(ω)||Hf^(ω)||2+λS(ω)2.1.2 Total Variation Prior (TV)
Regularization of Sobolev significantly reduces noise in the image but it causes blurring in the image edges. The TV prior33 is capable of reconstructing the sharp edges of the image. The TV prior is given as:
πa=∑i∇ai
Concerning the energy of Sobolev, it essentially measures L1 norm rather than L2 norm, the L1-norm functional is derived by removing the square as shown in equation (16). The L1-norm is the addition of absolute values of the image intensity components34, 35. The TV prior ᴨa of an image a is not a smooth function, hence it needs the utilization of optimization techniques to be applied in a minimization objective function such as the MAP for regularization. Another method is to make use of the smooth absolute value instead of the absolute value of the image intensity. The expression for the smoothened TV prior is given as:
ᴨa=∑i|∇ai|2+ε2+1
As ε draws near to zero, the smoothened values turn out to be nearer to the original TV (total variation), hence optimization is more complicated. As the ε value increases, the smoothed energy turns out to be nearer to Sobolev energy, thus degrading the image edges. Since the TV prior is non-quadratic, it cannot be expressed in the frequency domain. Hence, to optimize an objective function making use of a TV prior requires an iterative method such as the gradient descent.
The expression to apply the gradient descent optimization method for image deblurring is shown in equation (18). Initially the deblurred image ‘a’ assumed the intensity values of the blurred image and with each iteration the intensity values are updated to derive the restored sharp image.
a(k+1)=a(k)-γ(hf*hf*ak-b+λGrandᴨ(ak))
In which a(k+1)represents the value of ‘a’ in the current iteration and a(k) represents the value of ‘a’ in previous iterations.
The same procedure is repeated until the final stopping criteria is met. The stopping criteria is fixed to a total number of iterations in this case. The standard test image of a cameraman deblurred using the TV prior is shown in the Figure 8(d).
2.1.3 Sparsity Prior:
Prior of sparsity involves a synthesis dependent regularization, shown in Figure 8(g). An inadequate set of coefficients (xm*)m in an edge φ=(φm)m, i.e., wavelet change which is applied in the blurred picture. Subsequent to applying the wavelet change, the energy will be present in lower number of coefficients (low-low band coefficients are taken). The L1-standard of the inadequate arrangement of coefficients is taken as the prior. The deblurred picture x can acquired from
a*=argmina12b-hf*a+2λ||<a,φm>||1
a,φm1=∑ma,φm , b is the blurred picture and hf is the blurring filter (PSF). In order to solve this non smooth optimization issue, iterative soft thresholding is utilized. Series of images as are computed by the equation 19.
a(s+1)=Sτλφ(as-τhf*(hf*as-b))
Where τ<2hf*hf
2.1.4 Sparse and Redundant Prior
The proposed image deblurring algorithm distinguishes the blur parameters from the blurred image. This helps in recovering the blur model from the captured blurry image. The knowledge of the blur parameters is very useful in solving the inverse problem of extracting the unknown blur model from the captured in blurred image. These prior thinks about the substance that can fluctuate fundamentally over various pictures or various patches in a solitary picture. The ASDS (adaptive sparse and redundant priors) strategy35 utilizes these prior. These technique takes in different arrangements of bases from the pre collected dataset of model picture patches and afterward for an offered fix to be handled, one bunch of bases are adaptively chosen to describe the nearby local sparse domain. With the equation (21).
a=ϕα
Where Φ = [ϕ1,ϕ2,ϕ3,ϕ4,……..ϕm]
α^=argminα{||b-Hfϕα||22+λ|α|0
Where the L0-norm counts the quantity of non-zero coefficients in a vector α. Once αˆ is acquired, 'a' would then be able to be assessed as a^=ϕα^ . The L0-minimization is a NP-hard combinatorial inquiry issue, and is typically tackled by ravenous calculations. Set of minimized sub-word references from top notch model picture patches are gotten the hang of utilizing principal component analysis (PCA) strategy. For a picture fix to coded, the best sub-word reference which is generally applicable for a given fix is chosen.
In case that, ϕk,k=1,2,3,4…………K, will be the set of K orthonormal sub-dictionaries. Let 'a'be an image vector, and ai=Via,i=1,2,3,4,5,………..N, be the ith patch (size: 8 × 8) vector of 'a', where Viis a matrix extracting patch ai from 'a'. For patch ai, suppose that a sub- dictionary ϕki is selected for it. Then, ai can be approximated as a^=ϕkiαi^ , || αi||0≤U via sparse coding. The entire picture 'a'can be reframed by averaging all the reframed patches ai^, is arithmetically given as:
a^=(∑i=1NViUVi)-1∑i=1N(ViUϕkiαi)Proposed Method
The effective estimation of the true blur kernel is vastly dependent on the selection of the right prior. Numerous priors have been designed by various research scholars to explicitly and implicitly meet the required criteria of selection of right parameters in the recovered sharp image. The captured blurry image though assumed to be degraded by a space invariant blur function, the effects of the blur are not constant on the entire image. For example, the effect of blur is more prominent on an object in motion in the image scene whereas it is less prominent on the background objects of the image captured. The use a generic and constant prior over the entire image does not result in effectively recovering such variations in degradation. Hence, we propose adaptive image priors which vary according to the deblurring requirements of the image. A Novel Image Prior based on hybrid regularization techniques to adaptively induce sparsity is proposed in this paper.
3.1 Spectral Characteristics of Blurring
The proposed Adaptive priors are set to vary based on the spectral characteristics of the image. The spectral characteristics of the image provide insight into the nature of the blurring caused, the effect of possible additive noise. Considering blurring as an operation we can assess that it suppresses various spectral components of an image. The range of spectral components suppressed due to blurring depends on the type of blur acting on the image. For example, the below image in Figure 3(a) shows the spectral components of an original sharp image, Figure 3(b) shows the spectral components of the same image when it is blurred using a synthetic uniform blur, similarly Figure 3(c) shows the spectral components of the same image blurred using a synthetic Gaussian blur. The spectral components of various blurs are empirically analyzed to understand their effect on a captured blurred image. The statistical data from these experiments are used in our algorithm to vary the adaptive priors.
3.1.1 Blur Template Identification
A layout of blur is developed through the frequency domain representation of a few blurred images. Initially the Fourier range adequacy of blurred images and their threshold is estimated. Subsequent to thresholding, the morphological activity36 is conducted on the resultant image. A morphological activity applies an organizing component to a data image, giving an output image of a common size. Under the activity of morphology, an estimation of each pixel in an output image based on the correlation of comparing pixel in an info image along with its neighbors. By taking a state and size of the area, morphological activity which is sensitive to explicit shapes in an information image is developed. Hence, enlargement is done with a disk structuring component. A structuring element is a network composed with just 0's and 1's can have any self-assertive structure and size. The pixels of estimations 1 characterizes an area. Widening adds pixels for the limits of figures in a picture. The quantity of pixels included or eliminated through the items in a picture relies upon a size and state of structuring component utilized to deal with the picture. The estimation of an output pixel is a greatest estimation of a relative multitude of pixels in an input pixel's area. From the double picture, if any of a pixel is assigned to a value 1, an output pixel is assigned to 1. Widening is carried in order to define the limit pixels to interface the overall white areas.
Blurring suppresses certain frequencies in an image (a) the frequency spectrum of an original image which is uncorrupted by blur (b) the frequency spectrum of the same image which is blurred by a uniform blur (c) the frequency spectrum of the same image which is blurred by a Gaussian blur.
The appearance of a point spread function or a blur kernel is depicted for (a) Uniform blur (b) Gaussian blur
Through every blurred image, an enlarged image is developed. A mean image of this enlarged Gaussian blurred image is made and it is considered as the format for distinguishing Gaussian blur. Likewise, a format for unique blur ID is developed. The point spread function which follows a Gaussian distribution is given as:
hfn1,n2=12πρ2exp(-n12+n22ρ2)
Where ρ2a variance gaussian functional parameter of the PSF, Figure 5 shows the gaussian functional Fourier transform and its absolute value.
Frequency response with respect to gaussian Point Spread Function (PSF).
The uniform rectangular blur is mathematically represented by the below equation
hfn1,n2=1s2,n1<s2,n2<s20, elsewhere
In which 's' gives the area of smoothing. The characteristics of frequencies in equation 8 are represented in Figure 6.
Frequency Response with respect to uniform blur kernel
Prior information regarding some deblurring function and its factors will be of essential significance in a deblurring the distorted image37, 38, 39, 40, 36, 41, 42. This work implements a novel strategy for image enhancement by identifying the type of blur through the spectral components of the captured image. As the Fourier range of an image is affected under the influence of blur, this technique distinguishes the blur based on its Fourier amplitude range. By recognizing the kind of blur, its factors are likewise removed from a Fourier range threshold image through the use of appropriate adaptive priors. Similarly, formats for distinguishing the popular synthetic blur models such as Uniform blur, disk blur and motion blur are developed.
3.2 Estimation of the True Blur Kernel through the Proposed Method
In case of deblurring, initially the Fourier range sufficiency of a deblurred image is considered. The spectral components of the image greater than the estimated threshold is decreased through normalization. This image is further dilated, this dilated image is compared with the different blur templates derived in the previous section. The comparison is done through subtracting the dilated image with each of the blur templates identified. These difference images are further processed to check which one of them has the least number of non-zero elements. The blur template with which the dilated image has produced a difference image with the least number of non-zero elements is identified as the blur affecting the captured image. In an event that the blur affecting the image is some other type of blur then the number of non-zero elements in the difference image will be very high. Even in situations when none of the existing blur templates represent the type of blur which has affected the distorted image, one of these templates will have a minimum value compared to the others and hence it will be chosen as the blur affecting the captured image. Hence, to keep away from this incorrect selection a constraint is adapted such that the minimum value required for blur template selection is a constant.
Since Gaussian and uniform blur are statistical blurs, their variance is measured using the dimensions of the PSF, for the Gaussian blur the variance is proportional to the radius of a middle white part of the PSF. Fluctuations have impact over the span of a middle white part of the dilated image. i.e., radius of the middle white region of the PSF is proportional to the level of variance the Gaussian distribution follows. The value of variance relating to radius of the PSF is figured as mentioned below:
Value=(r-LrUr-Lr)Variance=0.5*Value+Lr
Where, Ur is the upper radius and Lr the lower radius of the PSF show’s the analysis of variance range and radius. Consider radius >55, the variance range manipulated by using euaqtion.12.
Value=1-((r-5562-55))Variance=0.5*Value+1
In case of uniform blur, height and breadth of a middle white portion is determined and least of it has been considered as a length 43. On the off chance that length is in the reach or not exactly a particular worth, at that point the portion size comparing to that is chosen. Along with these assessed blur type and blur factors, the blurring channel PSF is developed as indicated in segment. These assessed PSF parameters are utilized in the image deblurring algorithm to vary accordingly the regularization norm of the hybrid image deblurring algorithm.
3.2.1 Proposed Image Deblurring Algorithm
The proposed image deblurring algorithm is implemented in MATLAB. The MAP mathematical model is used for implementation due to its reduced computational complexity. The main reason for using the MAP model is due to its readiness in incorporating the image priors or the prior information regarding the image in its estimation of the true PSF. The algorithm is designed utilizing the MAP mathematical expression shown in equation (6). The algorithm is designed to adaptively vary the image priors based on the spectral analysis of the captured image, that is described in the previous sub section. Here the regularization term used is the Lp Norm Regularization. By varying the p value, the sparsity induced in the various image region varies. Hence this varies the image prior accordingly. The novel image prior so chosen from our algorithm is called the Sparsity through Adaptive Priors (SAP).
Results and Discussion
The six standard test images of size 512X512used from top left (clockwise): Cameraman, Cactus, Boat, Lena, Peppers, Baboon.
Comparison of image deblurring results of different methods using the standard test image of cameraman: (a) Original Image (b) Image blurred through Uniform Blur (c) Deblurred using Tikhonov & Sobolev Priors (d) Deblurred using TV Priors (e) Deblurred usingSparse Priors (f) Deblurred using L1 Priors (g) Deblurred using Sparse and Redundant Priors (h) Deblurred using the proposed method (SAP).
The performance of the proposed novel image prior (SAP) in an image deblurring algorithm using the MAP mathematical model is compared with the popular generic image priors used in the MAP model such as the Tikhonov (L2), Sobolev, Total variational (TV), Sparse, Sparse and redundant prior. Six standard test images are used to compare the performance, which consists of both grayscale and color images. The three standard grayscale test images used are Cameraman, Cactus and Boat. Similarly, the three standard color test images used are Baboon, Peppers and Lena shown in Figure 7. The investigation on performance of the proposed method is done through the use of performance metrics such as the Peak Signal to Noise Ratio (PSNR) and the Structural Similarity Index (SSIM). The test images are synthetically blurred using a Uniform blur of kernel size 12X12. The synthetically blurred images are deblurred using the popular generic image priors and the proposed SAP image prior. The PSNR & SSIM values for the resulting deblurred images using various methods are estimated and tabulated in the Table 1, Table 2. The resulting PSNR & SSIM values are plotted in the graphs shown in Figure 9, Figure 10.Figure 12
The PSNR result analysis in dB of the blurred and restored images blurred with Uniform kernel of size 12x12 and graphical analysis is shown in Figure 11.
Name
SAP
Sparsity
Sobolev
L2
TV
Blurred
Animal
27.10
25.86
24.98
24.68
24.75
21.80
Cactus
26.07
25.02
23.90
23.55
23.72
20.36
Lena
29.98
25.80
24.58
24.25
24.69
21.45
Cameraman
29.32
25.50
23.56
22.85
23.54
19.65
Boat
29.99
26.05
24.55
23.95
24.65
20.48
Building
25.35
23.68
21.56
21.56
21.45
18.69
Graphical analysis of PSNR result analysis in dB of the blurred and restored images blurred with Uniform kernel of size 12x12.
The SSIM result analysis of the restored images blurred with Uniform kernel of size 12 *12 and graphical analysis is shown in Figure 10.
Name
SAP
Sparsity
Sobolev
L2
TV
Animal
0.789
0.623
0.642
0.610
0.615
Cactus
0.786
0.595
0.580
0.585
0.585
Lena
0.832
0.599
0.595
0.596
0.595
Cameraman
0.956
0.625
0.608
0.605
0.610
Boat
0.865
0.526
0.599
0.580
0.580
Building
0.823
0.590
0.587
0.580
0.578
Graphical analysis of SSIM result analysis of the restored images blurred with Uniform kernel of size 12 *12
Now the test images are synthetically blurred using a Gaussian blur of standard deviation 3.5 and the deblurred results for the same is shown in the Table 3, Table 4. The performance metric values in the Table 3, Table 4 are plotted in the graphs in the Figure 11, Figure 12 respectively.
The PSNR result analysis in dB of the blurred and restored images blurred with Gaussian blur of standard deviation 3.5 and graphical analysis is shown in Figure 11.
Name
SAP
Sparsity
Sobolev
L2
TV
Blurring
Estimated Variance
Animal
26.50
25.880.
24.20
24.40
24.70
22.65
2.41
Cactus
25.68
25.30
23.99
23.41
23.75
21.45
2.402
Lena
28.50
25.89
24.70
24.75
24.89
22.34
2.35
Cameraman
25.65
25.29
22.98
22.98
22.90
20.40
2.39
Boat
26.89
25.79
24.45
21.35
24.25
21.50
2.40
Building
23.48
23.10
21.45
21.40
21.20
19.40
2.51
Graphical Analysis PSNR result analysis in dB of the blurred and restored images blurred with Gaussian blur of standard deviation 3.5.
The SSIMresult analysis of the restored images blurred with Gaussian blur of standard deviation 3.5 and graphical analysis is shown in Figure 12.
Name
SAP
Sparsity
Sobolev
L2
TV
Animal
0.730
0.615
0.613
0.610
0.615
Cactus
0.720
0.578
0.580
0.573
0.578
Lena
0.799
0.597
0.560
0.589
0.598
Cameraman
0.862
0.609
0.608
0.609
0.609
Boat
0.780
0.580
0.581
0.580
0.589
Building
0.762
0.582
0.582
0.579
0.580
Graphical Analysis SSIM result analysis in dB of the restored images blurred with Gaussian blur of standard deviation 3.5
The performance of the proposed SAP method is analyzed using real images also, we use naturally blurred satellite image due to atmospheric turbulence and a google image blurred using a uniform blur for the same. The deblurred results for the satellite images utilizing different priors and the google image using different priors are appeared in the Figure 13, Figure 14, respectively.
Comparison of image deblurring results of different methods: (a) Captured blurred satellite image (b) Deblurred using Sobolev Priors (c) Deblurred using TV Priors (d) Deblurred using Tikhonov Priors (e) Deblurred using Sparse Priors (f) Deblurred using the proposed method (SAP).
Comparison of image deblurring results of different methods: (a) Original Image (b) Image blurred through Uniform Blur (c) Deblurred using Sobolev Priors (d) Deblurred using TV Priors (e) Deblurred using Gaussian Priors (f) Deblurred using L1 Priors (g) Deblurred using Tikhonov Priors (h) Deblurred using Sparse Priors (i) Deblurred using the proposed method (SAP)
The Figure 15. Shows the performance analysis between existing and proposed methods with respective aspect ratio = 3.32
performance analysis between existing and proposed methods with respective aspect ratio=3.32.
As shown in Figure 16 performance analysis between existing and proposed methods with respective to real life blurring image.
performance analysis between existing and proposed methods with respective to real life blurring image
shows the performance analysis of SSIM and PSNR values of various conventional methods with the proposed method.
SI.NO
Reference Number
Methods
Protocol
SSIM (%)
PSNR (%)
1.
1
Sparsity
Uniform kernel of size 12x12 (Animal Image )
0.0623
0.2586
2
Sobolev
0.0642
0.2498
3
L2
0.0610
0.2468
44
TV
0.0615
0.2475
19
Blurring
-
0.2180
32
Estimated Variance
-
-
-
Proposed
0.789
0.2710
2.
23
Sparsity
standard deviation 3.5 (Animal Image)
0.0615
0.25880
45
Sobolev
0.0613
0.2420
46
L2
0.0610
0.2440
42
TV
0.0615
0.2470
41
Blurring
-
0.2265
40
Estimated Variance
-
0.0241
-
Proposed
0.0073
0.2650
Conclusion
The proposed method is implemented using the Bayesian MAP mathematical model approach, the novel SAP image priors are incorporated in the MAP model which effectively estimates the blur types and its parameters. Comparison of proposed method with conventional methods with respect to fixed uniform kernel value 12 * 12 , shows that ASP has 27.10 PSNR and 0.789 SSIM value in dB whereas conventional methods PSNR and SSIM values has less value than the proposed method. The proposed blurring filter makes use of sparse and redundant priors to improve constructive PSF efficiency up to 13.06 % as compared to conventional methods. Our method is compared with five other important priors like total variation, sparse, L2, Sobolev in a Bayesian framework. This performance analysis is made utilizing peak-signal-to-noise ratio (PSNR) and structural similarity index [SSIM] by considering sets of blur parameters 7,10 and 12 for uniform blur and parameter 2.6, 3.0 and 3.4 for Gaussian blur. As per the comparison and performance analysis, the sparse and redundant prior gives better results. ASP performance is improvement is in range of 25 to 30% in PSNR and 40 to 45 % improvement in SSIM values in dB with uniform kernel of size 12 * 12 and ASP performance improvement in range of 28 to 30% in PSNR and 26 to 32% in SSIM values of blurred and regained pictures with Gaussian blur have standard deviation 3.5.
Limitation and Future scope
The proposed image deblurring algorithm is implemented in MatLab and hence the execution time of the algorithm is slightly larger when compared to recent image deblurring methods implemented using platforms such C++. The proposed image deblurring model can be extended to a variety of applications such as MRI images and satellite images. Due to its similarity with the problem statement of Super Resolution, the same algorithm can be extended to generate super resolution images.
Acknowledgements
The authors would like to thank JSS Academy of Technical Education, Bengaluru, Visvesvaraya Technological University (VTU), Belagavi and Vision Group on Science and Technology (VGST) Karnataka Fund for Infrastructure strengthening in Science & Technology Level – 2 for all the support and encouragement provided by them to take up this research work and publish this paper
ReferencesMohanK.ChandrasekharP.JilaniS. A. K.Object Face Liveness Detection with Combined HOGlocal Phase Quantization using Fuzzy based SVM Classifier20171031100974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2017/v10i3/109035KumaravelSSundarK. Joseph AbrahamVaithiyanathanVSuper Resolution Image Reconstruction for Bone Images2019122616SaffariVahidGhazimoradiAmirsoheilAlirezanejadMehdiEffect of Laplacian of Gaussian Filter on Watermark Retrieval in Spatial domain Watermarking2015833140974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2015/v8i1/71226FairagFaisalChenKeBrito-LoezaCarlosAhmadShahbazA two-level method for image denoising and image deblurring models using mean curvature regularization20211210020-7160, 1029-0265Informa UK Limitedhttps://dx.doi.org/10.1080/00207160.2021.1929939YuXiaoYuanXieWeiSingle Image Blind Deblurring Based on Salient Edge-Structures and Elastic-Net Regularization2020628104910610924-9907, 1573-7683Springer Science and Business Media LLChttps://dx.doi.org/10.1007/s10851-020-00949-6ZhaoChenpingWangYingjunJiaoHongweiLp-Norm-Baesd Sparse Regularization Model for Liscence Plate Deblurring202082207222081AngerJ´er´emyFaccioloGabrieleDelbracioMauricioBlind image deblurring using the lo gradient prior, Image Process2019912414210.5201/ipol.2019.243Xu ZhenhuaChenHuasongLiZhenhuaBlind image deblurring using group sparse representation20201021-81810.1016/j.dsp.2020.102736Wen-Ze ShaoYuan-YuanLiuYe Lu-YueDeblurGAN+: Revisiting blind motion deblurring using conditional adversarial networks202016811010.1016/j.sigpro.2019.107338DongFangMaQiantingSingle image blind deblurring based on the fractional-order differential19607861960197710.1016/j.camwa.2019.03.033BlomgrenPChanT FMuletPWongC KTotal variation image restoration: numerical methods and extensions1997338438710.1109/ICIP.1997.632128Cai Jian-FengJi HuiLiu ChaoqiangShen ZuoweiFramelet-Based Blind Motion Deblurring From a Single Image20122125625721057-7149, 1941-0042Institute of Electrical and Electronics Engineers (IEEE)https://dx.doi.org/10.1109/tip.2011.2164413CandesE JWakinM BBoydS PEnhancing sparsity by reweighted l1 minimization20081487790510.1007/s00041-008-9045-xChambolleAntoninLionsPierre-LouisImage recovery via total variation minimization and related problems19977621671880029-599X, 0945-3245Springer Science and Business Media LLChttps://dx.doi.org/10.1007/s002110050258HsiehPo-WenShaoPei-ChiangBlind image deblurring based on the sparsity of patch minimum information20211090031-3203Elsevier BVhttps://dx.doi.org/10.1016/j.patcog.2020.107597MohanS ChandraAdaptive Super-Resolution Image Reconstruction with Lorentzian Error Norm”201710161610.17485/ijst/2017/v10i16/106780RajkumarS.MalathiG.A Comparative Analysis on Image Quality Assessment for Real Time Satellite Images20169341110974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2016/v9i34/96766MamtaRamtekeDuttaMaitreyeeGA based Blind Deconvolution Technique of Image Restoration using Cepstrum Domain of Motion Blur20171016180974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2017/v10i16/114303JananiP.PremaladhaJ.RavichandranK. S.Image Enhancement Techniques: A Study20158221120974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2015/v8i22/79318SureshGulivindalaRaoChanamallu SrinivasaRST Invariant Image Forgery Detection2016922180974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2016/v9i22/89227UmaK.PallaviA.R.SrilathaS.Comparision of Blur Detection and Segmentation Techniques20169S1170974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2016/v9is1/107619SinghRahulSinghSomeetNavjot Kaur “A Review: Techniques of Vehicle Detection in Fog20169471410.17485/ijst/2016/v9i45/106793UmashankarM LRamakrishnaM VMallikarjunaswamySDesign of High Speed Reconfigurable Deployment Intelligent Genetic Algorithm in Maximum Coverage Wireless Sensor Network201916https://doi.org/10.1109/IconDSC.2019.8816930AlamM. ZeshanQianQinchunGunturkBahadir K.Space-variant blur kernel estimation and image deblurring through kernel clustering20197641550923-5965Elsevier BVhttps://dx.doi.org/10.1016/j.image.2019.04.014ZhangHongWuYujieZhangLeiZhangZeyuLiYaweiImage deblurring using tri-segment intensity prior20203982652790925-2312Elsevier BVhttps://dx.doi.org/10.1016/j.neucom.2020.02.082MahendraH NMallikarjunaswamySRekhaVPuspalathaVSharmilaNPerformance Analysis of Different Classifier for Remote Sensing Application”20199171537158http://dx.doi.org/10.35940/ijeat.A1879.109119NMahendra HMallikarjunaswamySG K SiddeshM KomalaN SharmilaEvolution of real-time onboard processing and classification of remotely sensed data20201320201020200974-6846, 0974-564510.17485/ijst/v13i20.459Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/v13i20.459ThazeenSadiyaMallikarjunaswamySSiddeshG KSharmilaNConventional and Subspace Algorithms for Mobile Source Detection and Radiation Formation20213811351450765-0019, 1958-5608International Information and Engineering Technology Associationhttps://dx.doi.org/10.18280/ts.380114ChaitraSRekhaVHarishaA MMadhuT AMallikarjunaswamySA comprehensive review of parallel concatenation of LDPC code techniques2021145432444https://doi.org/10.17485/IJST/v13i20.459UmashankarM LAnithaT NMallikarjunaswamySAn efficient hybrid model for cluster head selection to optimize wireless sensor network using simulated annealing algorithm2021143270288https://doi.org/10.17485/IJST/v14i3.2318Raj K ShashiSiddeshG KMallikarjunaswamySRajK VivekInterference resilient stochastic prediction based dynamic resource allocation model for cognitive MANETs”2020134143324350https://doi.org/ 10.17485/IJST/v13i41.687CuiGuangmangYeXiaojieZhaoJufengZhuLiyaoChenYingMulti-frame motion deblurring using coded exposure imaging with complementary fluttering sequences20201261061190030-3992Elsevier BVhttps://dx.doi.org/10.1016/j.optlastec.2020.106119ChengShichaoLiuRishengHeYiFanXinLuoZhongxuanBlind image deblurring via hybrid deep priors modeling20203873343450925-2312Elsevier BVhttps://dx.doi.org/10.1016/j.neucom.2020.01.004fangFangM Donga QiantingSingle image blind deblurring based on the fractional-order differential201978619601977https://doi.org/10.1016/j.camwa.2019.03.033ShaoWen-ZeLinYun-ZhiLiuYuan-YuanWangLi-QianGeQiBaoBing-KunGradient-based discriminative modeling for blind image deblurring20204133053270925-2312Elsevier BVhttps://dx.doi.org/10.1016/j.neucom.2020.06.093QiQingGuoJichangJinWeipeiEGAN: Non-uniform image deblurring based on edge adversarial mechanism and partial weight sharing network2020881159520923-5965Elsevier BVhttps://dx.doi.org/10.1016/j.image.2020.115952KohJaihyunLeeJanghoYoonSungrohSingle-image deblurring with neural networks: A comparative survey20212031191077-3142Elsevier BVhttps://dx.doi.org/10.1016/j.cviu.2020.103134ZhangDongyangLiangZhenwenShaoJieJoint image deblurring and super-resolution with attention dual supervised network20204121871960925-2312Elsevier BVhttps://dx.doi.org/10.1016/j.neucom.2020.05.069ZhangYanfangLiWeihongLiZhenghaoNingTaigongDual attention per-pixel filter network for spatially varying image deblurring20211131221051-2004Elsevier BVhttps://dx.doi.org/10.1016/j.dsp.2021.103008ChenXiyuanLiuDiZhangYuLiuXiaoXuYuanShiChunfengRobust motion blur kernel parameter estimation for star image deblurring20212301230030-4026Elsevier BVhttps://dx.doi.org/10.1016/j.ijleo.2021.166288ZhangYaozongShiYuMaLeiWuJinmengWangLeiHongHanyuBlind natural image deblurring with edge preservation based on L0-regularized gradient prior20212251657351657350030-402610.1016/j.ijleo.2020.165735Elsevier BVhttps://dx.doi.org/10.1016/j.ijleo.2020.165735PengJuncaiShaoYuanjieSangNongGaoChangxinJoint image deblurring and matching with feature-based sparse representation prior20201031230031-3203Elsevier BVhttps://dx.doi.org/10.1016/j.patcog.2020.107300MallikarjunaswamySSharmilaNMaheshkumarDKomalaMMahendraH NImplementation of an effective hybrid model for islanded microgrid energy management2020132727332746https://dx.doi.org/10.17485/ijst/v13i27.982SreeP. HemaRaoK. SubbaA Novel Approach for Blurred Face Recognition System Using GLDA Features with LCDR Classification20191229180974-6846, 0974-5645Indian Society for Education and Environmenthttps://dx.doi.org/10.17485/ijst/2019/v12i29/146975LiuJingjingMaRuijieZengXiaoyangLiuWanquanWangMingyuChenHuiAn efficient non-convex total variation approach for image deblurring and denoising20213971190096-3003Elsevier BVhttps://dx.doi.org/10.1016/j.amc.2021.125977XuNAdaptively Sparse Regularization for Blind Image Restoration2021