Biological slices are an effective tool for studying the physiological structure and evolution mechanism of biological systems. However, due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing, leads to problems such as difficulty in preparing slice images and breakage of slice images. Therefore, we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation, achieving the high-fidelity reconstruction of slice images. We further discussed the relationship between deep convolutional neural networks and sparse representation, ensuring the high-fidelity characteristic of the algorithm first. A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature. And multi-layer deep sparse representation is used to implement dictionary learning, acquiring better signal expression. Compared with methods such as NLABH, Shearlet, Partial Differential Equation (PDE), K-Singular Value Decomposition (K-SVD), Convolutional Sparse Coding, and Deep Image Prior, the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data, which realized high-fidelity inpainting, under the condition of small-scale image data. And the

Biological slice is a technique that uses frozen or paraffin slicing to obtain thin slices of biological tissue, which is an important approach to studying the interaction mechanism of biological tissue and the system. For example, mouse brain slices are an important model for studying the development of mouse neural networks, synapses, and brain area function [

The example-based method [

Sparse representation (SR) [

The sparse coding reconstruction of an image is achieved by the linear combination of atoms in the dictionary [

The K-SVD algorithm [

In high-reliability requirements for the task of restoring biological slice images, deep learning uses generative means, but the authenticity of the reconstruction results cannot be guaranteed. Using deep neural networks such as Autoencoder and restricted Boltzmann machines as deep sparse representation models [

We summarize the contributions of this paper as follows: (1) We propose an end-to-end deep neural network model based on deep sparse representation to address the task of small-scale damage restoration in biological slice images. (2) We investigated the learnability of wavelet dictionaries and proposed a deep wavelet dictionary along with its updating algorithm. (3) We conducted an in-depth analysis of the relationship between sparse representation and deep neural networks, providing a comprehensive discussion on the interpretability of deep sparse representation.

The paper is organized as follows.

Wavelet transform can capture local features of an image from multiple perspectives and achieve energy concentration. But, wavelet basis functions are manually designed, and fixed basis functions may not adapt well to signal families. The wavelet function can be obtained through the discretization of the parameters a and b of the continuous wavelet function.

The response of the wavelet function to signal

In the

In the equation,

Deep learning has shown great charm with its powerful fitting ability [

Convolutional Sparse Coding (CSC) is one approach to sparse representation in image processing, which is supported by strong theoretical foundations and has good biological plausibility. However, in recent years, the performance of CSC has been surpassed by deep learning. Building a “deep” CSC model has potential application values in various fields such as image restoration [

Sparsity has been integrated into the development of deep neural networks [

Autoencoders, Restricted Boltzmann Machines, and other deep neural networks lack clear interpretability when solving

when the dictionary

Deep neural networks are developed based on the study of biological neural systems. The representation of signals in deep neural networks is non-linear, and the feature extraction is complex with a multi-scale network hierarchy. Deep convolutional neural networks (DCNN), as the representative of deep neural networks, use operations such as convolutional layers, linear layers, and pooling layers. The convolutional layer contains operations like convolution, activation, and bias. The linear layer performs linear transformations on the convolutional coefficients. The pooling layer implements the multi-resolution analysis of the neural network.

The convolutional layer simulates the functions and structures of biological neurons. For input signals

The starting point for the construction of sparse representation theory is the sparse response of neurons in the visual cortex of the brain to visual signals. The update of sparse coefficients is generally achieved by using an iterative thresholding function to realize sparsification, such as the ISTA algorithm shown in

As shown in

The above discussion confines the problem to a single-layer model. The multiscale nature of DCNN is evident, formally defined as follows:

By extending the basic deep sparse representation to multiple layers, we can obtain the Multi-layer Deep Sparse Representation (ML-DSR) model, formally defined below:

this indicates that deep sparse representation also has a multi-scale analysis mechanism.

DCNN and sparse representation share similarities in optimization and feedback mechanisms. In DCNN, backpropagation is applied in each iteration to update the network parameters with respect to the loss function, using the gradient descent method. Such optimization and feedback mechanisms ensure that the final solution moves towards minimizing the error. In sparse representation, the dictionary is updated based on the feedback from the reconstruction error, and the update direction is towards minimizing the loss. This optimization mechanism is evident in convolutional sparse models, improving the learnability of dictionaries and network parameters.

Starting from the basic form of deep sparse representation

Based on the Deep K-SVD algorithm, we constructed an artificial image dataset to validate its ability to effectively represent images through deep sparse representation, as shown in

Our end-to-end biological slice image inpainting model is shown in

Multiscale geometric analysis tools can achieve sparse representation of target images [

To meet the multiscale characteristics of wavelet decomposition, we first construct 2D discrete wavelet filters based on

The image

The dictionary should cover the entire space of x in space

The K-SVD algorithm tends to favor learning the parts of the image with obvious features and selects representative parts from many feature areas as atoms. Based on this characteristic, we decided to prioritize image blocks with clear edge texture features in the construction of the deep wavelet dictionary. Therefore, we use the mean gradient as a measuring factor, which can characterize the degree of grayscale change, to select potential image blocks. The formula is defined as follows:

Our sparsification strategy is to use a learnable ISTA algorithm to achieve sparse coding. The parameters of the ISTA algorithm rely entirely on manual design, which makes it difficult to achieve optimal sparse representation, especially for ill-posed inverse problems. To address this issue, Gregor et al. [

Due to the powerful representation capability of DCNN, the ISTA-Net algorithm [

ML-DSR shows that D or α can be a component of a layer, or can be used as the target signal. Therefore, we use the dictionary D as the target signal and deploy a shallow deep neural network model based on ML-DSR, which is updated in each iteration as defined below:

The issue of small-scale corruption can be described as

In the Deep K-SVD algorithm, the

As it is a global reconstruction for images with small-scale damage, information loss in the non-damaged areas of the image is inevitable. Therefore, we adopt the complement operation on the reconstructed result X, which allows us to focus on the reconstruction of the damaged areas and obtain better reconstruction results. The complement operation is defined as follows:

We used the frozen slice method to prepare the biological slices and stained them with uranyl acetate and lead citrate. Then we obtained 90 to 100 micrographs of lust microscopic slices by photographing them with a microscope. Processing the slice image by image enhancement methods such as shading correction and cropping. The process of slice image acquisition is shown in

Due to the complexity of the slicing process and the small size of the data, we extract a specified number of images from the training set to effectively address the problem of insufficient data. Specifically, a

Our model is built using the PyTorch framework and trained using the ADMM optimizer with a learning rate of 1e-4. The patch size is 8. The computer used for model training is equipped with an Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50 GHz and NVIDIA GeForce RTX 3080. For model testing, both MATLAB R2021a and Python 3.7.4 environments were used on a computer with an Intel(R) Core(TM) i7-9750H CPU @ 2.60 GHz and NVIDIA GeForce GTX 1650.

We design experiments from several aspects such as restoration effect, model complexity, and practicality, and use both simulated breakage and real breakage to show the restoration effect in the evaluation of inpainting effect.

As shown in

To evaluate the effectiveness of our proposed algorithm, we compared it with classical image inpainting algorithms and deep learning-based image inpainting algorithms, including NLABH [

PSNR | SSIM | RMSE | |
---|---|---|---|

NLABH | 53.6676 | 0.9995 | 0.0021 |

Shearlet | 34.5650 | 0.9295 | 0.0187 |

Mumford-Shah | 50.3852 | 0.9995 | 0.0030 |

K-SVD | 53.4971 | 0.0021 | |

LoBCoD | 53.0296 | 0.9992 | 0.0022 |

DIP | 39.3199 | 0.9670 | 0.0108 |

Proposed method | 0.9991 |

The time complexity of the algorithm is

Mumford-Shah | Mumford-Shah | Shearlet | LoBCoD | DIP | K-SVD | Proposed | |
---|---|---|---|---|---|---|---|

Time(s) | 22.1560 | 22.1560 | 175.4839 | 637.0052 | 5.0947e+03 | 2.1629e+04 | 12.8717 |

We conducted experimental comparisons on different forms of biological slice images to demonstrate the effectiveness of our proposed algorithm, as shown in

PSNR | SSIM | RMSE | |
---|---|---|---|

NLABH | 53.5701 | 0.9989 | 0.0021 |

Shearlet | 35.0695 | 0.8550 | 0.0176 |

Mumford-Shah | 53.1791 | 0.9991 | 0.0022 |

K-SVD | 45.5254 | 0.9974 | 0.0053 |

LoBCoD | 52.6031 | 0.9982 | 0.0023 |

DIP | 41.4306 | 0.9642 | 0.0085 |

Proposed |

To further highlight the advantages of our proposed model in terms of small scale and low complexity, we used two indicators: the total number of parameters and the number of floating-point operations per second (FLOPS). The total number of parameters reflects the scale of the model, while FLOPS reflects the complexity of the model. As shown in

DIP | Proposed | |
---|---|---|

Total parameters | 2.15 M | |

Total FLOPS | 1.930T |

Another issue worth discussing is whether it is possible to train the model with fewer data to cope with the difficulty of acquiring images. As shown in

Data size | Random sample | PSNR | SSIM | RMSE |
---|---|---|---|---|

16 | 4096 | 49.3497 | 0.9977 | 0.0034 |

32 | 8192 | 52.6963 | 0.9989 | 0.0023 |

44 | 11264 | 54.0027 | 0.9991 | 0.0020 |

Lastly, to demonstrate the practical application capability of the proposed algorithm in this paper, this chapter focuses on the application of the high-fidelity restoration algorithm to the problem of small-scale damages in biological slice images. As shown in

This article focuses on the issue of small-scale corruption in biological slice image preparation, particularly in the case of limited data. We analyzed the relationship between sparse representation and deep neural networks and established a deep network model based on deep sparse representation. Our proposed model can effectively inpainting small-scale corruption in biological slice images while preserving the edge texture and contour structure of the slice images. We conducted tests using simulated damages, compared with other methods such as PDE-based methods, Shearlet algorithm, DIP algorithm, K-SVD algorithm, and sparse coding algorithm, our proposed model achieved good results in terms of effectiveness, time, and model scale. And then we demonstrated the application capability of our proposed method in addressing true corruptions in biological slice images that high-fidelity inpainting has been achieved.

Unlike algorithms such as K-SVD and LoBCOD, the algorithm proposed in this article not only achieves high-fidelity inpainting of biological slice images but also benefits from well-time complexity, making it valuable for practical applications. The proposed algorithm can also be effectively applied to other cross-sectional image restoration tasks, such as MRI and CT scan images. In future work, we will focus on explainable deep learning research based on deep sparse representation. Furthermore, the efficiency of the proposed model still needs further improvement, we will focus on dictionary learning with an emphasis on the learnable wavelet dictionary.

The authors extend their appreciation to the anonymous reviewers for their constructive comments and suggestions.

This work was supported by the National Natural Science Foundation of China (Grant No. 61871380), the Shandong Provincial Natural Science Foundation (Grant No. ZR2020MF019), and Beijing Natural Science Foundation (Grant No. 4172034).

Study conception and design: Haitao Hu, Shuli Mei; data collection: Haitao Hu, Hongmei Ma; analysis and interpretation of results: Haitao Hu, Shuli Mei, Hongmei Ma; draft manuscript preparation: Haitao Hu, Shuli Mei, Hongmei Ma. All authors reviewed the results and approved the final version of the manuscript.

The data that support the findings of this study are available from the corresponding author upon reasonable request.

The authors declare that they have no conflicts of interest to report regarding the present study.