With increasingly more smart cameras deployed in infrastructure and commercial buildings, 3D reconstruction can quickly obtain cities’ information and improve the efficiency of government services. Images collected in outdoor hazy environments are prone to color distortion and low contrast; thus, the desired visual effect cannot be achieved and the difficulty of target detection is increased. Artificial intelligence (AI) solutions provide great help for dehazy images, which can automatically identify patterns or monitor the environment. Therefore, we propose a 3D reconstruction method of dehazed images for smart cities based on deep learning. First, we propose a fine transmission image deep convolutional regression network (FT-DCRN) dehazing algorithm that uses fine transmission image and atmospheric light value to compute dehazed image. The DCRN is used to obtain the coarse transmission image, which can not only expand the receptive field of the network but also retain the features to maintain the nonlinearity of the overall network. The fine transmission image is obtained by refining the coarse transmission image using a guided filter. The atmospheric light value is estimated according to the position and brightness of the pixels in the original hazy image. Second, we use the dehazed images generated by the FT-DCRN dehazing algorithm for 3D reconstruction. An advanced relaxed iterative fine matching based on the structure from motion (ARI-SFM) algorithm is proposed. The ARI-SFM algorithm, which obtains the fine matching corner pairs and reduces the number of iterations, establishes an accurate one-to-one matching corner relationship. The experimental results show that our FT-DCRN dehazing algorithm improves the accuracy compared to other representative algorithms. In addition, the ARI-SFM algorithm guarantees the precision and improves the efficiency.

Artificial intelligence (AI) has recently become very popular, and a wide range of applications use this technique [

In hazy environments, the reflected light of an object is attenuated before it reaches the camera or monitoring equipment, resulting in the degradation of the quality of an outdoor image [

We propose a fine transmission image deep convolutional regression network (FT-DCRN) dehazing algorithm that uses fine transmission image [

With increasingly more smart cameras deployed in infrastructure and commercial buildings, 3D reconstruction can quickly obtain information on cities and geographical regions [

The contributions of this paper are listed as follows:

We use a deep learning algorithm for dehazed images in smart cities. The FT-DCRN dehazing algorithm is proposed, which uses fine transmission image and atmospheric light value to compute dehazed image. First, this paper proposes a DCRN algorithm to obtain the coarse transmission image. The DCRN can not only expand the receptive field of the network, but can also retain the features to maintain the nonlinearity of the overall network. Second, the fine transmission image is obtained by refining the coarse transmission image using a guided filter. The guided filter is used to optimize the coarse transmission image to improve the accuracy of dehazed images.

We perform 3D reconstruction using the dehazed images generated from the FT-DCRN algorithm. The ARI-SFM algorithm is proposed, which can obtain fine matching corner pairs and reduce the number of iterations. Compared with other representative algorithms, the ARI-SFM algorithm establishes an accurate one-to-one corner matching relationship, which guarantees the precision and improves the efficiency.

The purpose of a dehazing algorithm is to restore a sharp image from a blurred image caused by haze. Deep learning algorithms can provide great help for dehazy images [

Step 1: Obtain the coarse transmission image. Input the hazy images, and the coarse transmission image is obtained by using the DCRN dehazing algorithm.

Step 2: Compute the fine transmission image. The fine transmission image is obtained by refining the coarse transmission image using a guided filter.

Step 3: Estimate the atmospheric light value. The atmospheric light value [

Step 4: Compute the dehazed image. According to the obtained fine transmission image and atmospheric light value, the dehazed image is inverted using the atmospheric physical scattering model.

To obtain a coarse transmission image, this paper proposes a DCRN dehazing algorithm, and the overall network structure is shown in

The DCRN is similar to the encoder–decoder network. The core unit of the encoder network is the convolutional unit (Conv), which is mainly composed of a convolutional layer [

The features of the overall network are extracted by the encoder network. The decoder network is used to ensure the size of the output transmission image and retain the features to maintain the nonlinearity of the overall network [

The main characteristic of an end-to-end network is that the input and output of the network are identical in size. However, due to the use of two pooling layers [

To solve the problem of information loss from the original image, this paper uses a deconvolutional layer [

In this paper, a guided filter is used to optimize the coarse transmission image to improve the accuracy of dehazed images [

where _{k}_{k}

where

where _{k}

When estimating the atmospheric light value, He et al. [

To solve the above problems, this paper uses the method of combining the pixel position and brightness to estimate the atmospheric light value. The relative height of each pixel is defined as

The process determines the pixels with the probability value

It is important to solve the image matching problem in the 3D SFM reconstruction algorithm. In the image matching process, the coarse matching relationship between corners is established by using the zero mean normalized cross-correlation method [

To establish an accurate one-to-one corner matching relationship, an advanced relaxed iterative (ARI) algorithm is proposed, which obtains fine matching corner pairs and reduces the number of iterations. The flowchart of the ARI algorithm is shown in

The steps of the ARI algorithm are as follows:

Step 1: Calculate the matching strength of coarse matching pairs. The matching strength is used as the indicator for fine matching corner selection [

Step 2: Judge the uniqueness of the corner pairs. The matching pairs are sorted according to the matching strength from large to small. We select the corner pairs _{M}_{1i}, _{2j}) and _{P}_{1i}, _{2j}) and use it to measure the uniqueness of the corner matching.

The value range of _{P}_{M}_{1i}, _{2j}) and _{P}_{1i}, _{2j}), all matching pairs in the set are sorted. If all corners are in the top 60% of the two items, the corner pairs are accurate matching pairs. Then, proceed to Step 4; otherwise, proceed to Step 3.

Step 3: Delete the correct corner after fine matching, and return to Step 1.

Step 4: Output the fine matching corner pairs.

The initial matching corner pairs are represented as (_{1i}, _{2j}), where _{1i} is the corner of image _{1} and _{2j} is the corner of image _{2}. _{1i}) and _{2j}) are neighborhoods with point _{1i} and point _{2j} as centers, respectively, and _{1k}, _{2f}) is the correct matching pair, there must be more correct matching pairs (_{1k}, _{2f}) in its neighborhoods

Condition 1: Only the matching corner pair (_{1k}, _{2f}) affects the matching corner pair (_{1i}, _{2j}).

Condition 2: The angle between

If the matching corner pairs (_{1i}, _{2j}) and (_{1k}, _{2f}) satisfy the above two conditions, the matching strength is calculated using formula

_{1i}, _{2j}) and (_{1k}, _{2f}), respectively.

_{1k}, _{2f}) to (_{1i}, _{2j}) is a power function with a negative exponent and relative distance deviation _{1k}, _{2f}) is ignored.

In this paper, synthetic hazy images and real hazy images are used to train and test the performance of the FT-DCRN dehazing algorithm. First, we adopt the Make3D dataset [

To verify the efficiency and accuracy of the ARI-SFM algorithm, the algorithm is implemented on an experimental platform with 64-bit Windows 10, an Intel(R) Core(TM) i5-10210U@1.60 GHZ CPU, and 8.00 GB of memory; the development platform is MATLAB R2018b.

Given a random value for the transmission image

where

To verify the effect of the FT-DCRN dehazing algorithm on the synthetic hazy images, the results of the algorithm are compared with some representative algorithms. Because different deep learning algorithms have their own advantages,we adopt the Tang’s algorithm [

To verify the effect of the FT-DCRN dehazing algorithm on real hazy images, we analyze 1200 real hazy images of outdoor scenes, such as building, garden and parking area. We compare the results of the FT-DCRN with those of Tang’s algorithm, Cai’s algorithm and Li’s algorithm.

To perform the quantitative evaluation, synthetic hazy images and real hazy images are selected. We adopt the structural similarity (SSIM) [

Indicator | Tang’s algorithm | Cai’s algorithm | Li’s algorithm | Our approach |
---|---|---|---|---|

SSIM | 0.8322 | 0.8522 | 0.9023 | 0.9128 |

PSNR | 24.2644 | 26.5940 | 27.1845 | 27.8906 |

IE | 7.2173 | 7.2802 | 7.3429 | 7.4018 |

Indicator | Tang’s algorithm | Cai’s algorithm | Li’s algorithm | Our approach |
---|---|---|---|---|

SSIM | 0.8452 | 0.8756 | 0.9048 | 0.9230 |

PSNR | 24.3325 | 26.7622 | 27.9630 | 28.2588 |

IE | 7.0518 | 7.2090 | 7.3281 | 7.4979 |

Indicator | Tang’s algorithm | Cai’s algorithm | Li’s algorithm | Our approach |
---|---|---|---|---|

SSIM | 0.8472 | 0.8822 | 0.9123 | 0.9278 |

PSNR | 25.2511 | 26.0771 | 26.6549 | 27.3971 |

IE | 7.2811 | 7.3573 | 7.4412 | 7.5134 |

Indicator | Tang’s algorithm | Cai’s algorithm | Li’s algorithm | Our approach |
---|---|---|---|---|

SSIM | 0.8485 | 0.8863 | 0.9194 | 0.9283 |

PSNR | 25.0496 | 26.2763 | 27.2319 | 27.4703 |

IE | 7.0188 | 7.1977 | 7.3857 | 7.4603 |

The results of Tang’s algorithm provided relatively low values for each indicator. Tang’s algorithm did not use the texture features in the image, which creates certain limitations on the dehazing effect. Cai’s algorithm and Li’s algorithm have significantly higher SSIM and PSNR values than those of Tang’s algorithm. However, Cai’s algorithm and Li’s algorithm do not result in normal visual color effects in

After using the ARI-SFM algorithm, the one-to-one relationship between corners is determined, and one-to-many relationship almost does not exist. In this experiment, we selected 6 images shown in

To verify the matching efficiency of the ARI-SFM algorithm, the results of the algorithm are compared with some representative algorithms. Because different image matching algorithms have their own advantages, we adopt the Hossain’s algorithm [

Building | Hossain’s algorithm | Zhang’s algorithm | Zhou’s algorithm | Our approach |
---|---|---|---|---|

Matching accuracy (%) | 83 | 85 | 86 | 90 |

Match time (s) | 15.69 | 14.77 | 13.56 | 10.83 |

AI solutions can provide great help for dehazing images, which can automatically identify patterns or monitor the environment. Therefore, we propose a 3D reconstruction method for dehazed images for smart cities based on deep learning. First, we propose an FT-DCRN dehazing algorithm that uses fine transmission images and atmospheric light values to compute dehazed images. The DCRN is used to obtain the coarse transmission image, which can not only expand the receptive field of the network, but can also retain the features to maintain the nonlinearity of the overall network. The fine transmission image is obtained by refining the coarse transmission image using a guided filter. The atmospheric light value is estimated according to the position and brightness of the pixels in the original hazy image. Second, we use the dehazed images generated by the FT-DCRN dehazing algorithm for 3D reconstruction. The ARI-SFM algorithm, which obtains the fine matching corner pairs and reduces the number of iterations, establishes an accurate one-to-one matching corner relationship. The experimental results show that our FT-DCRN dehazing algorithm improves the accuracy compared to other representative algorithms. In addition, the ARI-SFM algorithm guarantees the precision and improves the efficiency.

Developing AI systems supporting smart cities requires considerable data. Through the acquisition of effective information, smart cities can truly become sustainable developments. By 2021, one billion smart cameras will be deployed in infrastructure and commercial buildings. The large amount of raw data collected is far beyond the scope that can be viewed, processed or analyzed manually. Through the machine learning training process, images can be analyzed for city planning and development. AI algorithms have become the developmental trend and key point of smart cities [

The author would like to thank the anonymous reviewers for their valuable comments and suggestions that improve the presentation of this paper.