- 🔬 Data Science
- 🥠 Deep Learning and image translation
Introduction
The goal of this notebook is to detect and replace the cloud-contaminated region of the satellite images. Clouds often create obstructions in satellite imagery that can hinder results in processes like land cover classification and time series analysis. From a general perspective, this can be seen as an image denoising problem, where clouds are considered as noise in relation to the land surface in images. In this notebook, we will detect and replace cloud-contaminated regions of satellite imagery by combining a pretrained deep learning model, available in Living Atlas, with an image processing technique available in raster functions.
Necessary imports
import matplotlib.pyplot as plt
import arcpy
from arcgis import GIS
from arcgis.learn import prepare_data, Pix2PixHD, Pix2PixConnect to your GIS
gis = GIS('Home')The rasters used for exporting the training dataset are provided below:
cloud_covered = gis.content.get('b89f64b064e84423b596a0252c044921')
cloud_covered
Layer Package by api_data_owner
cloud_free = gis.content.get('5993a05593c945b0854ec87581750432')
cloud_free
Cloud removal workflow
The cloud removal workflow employed in this notebook consists of three key steps:
- Generate cloud mask - To initiate the cloud removal process, a cloud mask is generated using a pretrained model available in the Living Atlas. This cloud mask serves as a guide for identifying and isolating cloud-contaminated pixels within the target raster.
- Reclassification of cloud types - the generated cloud mask in step 1 has different cloud types. In this step, we assign a value of 1 to pixels associated with different cloud types, while assigning a value of 0 to all other non-cloud pixels.
- Use the stepwise local radiometric adjustment (SLRA) raster function - The final step involves the application of the Stepwise Local Radiometric Adjustment (SLRA) raster function. This function takes as input the original raster, a replacement raster, and the reclassified cloud mask from step 2. The function results in removal of clouds in input rasters.
Generate cloud mask
The cloud mask generation pre-trained model, trained on sentinel-L2A imagery is used to generate cloud mask for input raster. The pre-trained model is used in Classify Pixels Using Deep Learning tool resulting in a segmented cloud mask raster. Other ways of generating cloud mask include using sentinel2 cloud detector by ESA. The cloud mask is used by Stepwise local radiometric adjustment (SLRA) raster function as guidance for removal of cloud-contaminated pixels in the target raster.
gis.content.get('1e1ec9602f4743108708ccdf362e3c48')
Reclassify cloud types in the cloud mask
The pretrained cloud mask model will classify clouds into 3 classes. low, medium, and high density. As we want to remove all cloud contaminated regions, we will reclassify these 3 classes into one single class.
arcpy.env.workspace = r"location\to\arcgis_pro_project_geodatabase\cloud_removal.gdb"
cloud_mask = arcpy.Raster(r"cloud_mask.tif")
cloud_mask_arr = arcpy.RasterToNumPyArray(cloud_mask)
inRaster = "cloud_mask.tif"
field = "Value"
remapString = "1 1;2 1;3 1"
outRaster = "reclass_cloud_mask"
# Execute Reclassify
arcpy.ddd.Reclassify(cloud_mask, field, remapString, outRaster, "DATA")Visualize the generated cloud mask
plt.imshow(cloud_mask_arr[50:-50,50:-50])
plt.colorbar()
plt.show()
Use the stepwise local radiometric adjustment raster function
Next, we will use the Python raster function to blend areas using a Stepwise Local Radiometric Adjustment algorithm Li .et al, 2019. Stepwise Local Radiometric Adjustment is used to fill contaminated areas and is performed on each mask region of the target image. Using the replacement raster, the tool will replace pixels that are covered with clouds.
The raster functions can be downloaded as a python (.py) file from raster functions in the functions folder. The StepwiseLocalRadiometricAdjustment.py file can be opened using Open Python Raster function as shown in figure below. Once opened, save it using Save As available in drop down, it will result in a .rft(raster function template) file . You can experiment with different values for the Size of Window and Buffer Mask parameters to achieve varying results based on your requirements and cloud coverage.
Input Raster: cloud covered imageryInput Replacement raster: replacement rasterInput Mask: generated cloud maskSize of Window: default set to 80, can be any multiple of 20Buffer Mask: any value between 0-15, accordiong to coverage of cloud mask
The saved raster function can then be exported as an rft(raster function template). rft is used in the Generate Raster From Raster Function tool to get the final results, as this tool is designed for raster processing using multiple threads to help speed up the processing.
arcpy.management.GenerateRasterFromRasterFunction(r"path\to\the\rft_location\Stepwise Local Radiometric Adjustment.rft.xml", r"path\to\output\location", None, None, "TIFF", "CURRENT_SLICE")
Conclusion
In this notebook, we have demonstrated a method to remove clouds from satellite imagery with a combination of a pretrained deep learning model and Esri raster functions. Further, it demonstrated how you can use your developed python raster functions in ArcGIS Pro.
References
- Li, Zhiwei, Huanfeng Shen, Qing Cheng, Wei Li, and Liangpei Zhang. "Thick cloud removal in high-resolution satellite images using stepwise radiometric adjustment and residual correction." Remote Sensing 11, no. 16 (2019): 1925.