Raster analysis - advanced concepts¶

Raster functions are lightweight and process only the pixels visible on your screen, in memory, without creating intermediate files. They are powerful because you can chain them together and apply them on huge rasters and mosaics on the fly. This guide will introduce you to the different capabilities of Imagery Layers and how to apply raster functions. As a sample, it uses Landsat imagery data.

Note: This guide requires bokeh python plotting library. Install it in your conda environment using the command below. This command should be typed in the terminal, not in the notebook

conda install bokeh

Access Landsat imagery¶

ArcGIS Online provides multispectral landsat imagery layer, that we'll be using for this tutorial. Let's connect to ArcGIS Online and query for the "Landsat Multispectral" imagery layer:

In [1]:

from arcgis.gis import GIS

In [2]:

gis = GIS('https://pythonapi.playground.esri.com/portal', 'arcgis_python', 'amazing_arcgis_123')

In [3]:

landsat_item = gis.content.search('Multispectral Landsat', 'Imagery Layer', outside_org=True)[0]

In [12]:

landsat_item = gis.content.search(query='Multispectral Landsat AND owner:esri_livingatlas', 
                                  item_type='Imagery Layer', outside_org=True)[0]

In [13]:

landsat_item

Out[13]:

Landsat GLS Multispectral
Landsat GLS 30 and 60m multispectral and multitemporal imagery with on-the-fly renderings and indices for visualization and analysis.

Imagery Layer by esri_livingatlas
Last Modified: May 13, 2020
0 comments, 0 views

View Landsat imagery layer item description¶

In [11]:

from IPython.display import HTML
HTML(landsat_item.description)

Out[11]:

This layer includes Landsat GLS multispectral imagery for use in visualization and analysis. This layer is time enabled and includes a number band combinations and indices rendered on demand. The layer includes Landsat 7 ETM+, Landsat 5 TM, and Landsat 4 imagery at 30m, and includes Landsat MSS imagery at 60m.

Geographic Coverage

World-wide imagery coverage.

Temporal Coverage

This imagery layer includes data from epochs 2010, 2005, 2000, 1990 and 1975.

Analysis Ready

This imagery layer is analysis ready with Top of Atmosphere (TOA) correction applied.
The TOA reflectance values (ranging 0 – 1 by default) are scaled using a range of 0 – 10,000.
The scale is equivalent to other TOA reflectance products, including those provided by the USGS.

Image Selection/Filtering

Newer images are displayed by default on top.
The entire archive is accessible via custom filtering.
A number of fields are available for filtering, including Acquisition Date, Estimated Cloud Cover, and Product ID.
By setting the filter to Best is lesser than QQQQ one can control to see the best N scenes, where QQQQ=N*1million.

NOTE: Turning off all filters, and loading the entire archive, may affect performance.

Visual Rendering

Default layer is Agriculture (bands 5,4,1) with Dynamic Range Adjustment (DRA). Brighter green indicates more vigorous vegetation.
The DRA version of each layer enables visualization of the full dynamic range of the images.
Rendering (or display) of band combinations and calculated indices is done on-the-fly from the source images via Raster Functions.
Various pre-defined Raster Functions can be selected or custom functions can be created.
Similar Imagery Layers are also available: Panchromatic and Pansharpened.

Multispectral Bands

Band	Wavelength (µm)
	Landsat 7 ETM+	Landsat 4-5 TM	Landsat MSS
1	0.45 – 0.52	0.45 – 0.52	N/A
2	0.52 – 0.60	0.52 – 0.60	0.5 – 0.6
3	0.63 – 0.69	0.63 – 0.69	0.6 – 0.7
4	0.77 – 0.90	0.76 – 0.90	0.7 – 0.8
5	1.55 – 1.75	1.55 – 1.75	0.8 – 1.1
6	2.09 – 2.35	2.08 – 2.35	N/A

Other Layer Usage Notes...

Overviews exist with a spatial resolution of 300m and are updated weekly based on the best and latest imagery available at that time.
To work with individual source images at all scales, either use the ‘Lock Raster’ functionality or add a query filter to restrict the display to a specified image or group of images.

NOTE: ‘Lock Raster’ should only be used on the layer for short periods of time, as the imagery and associated record Object IDs may change daily.

Images can be exported up to a maximum of 2,000 columns x 2,000 rows per request.
This ArcGIS Server dynamic Imagery Layer can be used in Web Maps and ArcGIS Desktop as well as Web and Mobile applications using the REST based Image Services API.
WCS and WMS compatibility means this imagery can be consumed as WCS or WMS services.
Landsat Web App via Unlock Earth's Secrets.

Data Source

Landsat imagery is sourced from the U.S. Geological Survey (USGS) and the National Aeronautics and Space Administration (NASA). Data is hosted by the Amazon Web Services as part of their Public Data Sets program. Users can access full scenes from Landsat on AWS, or alternatively access LandsatLook to review and download full scenes from the complete USGS archive.

The Global Land Survey includes images from Landsat 1 through Landsat 7. Band numbers and band combinations differ from those of Landsat 8, but have been mapped to the most appropriate band as in the above table. For more information about the Global Land Survey, visit GLS.

Access the layers available with the Landsat Imagery Layer item¶

This imagery layer item contains the imagery layer that we'll be using for this tutorial. Let's save a reference to the layer in the landsat variable. Querying the variable will in the Jupyter notebook will quickly render it as an image

In [6]:

landsat = landsat_item.layers[0]
landsat

Out[6]:

Explore different wavelength bands¶

In [33]:

import pandas as pd

In [8]:

pd.DataFrame(landsat.key_properties()['BandProperties'])

Out[8]:

	BandIndex	BandName	DatasetTag	WavelengthMax	WavelengthMin
0	0	CoastalAerosol	MS	450	430
1	1	Blue	MS	510	450
2	2	Green	MS	590	530
3	3	Red	MS	670	640
4	4	NearInfrared	MS	880	850
5	5	ShortWaveInfrared_1	MS	1650	1570
6	6	ShortWaveInfrared_2	MS	2290	2110
7	7	Cirrus	MS	1380	1360

Let's visualize the layer by creating a map widget around our area of interest and adding the Landsat Imagery layer to it:

In [9]:

m = gis.map('Redlands, CA')
m

In [10]:

m.add_layer(landsat)

Apply built-in raster functions¶

The multispectral imagery layer can be rendered using several different raster functions (also known as raster function templates). Each raster function template incorporates different satellite bands to highlight different land cover features. Obtain list of predefined raster function templates defined by the service backing the imagery layer:

In [11]:

for rasterfunc in landsat.properties.rasterFunctionInfos:
    print(rasterfunc.name)

Agriculture with DRA
Bathymetric with DRA
Color Infrared with DRA
Geology with DRA
Natural Color with DRA
Short-wave Infrared with DRA
Agriculture
Bathymetric
Color Infrared
Geology
Natural Color
Short-wave Infrared
NDVI Colorized
Normalized Difference Moisture Index Colorized
NDVI Raw
NBR Raw
None

Let's apply the 'Color Infrared with DRA' raster function to visualize the color infrared view. This can be done using the apply function in the arcgis.raster.functions module. This function applies a server defined raster function template, given it's name, to the Imagery layer.

In [12]:

from arcgis.raster.functions import apply

In [13]:

color_infrared = apply(landsat, 'Color Infrared with DRA')

In [14]:

m = gis.map('Redlands, CA')
m.add_layer(color_infrared)
m

A true-color image uses red, green and blue satellite bands to create an image that looks like a photograph. The color infrared view, on the other hand, uses the near infrared, red and green satellite bands to create an image. As a result, vegetation appears bright red in the image above.

Interactive raster processing in Jupyter Notebook¶

Imagery layers can be queried in the Jupyter Notebok. This displays a raster representation of the layer as an image:

In [16]:

color_infrared

Out[16]:

ArcGIS API for Python Guide