RT_Apply

RT_Apply takes a raster column, an integer or a numeric column that represents the band ID, and a user-defined function. It applies the user-defined function to each pixel value in the specified band and returns a raster column.

The function is a Spark SQL lambda expression that takes exactly one argument (px). The lambda is translated to a Spark SQL lambda of the form: px -> <SQL expression>. All Spark SQL scalar functions are available inside the lambda, including spatial SQL functions.

The lambda argument px is a struct with the following fields:

• px.value—value of the current pixel in the target band.
• px.values— array of pixel values across all bands at this pixel. Indexing is 0-based, so px.values[0] is the value of the first band at this pixel.
• px.row— row index of the pixel (0-based).
• px.column— column index of the pixel (0-based).
• px.center— geometry representing the pixel center.

Band IDs are 1-based. If the band ID refers to an existing band, that band is overwritten. If the band ID is exactly one greater than the current number of bands, a new band is appended. All other band IDs are invalid and result in a null raster.

When using spatial SQL functions inside RT_Apply, e.g., ST_Contains, ST_Distance, the geometry representing the pixel center (px.center) is assumed to be in the same spatial reference as the geometry used in the lambda expression. If the spatial references are mismatched, transform the raster with RT_transform or the geometry with ST_transform to ensure their spatial references are the same before calling RT_Apply.

You can use RT_Apply to update pixel values (e.g., recode, scale, or, transform), append a new band, or mask pixels by returning NULL for pixels that should be invalid.

For more details, go to the GeoAnalytics Engine API reference for apply.

Examples

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

from geoanalytics.raster import functions as RT
from pyspark.sql import functions as F

data = [([1,2,3,4], )]
df = spark.createDataFrame(data, ["pixels"]) \
     .withColumn("raster", RT.create_raster("pixels", 2, 2, "int32"))

# Update pixel values
r1 = df.withColumn("raster", RT.apply('raster', 1, lambda px: px.value * 10))
r1.select(RT.band_values(F.col("raster"), 1).alias("band_values")).show(truncate=False)

# Add a band
r2 = df.withColumn("raster", RT.apply('raster', 2, lambda px: px.values[0] + 1))
r2.select(RT.num_bands(F.col("raster")).alias("num_bands"),
          RT.band_values(F.col("raster"), 1).alias("band1_values"),
          RT.band_values(F.col("raster"), 2).alias("band2_values")).show(truncate=False)

# Apply a mask
r3 = df.withColumn("raster", RT.apply('raster', 1, lambda px: F.when(px.value == 2, px.value).otherwise(F.lit(None))))
r3.select(RT.band_values(F.col("raster"), 1).alias("band_values"),
          RT.band_mask(F.col("raster"), 1).alias("band_mask")).show(truncate = False)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
+------------------------+
|band_values             |
+------------------------+
|[10.0, 20.0, 30.0, 40.0]|
+------------------------+

+---------+--------------------+--------------------+
|num_bands|band1_values        |band2_values        |
+---------+--------------------+--------------------+
|2        |[1.0, 2.0, 3.0, 4.0]|[2.0, 3.0, 4.0, 5.0]|
+---------+--------------------+--------------------+

+--------------------+---------------------------+
|band_values         |band_mask                  |
+--------------------+---------------------------+
|[0.0, 2.0, 0.0, 0.0]|[false, true, false, false]|
+--------------------+---------------------------+

Version table

Related topics

• Raster functions
• Raster data