Vector tiles

Vector tiles contain vector representations of data across a range of scales and can be used to visualize geometries in a Spark DataFrame. Using the vector-tile format when writing a DataFrame will the write a directory of tiles to an output location. These tiles can be visualized in applications like ArcGIS Pro and MapBox.

Below is an example of a vector tile result stored in PBF files.

vector tiles pbf layout

This layout includes a collection of folders with subfolders and .pbf files, as well as the vector-tile.json file that contains metadata like the path to the tiles folder, layer name, spatial reference and symbology.

The following table shows differences in terms between GeoAnalytics Engine and GIS software (such as ArcGIS Pro).

DescriptionGeoAnalytics Engine termGIS software term
A single record in your dataset.Record or rowFeatures
This is a DataFrame column or otherwise a field in GIS termsColumnField or attribute

Create vector tiles in GeoAnalytics Engine

When you write to vector tiles, specify the export format as vector-tile. There are also required and optional parameters explained in the table below.

Parameter nameExplanationTypeExample
layerNameThe name of the output vector tile layer.Required"earthquakes"
minLevelThe minimum zoom level created.
If a minScale is provided, minLevel will be ignored.
The default is 0.
Optional5

In this case, zoom levels from 5 to 15
will be included in the vector tile result.
The folder will include subfolders 5-15.
maxLevelThe maximum zoom level created.
The default is 15.
Optional12

In this case, zoom levels from 0 to 12
will be included in the vector tile result.
The folder will include subfolders 0-12.
minScaleThe minimum (smallest) scale at which the tiles are created, for zooming out as far as possible.
This parameter accepts the reciprocal of the scale, the scale itself is 1:minScale.
If the minScale parameter is not provided, the minLevel will be used.
Optional295828764

A value of 295828764 specifies a scale of 1:295828764.
When zooming in from whole-world extent, data will start to appear at a scale of 1:295828764.
maxScaleThe maximum (largest) scale at which records will display when zooming in.
This parameter accepts the reciprocal of the scale, the scale itself is 1:maxScale.
If maxScale is greater than the scale corresponding to maxLevel, then the high scale range is supported for ArcGIS viewers by overzoom, without writing additional levels of tiles.
Clients that do not support overzoom will only zoom in to the scale corresponding to maxLevel. If the maxScale parameter is not provided, the maxLevel will be used.
Optional564

A value of 564 specifies a scale of 1:564. ArcGIS Pro will overzoom in to 1:564; other clients may not overzoom, maxLevel should be increased in this case.
maxPointsPerTileThe maximum number of records that will be included
in a single tile. If the number of records exceeds this
limit, not all records will draw. In this case, the records will
be chosen arbitrarily.
The default is 50000.
Set this parameter to -1 to include all features in the vector tile result.
Optional1000
prioFieldThe prioritization field determines which records will be
prioritized to be included in the vector tile result. Use this
field when the number of records in a tile exceeds the maxPointsPerTile.
Optional"magnitude"

In this example, if the number of
records in a tile exceeds the maxPointsPerTile,
the records with highest magnitude
value will be prioritized to be included
in the vector tile result.
attributesThe columns to be included in the vector tiles.
By default, no columns are included.
Optional"magnitude,depth"

In this example, two columns will be
shown in client applications, such as
the popups in ArcGIS Pro.

The example below writes the DataFrame df to vector tiles with a maximum zoom level of 12, a maximum of 10,000 points per tile, and includes the magnitude field as an attribute in the result.

Python
Use dark colors for code blocksCopy
         
1
2
3
4
5
6
7
8
9
df.write.format("vector-tile") \
				.option("layerName", "earthquakes") \
				.option("maxLevel", 12) \
				.option("maxPointsPerTile", 10000) \
				.option("prioField", "magnitude") \
				.option("attributes", "magnitude") \
				.save(r"C:\vector_tile_output\earthquakes")

Visualize vector tiles

Most client applications require that vector tiles are hosted on a web server to visualize them. There are multiples ways to host vector tiles. See the following tutorials for some example workflows:

Visualize hosted vector tiles in ArcGIS Pro

The steps below outline how to view vector tiles in ArcGIS Pro.

  1. Create a new map or open an existing one.

  2. Click Map > Add Data > Data From Path.

    vector tiles add data

  3. For the path provide the URL to the hosted vector-tile.json file that was written by GeoAnalytics Engine for the vector tiles you want to visualize. For service type select "Vector Tile Service" and click Add.

    vector tiles add data dialog

The geometries will draw on the map and you will see a vector tile layer in the Contents pane.

You can also enable popups on vector tile layers and view attribute values. To enable popups, right-click on the vector tile layer in Contents and select "Enable Pop-ups".

vector tiles enable popups

Visualize hosted vector tiles in Mapbox

  1. Create a basic HTML application using the Mapbox API.

  2. Add the vector tile layer that references the hosted location.

    Use dark colors for code blocksCopy
          
    1
    2
    3
    4
    5
    6
    window.map.addSource('tiles', {
    "type": "vector",
    "minzoom": 0,
    "maxzoom": 9,
    "tiles": [`{z}/{x}/{y}.pbf`]
    });
    
  3. Add the layer to the map along with properties.

    Use dark colors for code blocksCopy
             
    1
    2
    3
    4
    5
    6
    7
    8
    9
    >map.addLayer({
     "id": "test",
     "type": "circle",
     "source": "tiles",
     "source-layer": "earthquakes",
     "paint": {
     "circle-color": "red"
     }
     }, "waterway-label");
    

    Example Mapbox app:

    HTML
    Use dark colors for code blocksCopy
                                                         
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    <!DOCTYPE html>
    <html>
    <head>
      <meta charset='utf-8' />
      <title>Display a map</title>
      <meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
      <script src='https://api.tiles.mapbox.com/mapbox-gl-js/v1.12.0/mapbox-gl.js'></script>
      <link href='https://api.tiles.mapbox.com/mapbox-gl-js/v1.12.0/mapbox-gl.css' rel='stylesheet' />
      <style>
        body {
          margin: 0;
          padding: 0;
        }
        #map {
          position: absolute;
          top: 0;
          bottom: 0;
          width: 100%;
        }
      </style>
    </head>
    <body>
      <div id='map'></div>
      <script>
        const agsKey = '<YOUR_KEY>';
        const agsStyle = 'ArcGIS:Community';
        var map = new mapboxgl.Map({
          container: 'map', // container id
          style: `https://basemaps-api.arcgis.com/arcgis/rest/services/styles/${agsStyle}/?type=style&token=${agsKey}`,
          center: [-85.50, 40],
    	  zoom: 0
        });
    
        map.on('load', function () {
          window.map.addSource('tiles', {
            "type": "vector",
            "minzoom": 0,
            "maxzoom": 15,
            "tiles": [`http://<YOUR_PATH>/{z}/{y}/{x}.pbf`] // Add the path to your hosted tiles here
          });
          map.addLayer({
            "id": "test",
            "type": "circle",
            "source": "tiles",
            "source-layer": "vector_layer",
            "paint": {
              "circle-color": "red"
            }
          });
        });
      </script>
    </body>
    </html>

Best practices

If you are using a large, dense dataset and wish to export multiple zoom levels, it may take a long time to export and render the vector tiles. In this case, it is recommended to use the maxPointsPerTile parameter to limit the number of records included in each tile.

The records to include are prioritized in an arbitrary way by default. You can also use a prioritization field in cases where you wish to include and display certain records first.

For example, if you are visualizing a dataset of world cities, using the population as the prioritization field will prioritize displaying the largest cities first in the shallower levels.

In case you want to get more detailed information about individual records in your vector tile layer, you can include attributes when exporting in GeoAnalytics Engine and then enable popups in ArcGIS Pro.

Limitations

Only points are supported. The only supported vector tile layout is folders of .pbf files.

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.