Shapefile

Shapefile is an Esri vector data storage format commonly used in geospatial analysis and GIS software applications. For more information on shapefiles, see the Shapefile format specification.

Shapefile data can be stored in a distributed file system such as HDFS, cloud storage such as S3, or a local directory.

When loading shapefile data, a geometry column will be automatically created in the result DataFrame and its spatial reference set. Shapefile supports point, line, polygon, and multipart collections of point, line, or polygon geometries. After loading shapefiles into a Spark DataFrame, you can perform analysis and visualize the data by using the SQL functions and tools available in GeoAnalytics for Microsoft Fabric in addition to functions offered in Spark. Once you save a DataFrame as shapefile, you can store the files or access and visualize them through other systems.

The following table shows examples of the Python syntax for loading and saving shapefiles with GeoAnalytics for Microsoft Fabric, where path is a path to a directory of shapefiles.

Load	Save
`spark.read.format("shapefile").load(path)`	`df.write.format("shapefile").save(path)`
`spark.read.load(path, format="shapefile")`	`df.write.save(path, format="shapefile")`

Additionally, Spark DataFrameReader and DataFrameWriter classes provide optional parameters that you can use when reading or writing shapefiles. For a full list of options, see the Spark API reference.

DataFrameReader option	Example	Description
`extent`	`.option("extent", "-90.0, 30.0, 90, 50")`	Filters the shapefile by the spatial extent specified using the format `"<x min>, <y min>, <x max>, <y max>"`.
`mergeSchemas`	`.option("mergeSchemas", True)`	Merge the schemas of a collection of shapefile datasets in the input directory.

DataFrameWriter option	Example	Description
`coalesce`	`.coalesce(1)`	Save data to the number of partitions specified. For example, `.coalesce(1)` will write the data to one shapefile.
`partitionBy`	`.partitionBy("date")`	Partition the output by the given column name. This example will partition the output Parquet files by values in the `date` column.
`overwrite`	`.mode("overwrite")`	Overwrite existing data in the specified path. Other available options are `append`, `error`, and `ignore`.

Usage notes

The default string format of timestamps saved to shapefiles is yyyy-MM-dd HH:mm:ss.SSS.
Shapefile doesn't support saving the generic geometry type. Geometry columns must be point, multipoint, line, or polygon type.
Writing to shapefile requires exactly one geometry field.
bin2d columns will be saved automatically as a long field in the result shapefile, where the field value contains the bin ID.
Spark will read shapefiles from multiple directories if the directory names start with column=. For example, the following example directory contains shapefile data that is partitioned by district. Spark can infer district as a column name in the DataFrame by reading the subdirectory names starting with district=.
When reading in a directory of shapefiles with subdirectories not following the naming convention of column=, Spark won't be able to read all data from subdirectories in bulk, you will need to add the glob pattern at the end of the root path (i.e., C:\shapefile\example\* or C:\shapefile\example\*\*)
Shapefiles have a maximum size of 2GB. Be cautious to use .coalesce(1) when writing large output DataFrame to shapefile format.
Writing to shapefile doesn't require a spatial reference to be set on the geometry column. However, it is recommended to always set and check the spatial reference of a DataFrame before writing to shapefile. Invalid or unavailable spatial reference in shapefile format could potentially lead to inaccurate visualization or analysis results.