Feature services are data sources that are hosted online. This tutorial
shows how to read from and manage feature service datasets. You can create
Spark DataFrames from feature service data sources and use them with any
operations supported on a DataFrame.
In this tutorial you will learn how to access public and protected
feature services. You will create DataFrames from feature services and perform basic queries.
Steps Import and authorize In your notebook, import geoanalytics and authorize the module using a username and password, license file, or token.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import geoanalytics
geoanalytics.auth(username= "user1" , password= "p@ssword" )
Read from a public feature service Read a public feature service into a DataFrame and query for countries where the average population
of the listed cities are greater than 50,000.
Read a feature service containing major USA cities and create a
DataFrame.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
myFS= "https://services.arcgis.com/P3ePLMYs2RVChkJx/ArcGIS/rest/services/World_Cities/FeatureServer/0"
myFSDataFrame = spark.read. format ( 'feature-service' ).load(myFS)
Group the DataFrame by country and find the average
population per country.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
group = myFSDataFrame.selectExpr( "CNTRY_NAME" , "POP" ).groupBy( "CNTRY_NAME" ).avg( "POP" )
Query for countries where the average population of major cities is
greater than 50,000.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
group.where( "avg(POP) > 50000" ).show()
Result
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
+-----------+------------------+
| CNTRY_NAME| avg(POP)|
+-----------+------------------+
| Chad|108714.28571428571|
| Russia| 533285.3092783506|
| Yemen|389769.23076923075|
| Senegal| 207347.7|
| Sweden| 66458.33333333333|
| Kiribati| 63017.0|
|Philippines|2116418.3333333335|
| Malaysia| 310692.3125|
| Singapore| 5703569.0|
| Turkey| 408656.7164179105|
| Malawi| 670284.6666666666|
| Iraq| 817833.3333333334|
| Germany| 660316.9166666666|
|Afghanistan| 142620.6896551724|
| Cambodia|108777.77777777778|
| Jordan| 892820.25|
| Rwanda| 85933.2|
| Sudan| 861142.8571428572|
| France| 279596.6538461539|
| Greece|122750.53846153847|
+-----------+------------------+
only showing top 20 rows
Read from a protected feature service Feature services that are protected can be read by either registering a GIS using the register_ function
or by passing a token. Below is a tutorial to show how to get access to Subscriber content
in ArcGIS Living Atlas of the World with a organizational subscription account .
Register a GIS to get access to a secured layer You can define a GIS name (for example, m), which will be used as a reference when loading feature service layers.
Pass the username and password of your account to log in to ArcGIS Online.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
geoanalytics.register_gis( "myGIS" , "https://arcgis.com" , username= "User" , password= "p@ssw0rd" )
Read in a service using the registered GIS to create a DataFrame. After loading the secured feature service layer into a
DataFrame, the data can be used for further analysis using GeoAnalytics tools and functions.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# Example layer: United States ZIP Code Boundaries 2021
url = r"https://services.arcgis.com/P3ePLMYs2RVChkJx/arcgis/rest/services/USA_Boundaries_2021/FeatureServer/0"
df = spark.read. format ( "feature-service" ) \
.option( "gis" , "myGIS" ) \
.load(url)
Unregister a GIS. You can unregister the GIS after the feature service layer is loaded.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
geoanalytics.unregister_gis( "myGIS" )
Use a token to get access to a secured layer Read a service in with an example token. The URL and token below are for example only and will need
to be updated to reflect your feature service URL and a valid token
from your organization.
Python
Use dark colors for code blocks Copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# Example layer: United States ZIP Code Boundaries 2021
url = r"https://services.arcgis.com/P3ePLMYs2RVChkJx/arcgis/rest/services/USA_Boundaries_2021/FeatureServer/0"
token = 'ABC123deFghIJKlmNOPQrs456'
df = spark.read. format ( 'feature-service' ) \
.option( 'token' , token) \
.load(url)
What's next? Learn about how to read in other data types or analyze your data through SQL functions and analysis tools: