Tutorial: Combine data

Learn how to overlay, merge, join, and dissolve features in feature data with the Map Viewer, ArcGIS APIs, and the spatial analysis service.

Average home value by census block in San Francisco.

Combine analyses allow you to merge, overlay, and join multiple sources of feature data to create new feature data. In most cases, the input feature collections or layers need to contain the same type of geometry.

In this tutorial, you use the Dissovle boundaries, Overlay layers, or Join features operations to find a range of average home values within San Francisco. You can perform the combine analyses either in Map Viewer or programmatically using the ArcGIS Python, ArcGIS REST JS, and ArcGIS REST APIs.

The analyses include:

  • Creating a feature layer containing census blocks for San Francisco.
  • Appending (joining) housing information to the census block feature layer.
  • Creating a new feature layer of census blocks with neighborhood information.
  • Styling the feature data based on average home value.

Prerequisites

Copy the web map

The tutorial web map contains predefined layers to use as the starting point for the analyses outlined in the steps below.

  1. Go to the Combine data tutorial web map and click Sign in.
  2. Sign in to your portal:

  3. Verify that you have the following layers by toggling the visibility on and off:
    • San Francisco Neighborhoods
    • San Francisco Housing Data
  4. Click Save > Save As > Save Map to save a copy.

Create a city boundary layer

When performing an analysis, it is often useful to have a study area layer that you can use to clip features from other layers. Use the Dissolve Boundaries operation on the San Francisco Neighborhoods layer to remove neighborhood attributes and boundaries. The operation will create a single area containing all neighborhoods, excluding Treasure Island and Yerba Buena Island.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the Settings (light) toolbar, click Analysis > Tools > Dissolve Boundaries.

  2. Set the following parameters:

    • Input features: San Francisco Neighborhoods.
    • Ensure Create multipart features is set to false.
  3. Set the Output name to: San Francisco Boundary.

  4. Click Estimate credits. The estimated cost for ArcGIS Location Platform is USD $0.09 and ArcGIS Online is 0.092 credits.

  5. Click Run.

  1. Implement authentication with the following privileges:

    • Spatial Analysis > Feature Analysis
  2. Define the parameters.

  3. Perform the operation.

    Note: This is a long transaction managed with a job request.

  4. Handle the results.


APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119

from arcgis import GIS
from arcgis.features.analysis import dissolve_boundaries

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

neighborhoods = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/SF_Neighborhoods/FeatureServer/0"

results = dissolve_boundaries(
    input_layer=neighborhoods,
    multi_part_features=True,
    # output_name="<OUTPUT NAME>" if you want to save the results as a hosted feature layer.
)

result_features = results.query()

print(f"The resulting layer contains {len(result_features.features)} new records")
# show results in notebook with the map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

The resulting layer is a boundary of the city without any neighborhood attributes.

Get the census blocks feature layer

Census blocks are statistical areas bounded by roads, property lines, and can correspond to city blocks. The blocks will provide a more detailed view of the distribution of home values in San Francisco. Retrieve the U.S. Census Blocks feature layer from Living Atlas to use in the tutorial.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the Contents (dark) toolbar, click Add > Browse layers > Living Atlas and type the following: U.S. Census Blocks, published by Esri_US_Federal_Data.

  2. Select the layer > Add to Map.

  3. Click on the features to display a popup and familiarize yourself with its attributes.

For each block, you should see the block number and its total area in acres.

  1. Implement authentication with the following privileges:

    • Spatial Analysis > Feature Analysis
  2. Define the parameters.

  3. Perform the operation.

    Note: This is a long transaction managed with a job request.

  4. Handle the results.

APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Expand
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis.gis import GIS
portal = GIS()
item = gis.content.get("d795eaa6ee7a40bdb2efeb2d001bf823")
print(item)

Service request

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
GET https://www.arcgis.com/sharing/rest/content/items/d795eaa6ee7a40bdb2efeb2d001bf823?f=pjson HTTP/1.1
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
{
  "id": "d795eaa6ee7a40bdb2efeb2d001bf823",
  "owner": "Esri_US_Federal_Data",
  "orgId": "FiaPA4ga0iQKduv3",
  "created": 1624991014000,
  "modified": 1648737887000,
  "guid": null,
  "name": "US_Census_Blocks_v1",
  "title": "U.S. Census Blocks",
  "type": "Feature Service",
  "typeKeywords": [
    "ArcGIS Server",
    "Data",
    "Feature Access",
    "Feature Service",
    "Service",
    "Singlelayer",
    "Hosted Service",
    "View Service"
  ],
}

Find census blocks within the city boundary

The U.S. Census Blocks hosted layer contains over eight million features. To limit these features to only those within San Francisco, use the Overlay layers operation.

The analysis creates new feature data based upon the input layer (from which you wish to extract features), the overlay layer (defining the area to extract), and a specified overlay operation: intersect, union, or erase. For this overlay analysis, you use the intersect method.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the Settings (light) toolbar, click Analysis > Tools > Overlay layers.

  2. Set the following parameters:

    • Input features: U.S. Census Blocks.
    • Overlayer feature: San Francisco Boundary.
    • Overlay type: Intersect.
    • Output geometry: Polygon.
    • Output name: Census blocks in San Francisco.
  3. Under Environment settings > Processing extent, select Display extent.

  4. Click Estimate credits. The estimated cost for ArcGIS Location Platform is USD $0.66 and ArcGIS Online is 6.6 credits.

  5. Click Run.

  1. Implement authentication with the following privileges:

    • Spatial Analysis > Feature Analysis
  2. Define the parameters.

  3. Perform the operation.

    Note: This is a long transaction managed with a job request.

  4. Handle the results.


APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Expand
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis import GIS
from arcgis.features.analysis import overlay_layers

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

census_blocks = "https://services2.arcgis.com/FiaPA4ga0iQKduv3/arcgis/rest/services/US_Census_Blocks_v1/FeatureServer/0"

neighborhoods = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/SF_Neighborhoods/FeatureServer/0"

process_extent = {
    "extent": {
        "xmin": -13644958.135312138,
        "ymin": 4535593.984072592,
        "xmax": -13611765.355779344,
        "ymax": 4556652.38536513,
        "spatialReference": {"wkid": 102100, "latestWkid": 3857},
    },
}
results = overlay_layers(
    input_layer=census_blocks,
    overlay_layer=neighborhoods,
    overlay_type="intersect",
    output_type="Input",
    context=process_extent,
    # output_name="<OUTPUT NAME>" if you want to save the results as a hosted feature layer.
)

result_features = results.query()

print(
    f"The overlay result layer contains {len(result_features.features)} new records"
)

# show results in map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

The overlay analysis returns feature data containing the census blocks within the boundary of San Francisco you created.

Join housing attributes to census blocks

The census blocks do not contain any attributes with home value data in San Francisco. However, the San Francisco Housing hosted feature layer does. To add housing attributes to the Census blocks within San Francisco hosted feature layer, use the Join Features operation. The operation joins the attributes from the layers based on their spatial relationship.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the Contents (dark) toolbar, click Bookmarks > and choose Study Area to set map extent.

  2. In the Settings (light) toolbar, click Analysis > Tools > Join Features.

  3. Set the following parameters:

    • Target layer: Census blocks within San Francisco.
    • Join layer: San Francisco Housing Data.
    • Use spatial relationship: "true"
      • Select Intersects from the dropdown.
    • Join operation: Join one to one.
    • Output name: Census blocks with data.
    • Environment settings > Processing extent > select Display extent.
  4. Click Estimate credits. The estimate cost for ArcGIS Location Platform is USD $0.11 and ArcGIS Online is 11 credits.

  5. Click Run.

  1. Implement authentication with the following privileges:

    • Spatial Analysis > Feature Analysis
  2. Define the parameters.

  3. Perform the operation.

    Note: This is a long transaction managed with a job request.

  4. Handle the results.


APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Expand
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis import GIS
from arcgis.features.analysis import join_features

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

census_blocks = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/Census_blocks_within_San_Francisco/FeatureServer/0"

housing_data = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/San_Francisco_Housing_Data/FeatureServer/0"

results = join_features(
    target_layer=census_blocks,
    join_layer=housing_data,
    join_operation="joinOneToOne",
    spatial_relationship="intersects",
    join_type="inner"
    # output_name="<OUTPUT NAME>" if you want to save the results as a hosted feature layer.
)

result_features = results.query()

print(
    f"The join features result layer contains {len(result_features.features)} new records"
)
# show results in notebook with the map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

The resulting layer contains demographic information for each block, including the average home value.

Add neighborhood names

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs

Find census blocks within neighborhoods

The Census blocks with data feature layer contains all census blocks within the San Francisco boundary along with relevant housing attributes. But, the features within the layer do not contain the neighborhood name corresponding to each census block. Use the Overlay layers operation to create new feature data that contain housing and neighborhood attributes.

  1. In the Settings (light) toolbar, click Analysis > Tools > Overlay layers.

  2. Set the following parameters:

    • Input features: San Francisco Neighborhoods.
    • Overlay features: Census blocks with data.
    • Overlay type: Intersect.
    • Output geometry: polygon.
    • Output name: Census blocks with neighborhood names.
  3. Click Estimate credits. The estimated cost for ArcGIS Location Platform is USD $0.009 and ArcGIS Online is 0.093 credits.

  4. Click Run.

  5. Click on a feature to see the neighborhood associated with each census block.

Style the layer

The Census blocks within neighborhoods layer contains the neighborhood name, average home value, its block group, and its block. The default style is set to display only the block locations. To make the feature layer styles more meaningful, use the renderer to show the range of average home values in the city.

  1. In the Contents (dark) toolbar, click Layers, then select the Census blocks with neighborhood names layer.

  2. In the Settings (light) toolbar, click Styles.

  3. Click + Field and choose AVGVAL_CY.

  4. Select Counts and Amounts (Color) > Style options.

  5. Check Classify Data > Manual Break with 5 classes. The Manual Break setting allows you to exclude homes with a value of 0 (outliers) when styling the map.

  6. Click on the scale to set the class breaks based on value. For example, set the lowest value to 200,000 and the highest value to 2,250,000.

  7. Click Symbol style to select a yellow to red color ramp.

  8. Click Done twice to exit out of the Styles pane.

  9. In the Content (dark) toolbar, click Save and open > Save to save your web map.

Find census blocks within neighborhoods

  1. Implement authentication with the following privileges:

    • Spatial Analysis > Feature Analysis
  2. Define the parameters.

  3. Perform the operation.

    Note: This is a long transaction managed with a job request.

  4. Handle the results.


APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis import GIS
from arcgis.features.analysis import overlay_layers

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

census_blocks = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/Census_blocks_with_data/FeatureServer/0"

neighborhoods = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/SF_Neighborhoods/FeatureServer/0"

results = overlay_layers(
    input_layer=census_blocks,
    overlay_layer=neighborhoods,
    overlay_type="intersect",
    output_type="Input",
)

result_features = results.query()

print(
    f"The overlay result layer contains {len(result_features.features)} new records"
)

# show results in map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

Style the layer

To learn how to style a feature layer, go to Visualization.

Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
{
  "renderer": {
    "authoringInfo": {
      "type": "classedColor"
    },
    "type": "classBreaks",
    "field": "AVGVAL_CY",
    "minValue": 200000,
    "classBreakInfos": [
      {

The resulting layer should look something like this.

What's next?

You performed a series of combine analyses to find the average home value by census block. Your web map should look something like this.

Learn how to use additional tools, APIs, and location services in these tutorials:

Summarize data

Aggregate and summarize features using summarize analysis operations.


Discover patterns in data

Find patterns and trends in data using spatial analysis operations.


Find and extract data

Find data with attribute and spatial queries using find analysis operations.


Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.