Combine data

Learn how to overlay, merge, join, and dissolve features in feature data with the Map Viewer, ArcGIS APIs, and the spatial analysis service.

Average home value by census block in San Francisco.

Combine analyses allow you to merge, overlay, and join multiple sources of feature data to create new feature data. In most cases, the input feature collections or layers need to contain the same type of geometry.

In this tutorial, you use the Dissovle boundaries, Overlay layers, or Join features operations to find a range of average home values within San Francisco. You can perform the combine analyses either in Map Viewer or programmatically using the ArcGIS Python, ArcGIS REST JS, and ArcGIS REST APIs.

The analyses include:

  • Creating a feature layer containing census blocks for San Francisco.
  • Appending (joining) housing information to the census block feature layer.
  • Creating a new feature layer of census blocks with neighborhood information.
  • Styling the feature data based on average home value.

Prerequisites

Copy the web map

The tutorial web map contains predefined layers to use as the starting point for the analyses outlined in the steps below.

  1. Go to the Combine data tutorial web map and click Sign in.

  2. Verify that you have the following layers by toggling the visibility on and off:

    • San Francisco Neighborhoods
    • San Francisco Housing Data
  3. Click Save > Save As > Save Map to save a copy.

Create a city boundary layer

When performing an analysis, it is often useful to have a study area layer that you can use to clip features from other layers. Use the Dissolve Boundaries operation on the San Francisco Neighborhoods layer to remove neighborhood attributes and boundaries. The operation will create a single area containing all neighborhoods, excluding Treasure Island and Yerba Buena Island.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the top bar, click Analysis > Manage Data > Dissolve Boundaries.

  2. Set the following parameters:

    • 1. Choose an area layer whose boundaries will be dissolved: SF Neighborhoods.
    • 2. Choose dissolve method : Areas that overlap or are adjacent.
  3. Set the Result layer name to: San Francisco Boundary.

  4. Click Show credits. The analysis will consume USD $0.0092 (.092 credits).

  5. Click Run Analysis.

  1. Implement user authentication to access the spatial analysis service.
  2. Define the parameters of the request.
  3. Execute the operation. Note: This is a long transaction managed with a job request.
  4. Handle the results.

APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119

from arcgis import GIS
from arcgis.features.analysis import dissolve_boundaries

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

neighborhoods = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/SF_Neighborhoods/FeatureServer/0"

results = dissolve_boundaries(
    input_layer=neighborhoods,
    multi_part_features=True,
    # output_name="<OUTPUT NAME>" if you want to save the results as a hosted feature layer.
)

result_features = results.query()

print(f"The resulting layer contains {len(result_features.features)} new records")
# show results in notebook with the map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

The resulting layer is a boundary of the city without any neighborhood attributes.

Get the census blocks feature layer

Census blocks are statistical areas bounded by roads, property lines, and can correspond to city blocks. The blocks will provide a more detailed view of the distribution of home values in San Francisco. Retrieve the U.S. Census Blocks feature layer from Living Atlas to use in the tutorial.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the top bar, click Add > Browse Living Atlas Layers and type the following: U.S. Census Blocks, published by Esri_US_Federal_Data.

  2. Select the layer > Add to Map.

  3. Click on the features to display a popup and familiarize yourself with its attributes.

For each block, you should see the block number and its area for both land and water in square meters.

  1. Implement user authentication to access the spatial analysis service.
  2. Define the parameters of the request.
  3. Execute the operation. Note: This is a long transaction managed with a job request.
  4. Handle the results.

APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Expand
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis.gis import GIS
portal = GIS()
item = gis.content.get("d795eaa6ee7a40bdb2efeb2d001bf823")
print(item)

Service request

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
GET https://www.arcgis.com/sharing/rest/content/items/d795eaa6ee7a40bdb2efeb2d001bf823?f=pjson HTTP/1.1
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
{
  "id": "d795eaa6ee7a40bdb2efeb2d001bf823",
  "owner": "Esri_US_Federal_Data",
  "orgId": "FiaPA4ga0iQKduv3",
  "created": 1624991014000,
  "modified": 1648737887000,
  "guid": null,
  "name": "US_Census_Blocks_v1",
  "title": "U.S. Census Blocks",
  "type": "Feature Service",
  "typeKeywords": [
    "ArcGIS Server",
    "Data",
    "Feature Access",
    "Feature Service",
    "Service",
    "Singlelayer",
    "Hosted Service",
    "View Service"
  ],
}

Find census blocks within the city boundary

The U.S. Census Blocks hosted layer contains over eight million features. To limit these features to only those within San Francisco, use the Overlay layers operation.

The analysis creates new feature data based upon the input layer (from which you wish to extract features), the overlay layer (defining the area to extract), and a specified overlay operation: intersect, union, or erase. For this overlay analysis, you use the intersect method.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the top bar, click Analysis > Manage Data > Overlay layers.

  2. Set the following parameters:

    • Choose input layer: U.S. Census Blocks.
    • Choose overlay layer: San Francisco Boundary.
    • Choose overlay method: Intersect.
    • Output: Areas.
    • Result layer name: Census blocks in San Francisco.
  3. Check Use current map extent.

  4. Click Show credits. The analysis will consume approximately USD $.7 (7 credits).

  5. Click Run Analysis.

  1. Implement user authentication to access the spatial analysis service.
  2. Define the parameters of the request.
  3. Execute the operation. Note: This is a long transaction managed with a job request.
  4. Handle the results.

APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Expand
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis import GIS
from arcgis.features.analysis import overlay_layers

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

census_blocks = "https://services2.arcgis.com/FiaPA4ga0iQKduv3/arcgis/rest/services/US_Census_Blocks_v1/FeatureServer/0"

neighborhoods = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/SF_Neighborhoods/FeatureServer/0"

process_extent = {
    "extent": {
        "xmin": -13644958.135312138,
        "ymin": 4535593.984072592,
        "xmax": -13611765.355779344,
        "ymax": 4556652.38536513,
        "spatialReference": {"wkid": 102100, "latestWkid": 3857},
    },
}
results = overlay_layers(
    input_layer=census_blocks,
    overlay_layer=neighborhoods,
    overlay_type="intersect",
    output_type="Input",
    context=process_extent,
    # output_name="<OUTPUT NAME>" if you want to save the results as a hosted feature layer.
)

result_features = results.query()

print(
    f"The overlay result layer contains {len(result_features.features)} new records"
)

# show results in map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

The overlay analysis returns feature data containing the census blocks within the boundary of San Francisco you created.

Join housing attributes to census blocks

The census blocks do not contain any attributes with home value data in San Francisco. However, the San Francisco Housing hosted feature layer does. To add housing attributes to the Census blocks within San Francisco hosted feature layer, use the Join Features operation. The operation joins the attributes from the layers based on their spatial relationship.

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs
  1. In the top bar, click Analysis > Summarize Data > Join Features.

  2. Set the following parameters:

    • Choose target layer: Census blocks within San Francisco.
    • Choose layer to join to target layer: San Francisco Housing.
  3. Click Choose a spatial relationship.

  4. Select Intersects from the dropdown.

  5. Select Join one to one.

  6. Set the Result layer name to: Census blocks with data.

  7. In the top bar, click Bookmarks > and choose Study Area to set map extent.

  8. Check 'Use current map extent`.

  9. Click Show credits. The analysis will consume approximately USD $.11 (11 credits).

  10. Click Run Analysis.

  1. Implement user authentication to access the spatial analysis service.
  2. Define the parameters of the request.
  3. Execute the operation. Note: This is a long transaction managed with a job request.
  4. Handle the results.

APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Expand
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis import GIS
from arcgis.features.analysis import join_features

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

census_blocks = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/Census_blocks_within_San_Francisco/FeatureServer/0"

housing_data = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/San_Francisco_Housing_Data/FeatureServer/0"

results = join_features(
    target_layer=census_blocks,
    join_layer=housing_data,
    join_operation="joinOneToOne",
    spatial_relationship="intersects",
    join_type="inner"
    # output_name="<OUTPUT NAME>" if you want to save the results as a hosted feature layer.
)

result_features = results.query()

print(
    f"The join features result layer contains {len(result_features.features)} new records"
)
# show results in notebook with the map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

The resulting layer contains demographic information for each block, including the average home value.

Add neighborhood names

Steps to use the Map ViewerSteps to use ArcGIS Python, REST JS, and REST APIs

Find census blocks within neighborhoods

The Census blocks with data feature layer contains all census blocks within the San Francisco boundary along with relevant housing attributes. But, the features within the layer do not contain the neighborhood name corresponding to each census block. Use the Overlay layers operation to create new feature data that contain housing and neighborhood attributes.

  1. In the top bar, click Analysis > Manage Data > Overlay layers.

  2. Set the following parameters:

    • Choose input layer: San Francisco Neighborhoods.
    • Choose overlay layer: Census blocks with data.
    • Choose overlay method: Intersect.
    • Output: Areas.
    • Result layer name: Census blocks with neighborhood names.
  3. Click Show credits. The analysis will consume approximately USD $0.6 (6 credits).

  4. Click Run Analysis.

  5. Click on a feature to see the neighborhood associated with each census block.

Style the layer

The Census blocks within neighborhoods layer contains the neighborhood name, average home value, its block group, and its block. The default style is set to display only the block locations. To make the feature layer styles more meaningful, use the renderer to show the range of average home values in the city.

  1. In Content, select Census blocks with neighborhood names > Change Style.

    Change style
  2. Choose Average home value

  3. Select Counts and Amounts (Color) > Options .

  4. Check Classify Data > Manual Break with 5 classes. The Manual Break setting allows you to exclude homes with a value of 0 (outliers) when styling the map.

  5. Click on the scale to set the class breaks based on value. For example, set the lowest value to 200,000 and the highest value to 2,250,000.

  6. Click Symbols to select a yellow to red color ramp. Click Ok when done.

    Style
  7. Click OK > Done to set the class breaks and styling.

  8. In the top bar click Save to save your web map.

Find census blocks within neighborhoods

  1. Implement user authentication to access the spatial analysis service.
  2. Define the parameters of the request.
  3. Execute the operation. Note: This is a long transaction managed with a job request.
  4. Handle the results.

APIs

ArcGIS API for PythonArcGIS API for PythonArcGIS REST JS
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
from arcgis import GIS
from arcgis.features.analysis import overlay_layers

portal = GIS(username="<USERNAME>", password="<PASSWORD>")

census_blocks = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/Census_blocks_with_data/FeatureServer/0"

neighborhoods = "https://services3.arcgis.com/GVgbJbqm8hXASVYi/arcgis/rest/services/SF_Neighborhoods/FeatureServer/0"

results = overlay_layers(
    input_layer=census_blocks,
    overlay_layer=neighborhoods,
    overlay_type="intersect",
    output_type="Input",
)

result_features = results.query()

print(
    f"The overlay result layer contains {len(result_features.features)} new records"
)

# show results in map widget
map_widget = portal.map("San Francisco, CA")
map_widget.add_layer(result_features)
map_widget

Service requests

Request
HTTPHTTPcURL
Use dark colors for code blocksCopy
1
2
3
4
5
POST arcgis.com/sharing/rest/portals/self HTTP/1.1
Content-Type: application/x-www-form-urlencoded

&f=json
&token=<ACCESS_TOKEN>
Response (JSON)
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
{
  "helperServices": {
    // Other parameters
    "analysis": {
      "url": "https://<YOUR_ANALYSIS_SERVICE>/arcgis/rest/services/tasks/GPServer"
    },
    "geoenrichment": {
      "url": "https://geoenrich.arcgis.com/arcgis/rest/services/World/GeoenrichmentServer"
    }
  }
}

Style the layer

To learn how to style a feature layer, go to Visualization.

Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
{
  "renderer": {
    "authoringInfo": {
      "type": "classedColor"
    },
    "type": "classBreaks",
    "field": "AVGVAL_CY",
    "minValue": 200000,
    "classBreakInfos": [
      {

The resulting layer should look something like this.

What's next?

You performed a series of combine analyses to find the average home value by census block. Your web map should look something like this.

Learn how to use additional tools, APIs, and location services in these tutorials:

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.