Data Summarization - Construction permits, part 2/2

In the "Explore and analyze construction permits" notebook, we explored your data and learned a little about the spatial and temporal trends of permit activity in Montgomery County. In this lesson, we'll move beyond exploration and run spatial analysis tools to answer specific questions that can't be answered by the data itself. In particular, we want to know why permits spiked in Germantown in 2011 and predict where future permit spikes - and, by extension, future growth - are likely to occur.

First, we'll aggregate the points by ZIP Code. We'll enrich each ZIP Code with demographic information and learn more about the demographic conditions that led to such rapid growth in such a short time. Once you determine why growth occurred where and when it did, we'll locate other ZIP Codes with similar demographic characteristics to predict future growth.

Aggregate points

from arcgis import GIS
gis = GIS('home')
data = gis.content.search("Commercial_Permits_since_2010 owner:api_data_owner",
                          'Feature layer',
                           outside_org=True)
data[0]
Commercial_Permits_since_2010
test dataFeature Layer Collection by api_data_owner
Last Modified: July 01, 2019
0 comments, 1 views
permits = data[0]
permit_layer = permits.layers[0]
zip_code = gis.content.search('title:ZIP Code Boundaries 2017 owner:esri_dm', 'Feature layer',
                           outside_org=True)
zip_code[0]
United States ZIP Code Boundaries 2017
This layer shows the ZIP Code level boundaries of United States in 2017. The boundaries are optimized to improve Data Enrichment analysis performance.Feature Layer Collection by esri_dm
Last Modified: June 21, 2019
0 comments, 69,632 views
zip_item = zip_code[0]

The USA_ZIP_Code layer is added as a new item. Since the item is a feature layer collection, using the layers property will give us a list of layers.

for lyr in zip_item.layers:
    print(lyr.properties.name)
USA_Country
USA_State
USA_County
USA_ZipCode
USA_Tract
USA_BlockGroup
zip_code_layer = zip_item.layers[3]

Next, you'll use this layer to aggregate permit points. By default, the parameters are set to use the ZIP Codes as the area layer, the permits as the layer to be aggregated, and the layer style to be based on permit count. These parameters are exactly what you want.

from arcgis.features.summarize_data import aggregate_points
from datetime import datetime as dt
permit_agg_by_zip = aggregate_points(permit_layer, zip_code_layer, 
                                     keep_boundaries_with_no_points=False,
                                     output_name='zipcode_aggregate' + str(dt.now().microsecond))
permit_agg_by_zip
zipcode_aggregate760079
Feature Layer Collection by arcgis_python
Last Modified: April 13, 2020
0 comments, 1 views

Aggregation results

agg_map = gis.map('Maryland')
agg_map
agg_map.add_layer(permit_agg_by_zip)

The new layer looks like a point layer, but it's actually a polygon layer with a point symbology. Each point represents the number of permits per ZIP Code area. Larger points indicate ZIP Codes with more permits.

import pandas as pd
sdf = pd.DataFrame.spatial.from_layer(permit_agg_by_zip.layers[0])
sdf.head(10)
AnalysisAreaOBJECTIDPOPULATIONPO_NAMEPoint_CountSHAPESQMISTATEZIP_CODE
02.356814114652Washington6{"rings": [[[-77.0266270001359, 38.98455799965...2.36DC20012
16.248092248592Hyattsville1{"rings": [[[-76.9414389999099, 39.02912599996...6.26MD20783
20.1910613219Glen Echo1{"rings": [[[-77.1384300001444, 38.96841399980...0.19MD20812
35.168331430017Bethesda1145{"rings": [[[-77.0943629995527, 39.02250799964...5.17MD20814
45.360828530001Chevy Chase586{"rings": [[[-77.0635971995511, 39.01197539974...5.35MD20815
54.607703616967Bethesda154{"rings": [[[-77.1429960002652, 38.97162000016...4.61MD20816
613.889993738385Bethesda732{"rings": [[[-77.1267290001432, 39.02947299977...13.89MD20817
70.97853681383Cabin John13{"rings": [[[-77.1573989998639, 38.98250600035...0.98MD20818
89.426954926858Olney216{"rings": [[[-77.0921479999302, 39.16957599993...9.43MD20832
922.871336108380Brookeville40{"rings": [[[-77.0616859999089, 39.27760500037...22.87MD20833
sdf.reset_index(inplace=True)
sdf.head()
indexAnalysisAreaOBJECTIDPOPULATIONPO_NAMEPoint_CountSHAPESQMISTATEZIP_CODE
002.356814114652Washington6{"rings": [[[-77.0266270001359, 38.98455799965...2.36DC20012
116.248092248592Hyattsville1{"rings": [[[-76.9414389999099, 39.02912599996...6.26MD20783
220.1910613219Glen Echo1{"rings": [[[-77.1384300001444, 38.96841399980...0.19MD20812
335.168331430017Bethesda1145{"rings": [[[-77.0943629995527, 39.02250799964...5.17MD20814
445.360828530001Chevy Chase586{"rings": [[[-77.0635971995511, 39.01197539974...5.35MD20815

Review some basic statistics about the data.

sdf['Point_Count'].mean()
249.42222222222222
sdf['Point_Count'].max()
1145
sdf['Point_Count'].min()
1
agg_layer = permit_agg_by_zip.layers[0]

Although most of the large point symbols on the map are in the southeast corner, near Washington, D.C., there are a few large points in the northwest. In particular, there is a very large circle in the ZIP Code located in Clarksburg. (If you're using different ZIP Code data, this area may be identified as ZIP Code 20871 instead.) The ZIP code has 948 permits. Additionally, this area geographically corresponds to the hot spot you identified in the previous lesson. This ZIP Code is one that you'll focus on when you enrich your layer with demographic data.

Enrich the data

Are there demographic characteristics about the Clarksburg ZIP Code that contributed to its high growth? If so, are there other areas with those characteristics that may experience growth in the future? To answer these questions, you'll use the Enrich Data analysis tool. This tool adds demographic attributes of your choice to your data. Specifically, you'll add Tapestry information to each ZIP Code. Tapestry is a summary of many demographic and socioeconomic variables, including age groups and lifestyle choices. It'll teach you more about the types of people who live in your area of interest and help you better understand the reasons why growth happened where it did.

from arcgis.features.enrich_data import enrich_layer
enrich_aggregate = enrich_layer(agg_layer, 
                                analysis_variables=["AtRisk.TSEGNAME"],
                                output_name="added_tapestry_var" + str(dt.now().microsecond))
enrich_aggregate
added_tapestry_var285913
Feature Layer Collection by arcgis_python
Last Modified: April 13, 2020
0 comments, 0 views
agg_lyr = enrich_aggregate.layers[0]
sdf = pd.DataFrame.spatial.from_layer(agg_lyr)
sdf.head()
AnalysisAreaENRICH_FIDHasDataIDOBJECTIDPOPULATIONPO_NAMEPoint_CountSHAPESQMISTATETSEGNAMEZIP_CODEaggregationMethodapportionmentConfidencepopulationToPolygonSizeRatingsourceCountry
02.356814110114652Washington6{"rings": [[[-77.026627, 38.9845580000001], [-...2.36DCCity Lights20012BlockApportionment:US.BlockGroups2.5762.191US
16.248092211248592Hyattsville1{"rings": [[[-76.9414389999999, 39.02912600000...6.26MDNeWest Residents20783BlockApportionment:US.BlockGroups2.5762.191US
20.1910613123219Glen Echo1{"rings": [[[-77.13843, 38.9684140000001], [-7...0.19MDUrban Chic20812BlockApportionment:US.BlockGroups2.5762.191US
35.168331413430017Bethesda1145{"rings": [[[-77.094363, 39.0225080000001], [-...5.17MDMetro Renters20814BlockApportionment:US.BlockGroups2.5762.191US
45.360828514530001Chevy Chase586{"rings": [[[-77.0635971999999, 39.0119754], [...5.35MDTop Tier20815BlockApportionment:US.BlockGroups2.5762.191US
enrich_aggregate_map = gis.map('Maryland')
enrich_aggregate_map
sdf.spatial.plot(kind='map', map_widget=enrich_aggregate_map,
        renderer_type='u',
        col='TSEGNAME') 
True

Click some of the ZIP Codes.

The Tapestry segment is displayed when you click a ZIP Code. The Tapestry segments have names such as Enterprising Professionals and Savvy Suburbanites. You can look up more information about each segment, including its specific demographic characteristics, on the Tapestry Segmentation help page.

What Tapestry segment is dominant for the Clarksburg ZIP Code where major growth occurred? Click the Clarksburg ZIP Code to find out. According to the pop-up, Boomburbs is the dominant Tapestry segment for the ZIP Code. Boomburbs have many young professionals with families living in affordable new housing. This description may explain why the area saw such rapid residential growth in 2011. It's possible that other ZIP Codes with similar demographic profiles may experience rapid growth in the near future.

Click the ZIP Code directly southwest of Clarksburg.

This ZIP Code is in Boyds. It also has the Boomburbs Tapestry segment. However, its number of permits has been relatively low since 2010. The county may be able to anticipate a similar spike in permit activity in this area.

Although Tapestry segments are based on several demographic characteristics, you could also perform this analysis with other variables. For instance, you could determine if there is a correlation between high permit activity and high population growth. Is a young population or a high income level a stronger indicator of growth? You can answer these questions and others with the analysis tools at your disposal. For the purposes of this lesson, however, your results are satisfactory.

enrich_aggregate_map.add_layer(agg_lyr, {'renderer':'ClassedSizeRenderer',
                                             'field_name':'POPULATION',
                                             'opacity':0.75})

Share your work

We've analyzed your data and come to a couple conclusions about your data. Next, we'll share your results online. Currently, our result layers are layers that are accessible only to you. Sharing our data will make it easier for county officials to use your data in other ArcGIS applications and communicate key information to the public. In particular, we'll share your work to ArcGIS Online. We'll share your enriched ZIP Codes dataset as feature layers that can be added to any web map.

The layer contains fields for both the count of permits per ZIP Code and the dominant Tapestry segment—basically all of the result data we created in your analysis. We'll only need to share this layer, not the original aggregation layer.

Using the share() method you can share your work with others.

enrich_aggregate.share(everyone=True)
{'notSharedWith': [], 'itemId': '0ad0439ef9394afd96ebb6d16e3653b2'}

In this notebook, we used ArcGIS API for Python to explore and analyze permit data for Montgomery County, Maryland. You answered questions about your data's spatial and temporal trends and located areas of the county with rapid growth. We compared your findings with demographic data, came to conclusions about the possible causes of growth, and even predicted an area that may experience similar growth in the future based on shared demographic characteristics. With ArcGIS API for Python, we can perform a similar workflow on any of your data to better understand what it contains and what questions it can answer.

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.